Good point Dan, if the record has simply been corrupted in transit,
throwing it out is the best thing to do. As long as the writer has not got
a bug in their program, in which case they should fix it and regenerate the
records.
On Fri, May 29, 2015 at 8:57 AM, Dan Swank - NOAA Affiliate <
dan.swank@xxxxxxxx> wrote:
> John, I'm sure you may have considered this already,
>
> From my point of view, dealing with GRIB for 12+ years:
> when a case where verifying a GRIB record length (V1 or 2) and
> you do not immediately get exactly 7777 in the expected
> location of the terminator ~ the record* must *be thrown out.
>
> Indeed Some of the record may be salvagable, but there is
> just no way to knowing where the corruption landed inside
> the record. In worst cases, it can hit the record length indicator
> itself.
> Salvaging is best left to a subjective human armed with a hex editor
> and the GRIB specifcation document.
>
> When the terminator is not found at the expected end of the message,
> discard the Section 0 and continue
> reading the file ~ byte by byte, for the next "GRIB" indicator.
>
> Repeat.
>
> -Dan
>
>
>
>
> On Wed, May 27, 2015 at 1:11 PM, Daniele Romagnoli <
> daniele.romagnoli@xxxxxxxxxxxxxxxx> wrote:
>
>> Thanks John for your investigation. Really appreciated.
>> I'll report your feedbacks to the user we provided me the dataset to
>> notify him about the issue.
>> I'll let you know.
>>
>> Best Regards,
>> Daniele
>>
>>
>>
>>
>>
>> On Wed, May 27, 2015 at 6:38 PM, John Caron <caron@xxxxxxxx> wrote:
>>
>>> Hi Daniele:
>>>
>>> So the problem with that file is that the GRIB-2 indicator section
>>> "Total length of GRIB message in octets (including Section 0)" is wrong
>>> for all records.
>>>
>>> We were tolerating that before a fix on 5/19/2014. The records seem to
>>> read correctly, so its hard to say if the length is wrong or if garbage got
>>> into the record before the ending 7777.
>>>
>>> Im not sure if we should allow lengths to be wrong - tricky to handle
>>> all possible cases of corrupt messages and salvage what can be done.
>>>
>>> Perhaps first thing to do is notify the writer, and see what they have
>>> to say?
>>>
>>> Grib2IdentificationSection
>>> Center = (74) UK Meteorological Office - Exeter (RSMC)
>>> SubCenter = (5) ESA ERS Central Facility
>>> Master Table = 6
>>> Local Table = 1
>>>
>>> What do you think?
>>>
>>> John
>>>
>>>
>>> On Wed, May 27, 2015 at 6:41 AM, Daniele Romagnoli <
>>> daniele.romagnoli@xxxxxxxxxxxxxxxx> wrote:
>>>
>>>> Hi,
>>>> I have created a new ticket related to latest NetCDFJava versions being
>>>> unable to open a GRIB file which was previously working on 4.3.x:
>>>>
>>>> In attachment, a sample file to check the issue.
>>>> I think we can discuss further there:
>>>> https://github.com/Unidata/thredds/issues/135
>>>>
>>>> Cheers,
>>>> Daniele
>>>>
>>>>
>>>> --
>>>> ==
>>>> Meet us at the INSPIRE Conference in Lisbon 25-29 May 2015! Visit
>>>> http://goo.gl/WHKDXT for more information.
>>>> ==
>>>>
>>>> Ing. Daniele Romagnoli
>>>> Senior Software Engineer
>>>>
>>>> GeoSolutions S.A.S.
>>>> Via Poggio alle Viti 1187
>>>> 55054 Massarosa (LU)
>>>> Italy
>>>> phone: +39 0584 962313
>>>> fax: +39 0584 1660272
>>>>
>>>> http://www.geo-solutions.it
>>>> http://twitter.com/geosolutions_it
>>>>
>>>> -------------------------------------------------------
>>>>
>>>> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>>>>
>>>> Le informazioni contenute in questo messaggio di posta elettronica e/o
>>>> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
>>>> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
>>>> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
>>>> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
>>>> darcene notizia via e-mail e di procedere alla distruzione del messaggio
>>>> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
>>>> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
>>>> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
>>>> principi dettati dal D.Lgs. 196/2003.
>>>>
>>>>
>>>>
>>>> The information in this message and/or attachments, is intended solely
>>>> for the attention and use of the named addressee(s) and may be confidential
>>>> or proprietary in nature or covered by the provisions of privacy act
>>>> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
>>>> Code).Any use not in accord with its purpose, any disclosure, reproduction,
>>>> copying, distribution, or either dissemination, either whole or partial, is
>>>> strictly forbidden except previous formal approval of the named
>>>> addressee(s). If you are not the intended recipient, please contact
>>>> immediately the sender by telephone, fax or e-mail and delete the
>>>> information in this message that has been received in error. The sender
>>>> does not give any warranty or accept liability as the content, accuracy or
>>>> completeness of sent messages and accepts no responsibility for changes
>>>> made after they were sent or for other risks which arise as a result of
>>>> e-mail transmission, viruses, etc.
>>>>
>>>>
>>>> _______________________________________________
>>>> netcdf-java mailing list
>>>> netcdf-java@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe, visit:
>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>
>>>
>>>
>>
>>
>> --
>> ==
>> Meet us at the INSPIRE Conference in Lisbon 25-29 May 2015! Visit
>> http://goo.gl/WHKDXT for more information.
>> ==
>>
>> Ing. Daniele Romagnoli
>> Senior Software Engineer
>>
>> GeoSolutions S.A.S.
>> Via Poggio alle Viti 1187
>> 55054 Massarosa (LU)
>> Italy
>> phone: +39 0584 962313
>> fax: +39 0584 1660272
>>
>> http://www.geo-solutions.it
>> http://twitter.com/geosolutions_it
>>
>> -------------------------------------------------------
>>
>> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>>
>> Le informazioni contenute in questo messaggio di posta elettronica e/o
>> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
>> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
>> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
>> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
>> darcene notizia via e-mail e di procedere alla distruzione del messaggio
>> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
>> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
>> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
>> principi dettati dal D.Lgs. 196/2003.
>>
>>
>>
>> The information in this message and/or attachments, is intended solely
>> for the attention and use of the named addressee(s) and may be confidential
>> or proprietary in nature or covered by the provisions of privacy act
>> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
>> Code).Any use not in accord with its purpose, any disclosure, reproduction,
>> copying, distribution, or either dissemination, either whole or partial, is
>> strictly forbidden except previous formal approval of the named
>> addressee(s). If you are not the intended recipient, please contact
>> immediately the sender by telephone, fax or e-mail and delete the
>> information in this message that has been received in error. The sender
>> does not give any warranty or accept liability as the content, accuracy or
>> completeness of sent messages and accepts no responsibility for changes
>> made after they were sent or for other risks which arise as a result of
>> e-mail transmission, viruses, etc.
>>
>>
>> _______________________________________________
>> thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
>
> --
>
> Dan Swank
> NOMADS System Support, STG Inc. - Federal Government Contractor
> NOAA's National Centers for Environmental Information (NCEI)
> Data Stewardship Devision (DSD)
> 151 Patton Ave
> Asheville, NC 28801
> dan.swank@xxxxxxxx
> Phone: 828-271-4007
> Customer Support: 828-271-4800 or http://www.ncdc.noaa.gov/contact
>
> The newly formed NCEI merges the National Climatic Data Center (NCDC), the
> National Geophysical Data Center (NGDC), and the National Oceanographic
> Data Center (NODC).
>
> Facebook: http://www.facebook.com/NOAANationalClimaticDataCenter Twitter:
> @NOAANCDC @NOAAOceanData
>
> ======================= Disclaimer ===========
> I am not a Federal Employee.
> Any opinions expressed in this message are mine personally and
> do not represent official positions from STG Inc, ERT, NCEI, or NOAA
> ============================================
>
