All – thank you for the quick responses, after some testing we confirmed using
wgrib2 v3.1.1 the temporary workaround was successful for both NAWIPS and AWIPS
EDEX.
Pete
From: Ryan Hickman <ryan@xxxxxxxxxxxxxxxx>
Sent: Thursday, June 29, 2023 1:58 PM
To: Shay Carter <srcarter@xxxxxxxx>
Cc: Manousos, Peter C <pmanousos@xxxxxxxxxxxxxxxxxxx>;
awips2-users@xxxxxxxxxxxxxxxx
Subject: [EXTERNAL] Re: [awips2-users] ECMWF grib2 files from WMO Essential no
longer being decoded properly by NAWIPS or AWIPS
External sender, use caution with links/attachments. Click 'Report Message' in
Outlook if suspicious.
ECMWF Cycle 48r1 went live when these issues began to occur. Relevant links to
the compression methods are below.
https://www.ecmwf.int/en/elibrary/81320-impact-grib-compression-weather-forecast-data-and-data-handling-applications
https://confluence.ecmwf.int/display/FCST/Implementation+of+IFS+Cycle+48r1
https://public.ccsds.org/Pubs/121x0b3.pdf
On Thu, Jun 29, 2023 at 11:43 AM Shay Carter
<srcarter@xxxxxxxx<mailto:srcarter@xxxxxxxx>> wrote:
Hi Pete,
Thanks for the heads up!
One of my coworkers filled me in with this pertinent information:
A couple of days ago the ECMWF Open Data changed (grib2 compression type) and
most software is having problems reading it now. Wgrib2 can convert it to a
different compression which other software can then read, so there is a
work-around.
So, in order to get ECMWF data back into your EDEX, I think you just need to
change the compression which can be done with wgrib2. The same coworker linked
this post<http://gradsusr.org/pipermail/gradsusr/2023-June/043431.html> about
the issue, with the relevant code being something along the lines of this:
wgrib2 IN.grb -set_grib_type c3 -grib_out OUT.grb*
You'll need to change compression before inserting it into the EDEX queue.
Let me know if this makes sense, if you're able to test it out, and how it goes?
Thanks!
--Shay
On Thu, Jun 29, 2023 at 10:29 AM Manousos, Peter C via awips2-users
<awips2-users@xxxxxxxxxxxxxxxx<mailto:awips2-users@xxxxxxxxxxxxxxxx>> wrote:
(apologies if you are both on the GEMBUD and AWIPS email lists as this will be
a duplicate…)
Greetings. As of 06 28 2023 12z it seems the ECMWF’s freely available WMO
Essential grib2 data has changed
(here<https://data.ecmwf.int/forecasts/20230628/12z/0p4-beta/oper/>). It is
no longer fully processable by dcgrib2 or AWIPS EDEX
We did not see any change notice on ECMWF web site or Google and were wondering
if anyone else is experiencing the following (attached is a one-field <500 mb
HGT> grib2 file), and if you did would you happen to know the cause and any
workarounds?
wgrib2 -v -V test.grb2 shows what looks like a viable grib2 file
1:0:vt=2023062803:500 mb:3 hour fcst:HGT Geopotential Height [gpm]:
ndata=405900:undef=0:mean=5562.91:min=4689.51:max=5955.01
grid_template=0:winds(N/S):
lat-lon grid:(900 x 451) units 1e-06 input WE:NS output WE:SN res 48
lat 90.000000 to -90.000000 by 0.400000
lon 180.000000 to 179.600000 by 0.400000 #points=405900
basic_ang=0 sub_angle=1000000 units=0.000001
unscaled lat=90000000 to -90000000 lon=180000000 to 179600000
As additional info a wgrib2 -v –V on a grib2 file previous to June 28 shows the
same thing as above.
Submitting to EDEX shows the following error (at the end in bold with preceding
traceback messages)
ERROR 2023-06-28 19:02:42,848 4308 [Ingest.GribDecode-6] grib: EDEX - Error
processing file [/awips2/data_store/manual/grib/20230628/19/test.grb2]:
Traceback (most recent call last):
File
"/awips2/edex/lib/plugins/com.raytheon.edex.plugin.grib.jar/GribDecoder.py",
line 184, in decode
gribDictList = grib2.decode(gribFile, self.startPosition,
self.messageLength)
error: Failed to get grib field.
If you have NAWIPS…
NAWIPS 7.14.0.1 dcgrib2 -d temp.log -v 100 test.grd < test.grb2 gives the
following error (in bold)
[27248] 230628/1520[DCINIT 3] Version 7.14.0.1
[27248] 230628/1520[DC 2] read 102399/102399 bytes strt 0 newstrt 102399
[27248] 230628/1520[DC 2] read 1/1 bytes strt 102399 newstrt 102400
[27248] 230628/1520[DC 2] read 83204/102397 bytes strt 0 newstrt 83204
[27248] 230628/1520[CED_GNUM -1] No GID KXKY [900 451] LL [90.000000
180.000000] UR [-90.000000 179.600006]
[27248] 230628/1520[CED_GNUM -1] No GID KXKY [900 451] LL [90.000000
180.000000] UR [-90.000000 179.600006]
[27248] 230628/1520[DC 2] read 0/19196 bytes strt 83204 newstrt 83204
[27248] 230628/1520[DCGRIB -9] End of Data
[27248] 230628/1520[DC 5] Normal termination.
[27248] 230628/1520[DC 2] Number of bulletins read and processed: 1
[27248] 230628/1520[DC 6] Shutting down.
And nagrib2 also gives an error (in bold)
getdrstemplate: DRS Template 5.42 not defined.
[NAGRIB2 -25] Error unpacking GRIB2 metadata
0 GRIB messages were read or scanned from the GRIB file:
test.grb2
0 grids were written to the GEMPAK file:
test.grd
Parameters requested: GBFILE,GDOUTF,PROJ,GRDAREA,KXKY,MAXGRD,CPYFIL,GAREA,
OUTPUT,G2TBLS,G2DIAG,OVERWR,PDSEXT.
________________________________
The information contained in this message is intended only for the personal and
confidential use of the recipient(s) named above. If the reader of this message
is not the intended recipient or an agent responsible for delivering it to the
intended recipient, you are hereby notified that you have received this
document in error and that any review, dissemination, distribution, or copying
of this message is strictly prohibited. If you have received this communication
in error, please notify us immediately, and delete the original message.
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
awips2-users mailing list
awips2-users@xxxxxxxxxxxxxxxx<mailto:awips2-users@xxxxxxxxxxxxxxxx>
For list information, to unsubscribe, or change your membership options, visit:
https://www.unidata.ucar.edu/mailing_lists/
--
(she/her/hers)
Software Engineer III
UCAR - Unidata
True Colors Chair
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
awips2-users mailing list
awips2-users@xxxxxxxxxxxxxxxx<mailto:awips2-users@xxxxxxxxxxxxxxxx>
For list information, to unsubscribe, or change your membership options, visit:
https://www.unidata.ucar.edu/mailing_lists/
------------------------------------------------------------------------------
The information contained in this message is intended only for the personal and
confidential use of the recipient(s) named above. If the reader of this message
is not the intended recipient or an agent responsible for delivering it to the
intended recipient, you are hereby notified that you have received this
document in error and that any review, dissemination, distribution, or copying
of this message is strictly prohibited. If you have received this communication
in error, please notify us immediately, and delete the original message.
