Tiffany,
It actually appears that while the TIU* files are being ingested by the
GOES-R ingester, they're not able to be displayed in AWIPS.
INFO 2020-12-30 18:24:39,600 1263 [Ingest.GOESR-2] Ingest: EDEX: Ingest -
> goes-r::
> /awips2/data_store/goesr/noaaport/TIUR07_KNES_301823_PAA_925331.gini
> processed in: 0.0460 (sec) Latency: 0.2260 (sec)
> INFO 2020-12-30 18:24:40,583 1274 [Ingest.GOESR-2] Ingest: EDEX: Ingest -
> goes-r::
> /awips2/data_store/goesr/noaaport/TIUR08_KNES_301823_PAA_925339.gini
> processed in: 0.0470 (sec) Latency: 0.2080 (sec)
> INFO 2020-12-30 18:24:40,674 1279 [Ingest.GOESR-2] Ingest: EDEX: Ingest -
> goes-r::
> /awips2/data_store/goesr/noaaport/TIUR11_KNES_301823_PAA_925341.gini
> processed in: 0.0420 (sec) Latency: 0.2830 (sec)
>
[awips@edex-new satellite]$ ls
> AKREGI ECONUS EFD EMESO-1 EMESO-2 HIREGI PRREGI WCONUS WFD
> WMESO-1 WMESO-2
> [awips@edex-new satellite]$ ls -al WMESO-1/CH-01-0.47um/
> total 57372
> drwxr-xr-x 2 awips fxalpha 4096 Dec 30 18:25 .
> drwxr-xr-x 18 awips fxalpha 4096 Dec 30 17:48 ..
> -rw-r--r-- 1 awips fxalpha 20016296 Dec 30 18:00
> satellite-2020-12-30-17.h5
> -rw-r--r-- 1 awips fxalpha 38714523 Dec 30 18:25
> satellite-2020-12-30-18.h5
>
[image: Screen Shot 2020-12-30 at 11.26.31 AM.jpg]
Is there potentially another change I might need to make?
On Wed, Dec 30, 2020 at 11:21 AM Tiffany Meyer <tiffanym@xxxxxxxx> wrote:
> Hi Ryan,
>
> Glad you got this issue resolved. There are two ways to get/ingest the
> GOES 16/17 ABI data.
>
> 1. Use the NOAAPort feed which get stored as the .gini files (The top
> option below)
> 2. Use the NIMAGE data Unidata is sending out which are the netcdf files
> stitched together, with the WMO header stripped so the data can also be
> read into other software (the bottom option below)
>
> #
> # GOES 16/17 ABI
> # NOAAPort
> NOTHER ^(TI....) (KNES) (......) (...)
> FILE -close -edex
> /awips2/data_store/GOES/GINI/\1_\2_\3_\4_(seq).gini
> #
> # GOES 16/17 ABI
> # Unidata-stitched
> NIMAGE
> ^/data/ldm/pub/native/satellite/GOES/([^/]*)/Products/CloudAndMoistureImagery/([^/]*)/([^/]*)/([0-9]{8})/([^/]*).nc
> FILE -close -edex
> /awips2/data_store/GOES/CMI/\5.nc4
>
>
> In-house we are ingesting the NIMAGE netcdf files, but either way works.
>
> Thanks,
> ~Tiffany
>
> On Wed, Dec 30, 2020 at 11:49 AM Ryan Hickman <ryan@xxxxxxxxxxxxxxxx>
> wrote:
>
>> I noticed my goof. You need to edit `distribution/goesr.xml` not
>> satellite.xml.
>>
>> "<regex>^TI[SR]</regex>" becomes "<regex>^TI[SRU]</regex>" and all is
>> well.
>>
>> INFO 2020-12-30 17:49:16,588 2820 [Ingest.GOESR-1] Ingest: EDEX: Ingest
>>> - goes-r::
>>> /awips2/data_store/goesr/noaaport/TIUR13_KNES_301748_PAA_921505.gini
>>> processed in: 0.0600 (sec) Latency: 0.0830 (sec)
>>>
>>
>> Consider this resolved.
>>
>> On Wed, Dec 30, 2020 at 10:17 AM Ryan Hickman <ryan@xxxxxxxxxxxxxxxx>
>> wrote:
>>
>>> I am unable to get EDEX to ingest the GOES-17 Mesoscale sectors from
>>> NOAAPORT. This started with a report of that data missing and I was able to
>>> find entries in the "unrecognized files" log.
>>>
>>> WARN 2020-12-30 17:01:41,160 6124 [external.dropbox-2] RouteFailedLog:
>>>> TRDA01 KWBC 301600
>>>> WARN 2020-12-30 17:01:41,161 6125 [external.dropbox-5] RouteFailedLog:
>>>> THBA01 KWBC 301500
>>>> WARN 2020-12-30 17:01:41,162 6126 [external.dropbox-4] RouteFailedLog:
>>>> TRDA01 KWBC 301700
>>>> WARN 2020-12-30 17:01:41,166 6127 [external.dropbox-5] RouteFailedLog:
>>>> TRAA01 KWBC 301600
>>>> WARN 2020-12-30 17:01:41,167 6128 [external.dropbox-4] RouteFailedLog:
>>>> TRBA01 KWBC 301500
>>>> WARN 2020-12-30 17:01:41,171 6129 [external.dropbox-5] RouteFailedLog:
>>>> TRDA01 KWBC 301600
>>>> WARN 2020-12-30 17:01:41,171 6130 [external.dropbox-4] RouteFailedLog:
>>>> THDA01 KWBC 301600
>>>> WARN 2020-12-30 17:01:41,174 6131 [external.dropbox-2] RouteFailedLog:
>>>> THDA01 KWBC 301600
>>>> WARN 2020-12-30 17:01:42,308 6132 [external.dropbox-5] RouteFailedLog:
>>>> TRBA01 KWBC 301500
>>>> WARN 2020-12-30 17:01:42,314 6133 [external.dropbox-5] RouteFailedLog:
>>>> THDA01 KWBC 301600
>>>>
>>>
>>> The pqact.conf entry is the default one that was included with EDEX.
>>>
>>>> #
>>>> # GOES 16/17 ABI
>>>> #
>>>> NOTHER ^(TI....) (KNES) (......) (...)
>>>> FILE -close -edex
>>>> /awips2/data_store/goesr/noaaport/\1_\2_\3_\4_(seq).gini
>>>>
>>>
>>> Since the `distribution/satellite.gini.xml` includes
>>> "<regex>.*.gini.*</regex>" you'd figure it would get ingested. Regardless,
>>> I added "<regex>TIU... KNES</regex>" and did an `edex restart ingest` which
>>> got the data flowing to the right ingester.
>>>
>>> INFO 2020-12-30 17:09:30,981 9392 [Ingest.Gini-1] Ingest: EDEX: Ingest
>>>> - satellite::
>>>> /awips2/data_store/goesr/noaaport/TIUR06_KNES_301708_PAA_917280.gini
>>>> processed in: 0.0040 (sec) Latency: 0.0240 (sec)
>>>> INFO 2020-12-30 17:09:33,202 9397 [Ingest.Gini-1] Ingest: EDEX: Ingest
>>>> - satellite::
>>>> /awips2/data_store/goesr/noaaport/TIUR08_KNES_301708_PAA_917282.gini
>>>> processed in: 0.0030 (sec) Latency: 0.1670 (sec)
>>>> INFO 2020-12-30 17:09:33,211 9400 [Ingest.Gini-1] Ingest: EDEX: Ingest
>>>> - satellite::
>>>> /awips2/data_store/goesr/noaaport/TIUR04_KNES_301708_PAA_917283.gini
>>>> processed in: 0.0040 (sec) Latency: 0.1760 (sec)
>>>>
>>>
>>> However the data is still not displayable in AWIPS CAVE ("No Data
>>> Available") and the log above seems to indicate it's being processed as a
>>> generic "Gini-1" instead of "GOESR-2" like the rest of the GOES-17 products.
>>>
>>> Has anyone else had similar issues with the NOAAPORT GOES-17 Mesoscale
>>> data? Is there another configuration file I might need to change to get it
>>> to recognize these files as GOES-17 HDFs?
>>>
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web. Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> awips2-users mailing list
>> awips2-users@xxxxxxxxxxxxxxxx
>> For list information, to unsubscribe, or change your membership options,
>> visit: https://www.unidata.ucar.edu/mailing_lists/
>>
>
>
> --
>
> Tiffany Meyer
> AWIPS Lead Software Engineer IV
> UCAR - Unidata
>
>
