Hi Ryan,
I was able to replicate the issue of not displaying, but was able to get
around it by closing CAVE, remove ~/caveData, start up CAVE again and then
try loading.
It's probably an issue with what's cached, but by removing caveData it
clears out the cache.
Let me know if that resolves the issue. Thanks,
~Tiffany
On Wed, Dec 30, 2020 at 12:28 PM Ryan Hickman <ryan@xxxxxxxxxxxxxxxx> wrote:
> Tiffany,
>
> It actually appears that while the TIU* files are being ingested by the
> GOES-R ingester, they're not able to be displayed in AWIPS.
>
> INFO 2020-12-30 18:24:39,600 1263 [Ingest.GOESR-2] Ingest: EDEX: Ingest -
>> goes-r::
>> /awips2/data_store/goesr/noaaport/TIUR07_KNES_301823_PAA_925331.gini
>> processed in: 0.0460 (sec) Latency: 0.2260 (sec)
>> INFO 2020-12-30 18:24:40,583 1274 [Ingest.GOESR-2] Ingest: EDEX: Ingest
>> - goes-r::
>> /awips2/data_store/goesr/noaaport/TIUR08_KNES_301823_PAA_925339.gini
>> processed in: 0.0470 (sec) Latency: 0.2080 (sec)
>> INFO 2020-12-30 18:24:40,674 1279 [Ingest.GOESR-2] Ingest: EDEX: Ingest
>> - goes-r::
>> /awips2/data_store/goesr/noaaport/TIUR11_KNES_301823_PAA_925341.gini
>> processed in: 0.0420 (sec) Latency: 0.2830 (sec)
>>
>
> [awips@edex-new satellite]$ ls
>> AKREGI ECONUS EFD EMESO-1 EMESO-2 HIREGI PRREGI WCONUS WFD
>> WMESO-1 WMESO-2
>> [awips@edex-new satellite]$ ls -al WMESO-1/CH-01-0.47um/
>> total 57372
>> drwxr-xr-x 2 awips fxalpha 4096 Dec 30 18:25 .
>> drwxr-xr-x 18 awips fxalpha 4096 Dec 30 17:48 ..
>> -rw-r--r-- 1 awips fxalpha 20016296 Dec 30 18:00
>> satellite-2020-12-30-17.h5
>> -rw-r--r-- 1 awips fxalpha 38714523 Dec 30 18:25
>> satellite-2020-12-30-18.h5
>>
>
> [image: Screen Shot 2020-12-30 at 11.26.31 AM.jpg]
>
> Is there potentially another change I might need to make?
>
> On Wed, Dec 30, 2020 at 11:21 AM Tiffany Meyer <tiffanym@xxxxxxxx> wrote:
>
>> Hi Ryan,
>>
>> Glad you got this issue resolved. There are two ways to get/ingest the
>> GOES 16/17 ABI data.
>>
>> 1. Use the NOAAPort feed which get stored as the .gini files (The top
>> option below)
>> 2. Use the NIMAGE data Unidata is sending out which are the netcdf files
>> stitched together, with the WMO header stripped so the data can also be
>> read into other software (the bottom option below)
>>
>> #
>> # GOES 16/17 ABI
>> # NOAAPort
>> NOTHER ^(TI....) (KNES) (......) (...)
>> FILE -close -edex
>> /awips2/data_store/GOES/GINI/\1_\2_\3_\4_(seq).gini
>> #
>> # GOES 16/17 ABI
>> # Unidata-stitched
>> NIMAGE
>> ^/data/ldm/pub/native/satellite/GOES/([^/]*)/Products/CloudAndMoistureImagery/([^/]*)/([^/]*)/([0-9]{8})/([^/]*).nc
>> FILE -close -edex
>> /awips2/data_store/GOES/CMI/\5.nc4
>>
>>
>> In-house we are ingesting the NIMAGE netcdf files, but either way works.
>>
>> Thanks,
>> ~Tiffany
>>
>> On Wed, Dec 30, 2020 at 11:49 AM Ryan Hickman <ryan@xxxxxxxxxxxxxxxx>
>> wrote:
>>
>>> I noticed my goof. You need to edit `distribution/goesr.xml` not
>>> satellite.xml.
>>>
>>> "<regex>^TI[SR]</regex>" becomes "<regex>^TI[SRU]</regex>" and all is
>>> well.
>>>
>>> INFO 2020-12-30 17:49:16,588 2820 [Ingest.GOESR-1] Ingest: EDEX: Ingest
>>>> - goes-r::
>>>> /awips2/data_store/goesr/noaaport/TIUR13_KNES_301748_PAA_921505.gini
>>>> processed in: 0.0600 (sec) Latency: 0.0830 (sec)
>>>>
>>>
>>> Consider this resolved.
>>>
>>> On Wed, Dec 30, 2020 at 10:17 AM Ryan Hickman <ryan@xxxxxxxxxxxxxxxx>
>>> wrote:
>>>
>>>> I am unable to get EDEX to ingest the GOES-17 Mesoscale sectors from
>>>> NOAAPORT. This started with a report of that data missing and I was able to
>>>> find entries in the "unrecognized files" log.
>>>>
>>>> WARN 2020-12-30 17:01:41,160 6124 [external.dropbox-2] RouteFailedLog:
>>>>> TRDA01 KWBC 301600
>>>>> WARN 2020-12-30 17:01:41,161 6125 [external.dropbox-5]
>>>>> RouteFailedLog: THBA01 KWBC 301500
>>>>> WARN 2020-12-30 17:01:41,162 6126 [external.dropbox-4]
>>>>> RouteFailedLog: TRDA01 KWBC 301700
>>>>> WARN 2020-12-30 17:01:41,166 6127 [external.dropbox-5]
>>>>> RouteFailedLog: TRAA01 KWBC 301600
>>>>> WARN 2020-12-30 17:01:41,167 6128 [external.dropbox-4]
>>>>> RouteFailedLog: TRBA01 KWBC 301500
>>>>> WARN 2020-12-30 17:01:41,171 6129 [external.dropbox-5]
>>>>> RouteFailedLog: TRDA01 KWBC 301600
>>>>> WARN 2020-12-30 17:01:41,171 6130 [external.dropbox-4]
>>>>> RouteFailedLog: THDA01 KWBC 301600
>>>>> WARN 2020-12-30 17:01:41,174 6131 [external.dropbox-2]
>>>>> RouteFailedLog: THDA01 KWBC 301600
>>>>> WARN 2020-12-30 17:01:42,308 6132 [external.dropbox-5]
>>>>> RouteFailedLog: TRBA01 KWBC 301500
>>>>> WARN 2020-12-30 17:01:42,314 6133 [external.dropbox-5]
>>>>> RouteFailedLog: THDA01 KWBC 301600
>>>>>
>>>>
>>>> The pqact.conf entry is the default one that was included with EDEX.
>>>>
>>>>> #
>>>>> # GOES 16/17 ABI
>>>>> #
>>>>> NOTHER ^(TI....) (KNES) (......) (...)
>>>>> FILE -close -edex
>>>>> /awips2/data_store/goesr/noaaport/\1_\2_\3_\4_(seq).gini
>>>>>
>>>>
>>>> Since the `distribution/satellite.gini.xml` includes
>>>> "<regex>.*.gini.*</regex>" you'd figure it would get ingested. Regardless,
>>>> I added "<regex>TIU... KNES</regex>" and did an `edex restart ingest` which
>>>> got the data flowing to the right ingester.
>>>>
>>>> INFO 2020-12-30 17:09:30,981 9392 [Ingest.Gini-1] Ingest: EDEX: Ingest
>>>>> - satellite::
>>>>> /awips2/data_store/goesr/noaaport/TIUR06_KNES_301708_PAA_917280.gini
>>>>> processed in: 0.0040 (sec) Latency: 0.0240 (sec)
>>>>> INFO 2020-12-30 17:09:33,202 9397 [Ingest.Gini-1] Ingest: EDEX:
>>>>> Ingest - satellite::
>>>>> /awips2/data_store/goesr/noaaport/TIUR08_KNES_301708_PAA_917282.gini
>>>>> processed in: 0.0030 (sec) Latency: 0.1670 (sec)
>>>>> INFO 2020-12-30 17:09:33,211 9400 [Ingest.Gini-1] Ingest: EDEX:
>>>>> Ingest - satellite::
>>>>> /awips2/data_store/goesr/noaaport/TIUR04_KNES_301708_PAA_917283.gini
>>>>> processed in: 0.0040 (sec) Latency: 0.1760 (sec)
>>>>>
>>>>
>>>> However the data is still not displayable in AWIPS CAVE ("No Data
>>>> Available") and the log above seems to indicate it's being processed as a
>>>> generic "Gini-1" instead of "GOESR-2" like the rest of the GOES-17
>>>> products.
>>>>
>>>> Has anyone else had similar issues with the NOAAPORT GOES-17 Mesoscale
>>>> data? Is there another configuration file I might need to change to get it
>>>> to recognize these files as GOES-17 HDFs?
>>>>
>>> _______________________________________________
>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>> recorded in the Unidata inquiry tracking system and made publicly
>>> available through the web. Users who post to any of the lists we
>>> maintain are reminded to remove any personal information that they
>>> do not want to be made public.
>>>
>>>
>>> awips2-users mailing list
>>> awips2-users@xxxxxxxxxxxxxxxx
>>> For list information, to unsubscribe, or change your membership options,
>>> visit: https://www.unidata.ucar.edu/mailing_lists/
>>>
>>
>>
>> --
>>
>> Tiffany Meyer
>> AWIPS Lead Software Engineer IV
>> UCAR - Unidata
>>
>>
--
Tiffany Meyer
AWIPS Lead Software Engineer IV
UCAR - Unidata
