[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [conduit] NCO Conduit Transition to RHEL8

All,

I've also noticed that since the update, the GFS products are coming through without any product ID on them other than the file name. 

Before, they had unique product IDs after the exclmation point, like this (from an ldmadmin watch from 2021)

20210621T163257.947961Z pqutil[37223]               pqutil.c:display_watch:1186         INFO       75741 20210621163239.720822 CONDUIT 064  data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.1p00.f225 !grib2/ncep/GFS/#000/202106211200F225/HGHT/0 hPa PRES! 000064
20210621T163257.948000Z pqutil[37223]               pqutil.c:display_watch:1186         INFO       77396 20210621163239.820916 CONDUIT 065  data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.1p00.f225 !grib2/ncep/GFS/#000/202106211200F225/TMPK/0 hPa PRES! 000065
20210621T163257.948039Z pqutil[37223]               pqutil.c:display_watch:1186         INFO     1213997 20210621163241.523907 CONDUIT 047  data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.0p25.f225 !grib2/ncep/GFS/#000/202106211200F225/SPFH/0 hPa PRES! 000047
20210621T163257.948126Z pqutil[37223]               pqutil.c:display_watch:1186         INFO      170943 20210621163228.510736 CONDUIT 738  data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.0p50.f222 !grib2/ncep/GFS/#000/202106211200F222/VREL/-2 K*m**2*kg-1*s-1 POTV! 000738
20210621T163257.948233Z pqutil[37223]               pqutil.c:display_watch:1186         INFO     1238726 20210621163241.792977 CONDUIT 459  data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.0p25.f222 !grib2/ncep/GFS/#000/202106211200F222/OZMR/800 hPa PRES! 000459
20210621T163257.948279Z pqutil[37223]               pqutil.c:display_watch:1186         INFO       69594 20210621163242.150502 CONDUIT 119  data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.1p00.f225 !grib2/ncep/GFS/#000/202106211200F225/VVEL/3 hPa PRES! 000119
20210621T163257.948360Z pqutil[37223]               pqutil.c:display_watch:1186         INFO      970175 20210621163238.587907 CONDUIT 681  data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.0p25.f219 !grib2/ncep/GFS/#000/202106211200F219/VREL/50 m HGHT! 000681


Now every product comes through with generic filler info like:

20231015T041432.757539Z pqutil[27287]               pqutil.c:display_watch:1186         INFO       82796 20231015041414.643571 CONDUIT 659  data/nccf/com/gfs/v16.3/gfs.20231015/00/atmos/gfs.t00z.pgrb2.1p00.f162 !grib2/ncep/GFS/#000/FHRS//LVL! 000659
20231015T041432.757575Z pqutil[27287]               pqutil.c:display_watch:1186         INFO       67717 20231015041414.636494 CONDUIT 650  data/nccf/com/gfs/v16.3/gfs.20231015/00/atmos/gfs.t00z.pgrb2.1p00.f162 !grib2/ncep/GFS/#000/FHRS//LVL! 000650
20231015T041432.757608Z pqutil[27287]               pqutil.c:display_watch:1186         INFO       81162 20231015041414.644156 CONDUIT 660  data/nccf/com/gfs/v16.3/gfs.20231015/00/atmos/gfs.t00z.pgrb2.1p00.f162 !grib2/ncep/GFS/#000/FHRS//LVL! 000660
20231015T041432.757649Z pqutil[27287]               pqutil.c:display_watch:1186         INFO      118893 20231015041414.654915 CONDUIT 669  data/nccf/com/gfs/v16.3/gfs.20231015/00/atmos/gfs.t00z.pgrb2.1p00.f162 !grib2/ncep/GFS/#000/FHRS//LVL! 000669
20231015T041432.757682Z pqutil[27287]               pqutil.c:display_watch:1186         INFO      120383 20231015041414.656654 CONDUIT 670  data/nccf/com/gfs/v16.3/gfs.20231015/00/atmos/gfs.t00z.pgrb2.1p00.f162 !grib2/ncep/GFS/#000/FHRS//LVL! 000670

I have some cases where I had been keying in on that to only save certain fields to a file, and now they aren't getting saved because it doesn't see that.

For what it's worth..
Pete

Pete Pokrandt - System Engineer IV
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086  - address@hidden
From: conduit <address@hidden> on behalf of Mike Zuranski <address@hidden>
Sent: Sunday, October 15, 2023 11:52 AM
To: Herzmann, Daryl E [AGRON] <address@hidden>
Cc: address@hidden <address@hidden>; CONDUIT Users List <address@hidden>; Unidata CONDUIT Support <address@hidden>
Subject: Re: [conduit] NCO Conduit Transition to RHEL8
 
Hi Daryl,

For some unknown amount of time, there has been duplicate NDFD data on CONDUIT that would come with two separate product IDs, though the data was identical.  Here is an example of the differences; the two products were otherwise bytewise identical:

/common/data/products/NDFD/grib2/YCUZ98_KWBN_121351.grib2 !grib2/nwstg/NWS_0/#000/202310121400F058/SPED/10 m HGHT! 000042
data2/ndfd/YCUZ98_KWBN_121351.grib2 !grib2/nwstg/NWS_0/#000/202310121400F058/SPED/10 m HGHT! 000042

I have been in talks with NCEP about this and I know they were looking into it; while I wasn't aware of any plans, it would appear they have consolidated/rectified the different product IDs with this update.  From my records it looks like the latter pattern stopped around 10/12 14Z, the former pattern continues to be used.

If you search using the former product ID pattern, do you see all the expected NDFD products then?

-Mike

Mike Zuranski
Data Engineer II
Unidata Program Center
University Corporation for Atmospheric Research


On Sun, Oct 15, 2023 at 10:58 AM Herzmann, Daryl E [AGRON] <address@hidden> wrote:
Greetings,

Since this transition, I am noticing that NDFD data is not flowing over CONDUIT.  Anybody else able to verify?

daryl

________________________________________
From: conduit <address@hidden> on behalf of NCEP Dataflow via conduit <address@hidden>
Sent: Wednesday, October 11, 2023 2:18 PM
To: CONDUIT Users List
Cc: Unidata CONDUIT Support
Subject: Re: [conduit] NCO Conduit Transition to RHEL8

Hello CONDUIT Users,

The start date of the transition of NCO's Conduit servers to RHEL8 was pushed back by one day due to Critical Weather Day on Tuesday 10/10. Therefore our migration to the new systems began today, Wednesday October 11th. The transition will still take place over three days with our operational servers now planned to migrate tomorrow, Thursday October 12th at ~1300z.

Changes on your end are not necessary and we are anticipating a seamless transition. If however you do encounter issues that are not resolved by an LDM restart, please report them to NCO Dataflow (address@hidden<mailto:address@hidden>) as soon as possible with a subject line that includes the text "CONDUIT Issue".

Thank you
--
NCO Dataflow Team
address@hidden<mailto:address@hidden>


On Fri, Oct 6, 2023 at 12:44 PM Dustin Sheffler - NOAA Federal <address@hidden<mailto:address@hidden>> wrote:
Attention CONDUIT Users,

Beginning Tuesday, October 10th, 2023 at 1300z, NCO Dataflow will begin the migration of our CONDUIT systems to RHEL 8. The transition will take place over three days with our operational servers migrating on Wednesday, October 11th at 1300z. Changes on your end are not necessary and we are anticipating a seamless transition. If however you do encounter issues that are not resolved by an LDM restart, please report them to NCO Dataflow (address@hidden<mailto:address@hidden>) as soon as possible with a subject line that includes the text "CONDUIT Issue".

Thank you
--
NCO Dataflow Team
address@hidden<mailto:address@hidden>

To sign up for the NOMADS/FTPPRD Users List:
https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nomads-ftpprd
<tel:%28301%29%20683-1400>
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


conduit mailing list
address@hidden
For list information or to unsubscribe, visit: https://www.unidata.ucar.edu/mailing_lists/ 
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


conduit mailing list
address@hidden
For list information or to unsubscribe, visit: 
https://www.unidata.ucar.edu/mailing_lists/