Unidata is definitely seeing issues:
https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+conduit.unidata.ucar.edu
Gilbert Sebenste
Meteorology Support Analyst
-----Original Message-----
From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx> On Behalf Of Tyle, Kevin R
Sent: Tuesday, October 17, 2023 1:37 PM
To: David Ovens <ovens@xxxxxxxxxxxxxxxxxxxx>; Mike Zuranski
<zuranski.wx@xxxxxxxxx>
Cc: ncep.pmb.dataflow@xxxxxxxx; CONDUIT Users List <conduit@xxxxxxxxxxxxxxxx>;
Unidata CONDUIT Support <support-conduit@xxxxxxxxxxxxxxxx>
Subject: [External] Re: [conduit] NCO Conduit Transition to RHEL8
CAUTION: This email originated from outside of COD’s system. Do not click
links, open attachments, or respond with sensitive information unless you
recognize the sender and know the content is safe.
I'm seeing "similar but different" incomplete GFS 12Z reception. Here is what
we received for the 0.5 degree grids (we pull every 6 forecast hours):
161091504 Oct 17 16:42 GFS_231017_12_066
161020526 Oct 17 16:41 GFS_231017_12_060
161532465 Oct 17 16:40 GFS_231017_12_030
160720347 Oct 17 16:39 GFS_231017_12_006
161437760 Oct 17 16:38 GFS_231017_12_024
161427788 Oct 17 16:38 GFS_231017_12_012
160771248 Oct 17 16:37 GFS_231017_12_054
149524822 Oct 17 16:34 GFS_231017_12_000
_________________________________________________
Kevin Tyle, M.S.; Manager of Departmental Computing NSF ACCESS Campus Champion
Dept. of Atmospheric & Environmental Sciences UAlbany ETEC Bldg - Harriman
Campus
1220 Washington Avenue, Room 419
Albany, NY 12222
ktyle@xxxxxxxxxx | 518-442-4578 | @nywxguy | he/him/his
_________________________________________________
-----Original Message-----
From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx> On Behalf Of David Ovens via
conduit
Sent: Tuesday, October 17, 2023 2:16 PM
To: Mike Zuranski <zuranski.wx@xxxxxxxxx>
Cc: ncep.pmb.dataflow@xxxxxxxx; CONDUIT Users List <conduit@xxxxxxxxxxxxxxxx>;
Unidata CONDUIT Support <support-conduit@xxxxxxxxxxxxxxxx>
Subject: Re: [conduit] NCO Conduit Transition to RHEL8
Hello everyone,
Our problems receiving CONDUIT data have continued to show up sporadically.
Today's 12Z GFS run produced only these files in our
/home/disk/data/conduit/gfs.20231017/12/atmos/ directory:
size time (PDT) filename
441024023 Oct 17 09:28 gfs.t12z.pgrb2.0p25.anl
504326579 Oct 17 09:27 gfs.t12z.pgrb2.0p25.f000
538762730 Oct 17 09:34 gfs.t12z.pgrb2.0p25.f003
543944330 Oct 17 09:29 gfs.t12z.pgrb2.0p25.f006
544617905 Oct 17 09:30 gfs.t12z.pgrb2.0p25.f015
546797452 Oct 17 09:28 gfs.t12z.pgrb2.0p25.f018
547155169 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f030
543889075 Oct 17 09:30 gfs.t12z.pgrb2.0p25.f033
546571525 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f036
548007247 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f042
544924791 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f045
546482726 Oct 17 09:29 gfs.t12z.pgrb2.0p25.f048
544314349 Oct 17 09:29 gfs.t12z.pgrb2.0p25.f051
544234794 Oct 17 09:35 gfs.t12z.pgrb2.0p25.f054
543829161 Oct 17 09:35 gfs.t12z.pgrb2.0p25.f057
541267075 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f069
538371335 Oct 17 09:34 gfs.t12z.pgrb2.0p25.f075
543307726 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f078
539918825 Oct 17 09:35 gfs.t12z.pgrb2.0p25.f084
540995533 Oct 17 09:36 gfs.t12z.pgrb2.0p25.f087
537682133 Oct 17 09:36 gfs.t12z.pgrb2.0p25.f090
As you can see, we are missing multiple forecast hours. And, typical arrival
times would be nearly one hour earlier for these files (we got the 0p25.anl
file at 08:34 PDT yesterday). Are others seeing similar delays and missing
data?
Cheers,
David
--
David Ovens e-mail: ovens@xxxxxxxxxxxxxxxxxxxx
Research Meteorologist phone: (206) 685-8108
Dept of Atm. Sciences plan: Real-time WRF forecasting for the
Box 351640 Pacific Northwest
University of Washington https://atmos.uw.edu/wrfrt
Seattle, WA 98195 Weather Graphics and Loops
https://atmos.uw.edu/~ovens/loops
On Sun, Oct 15, 2023 at 08:56:21PM -0500, Mike Zuranski wrote:
> On Sun, Oct 15, 2023 at 8:38???PM Herzmann, Daryl E [AGRON] <
> akrherz@xxxxxxxxxxx> wrote:
>
> > Hi Mike,
> >
> > Thanks, that was the clue-by-four I needed to be hit with. I
> > adjusted my LDM request and got the other half of the NDFD products
> > now being transmitted!
> >
>
> Awesome, glad you're getting everything!
>
>
>
> > PS, that screenshot is ultra fancy, is it available for us mere
> > mortals to use?
> >
> > daryl
> >
>
> Hey Thanks! :D
>
> I'm afraid not yet, it's far from ready for primetime and isn't always
> running attm. But I've got some irons in the fire for interfaces that
> would be more useful to you mere mortals our beloved & valued users.
> There's a bit for us to juggle right now so I can't say much in terms
> of timing, other than *I'm Working On It *(tm).
>
>
> Best,
> -Mike
>
> *Mike Zuranski*
> Data Engineer II
> Unidata Program Center
> University Corporation for Atmospheric Research
>
>
>
> > ________________________________________
> > From: Mike Zuranski <zuranski.wx@xxxxxxxxx>
> > Sent: Sunday, October 15, 2023 3:18 PM
> > To: Herzmann, Daryl E [AGRON]
> > Cc: CONDUIT Users List; ncep.pmb.dataflow@xxxxxxxx; Unidata CONDUIT
> > Support
> > Subject: Re: [conduit] NCO Conduit Transition to RHEL8
> >
> > Daryl,
> >
> > Looks like it's there to me, all under the
> > "/common/data/products/NDFD/grib2/..." header. 20Z coming in now actually:
> >
> > [image.png]
> >
> > Are you not seeing these?
> >
> > -Mike
> >
> >
> > Mike Zuranski
> > Data Engineer II
> > Unidata Program Center
> > University Corporation for Atmospheric Research
> >
> >
> > On Sun, Oct 15, 2023 at 1:00???PM Herzmann, Daryl E [AGRON] <
> > akrherz@xxxxxxxxxxx<mailto:akrherz@xxxxxxxxxxx>> wrote:
> > Hi Mike,
> >
> > Thanks, but I am not seeing either of those two. Do you have
> > examples from today?
> >
> > daryl
> >
> > ________________________________________
> > From: Mike Zuranski
> > <zuranski.wx@xxxxxxxxx<mailto:zuranski.wx@xxxxxxxxx>>
> > Sent: Sunday, October 15, 2023 11:52 AM
> > To: Herzmann, Daryl E [AGRON]
> > Cc: CONDUIT Users List; ncep.pmb.dataflow@xxxxxxxx<mailto:
> > ncep.pmb.dataflow@xxxxxxxx>; Unidata CONDUIT Support
> > Subject: Re: [conduit] NCO Conduit Transition to RHEL8
> >
> > Hi Daryl,
> >
> > For some unknown amount of time, there has been duplicate NDFD data
> > on CONDUIT that would come with two separate product IDs, though the
> > data was identical. Here is an example of the differences; the two
> > products were otherwise bytewise identical:
> >
> > /common/data/products/NDFD/grib2/YCUZ98_KWBN_121351.grib2
> > !grib2/nwstg/NWS_0/#000/202310121400F058/SPED/10 m HGHT! 000042
> > data2/ndfd/YCUZ98_KWBN_121351.grib2
> > !grib2/nwstg/NWS_0/#000/202310121400F058/SPED/10 m HGHT! 000042
> >
> > I have been in talks with NCEP about this and I know they were
> > looking into it; while I wasn't aware of any plans, it would appear
> > they have consolidated/rectified the different product IDs with this
> > update. From my records it looks like the latter pattern stopped
> > around 10/12 14Z, the former pattern continues to be used.
> >
> > If you search using the former product ID pattern, do you see all
> > the expected NDFD products then?
> >
> > -Mike
> >
> >
> > Mike Zuranski
> > Data Engineer II
> > Unidata Program Center
> > University Corporation for Atmospheric Research
> >
> >
> > On Sun, Oct 15, 2023 at 10:58???AM Herzmann, Daryl E [AGRON] <
> > akrherz@xxxxxxxxxxx<mailto:akrherz@xxxxxxxxxxx><mailto:akrherz@iasta
> > te.edu <mailto:akrherz@xxxxxxxxxxx>>> wrote:
> > Greetings,
> >
> > Since this transition, I am noticing that NDFD data is not flowing
> > over CONDUIT. Anybody else able to verify?
> >
> > daryl
> >
> > ________________________________________
> > From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx<mailto:
> > conduit-bounces@xxxxxxxxxxxxxxxx><mailto:conduit-bounces@xxxxxxxxxxx
> > r.edu <mailto:conduit-bounces@xxxxxxxxxxxxxxxx>>> on behalf of NCEP
> > Dataflow via conduit
> > <conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx
> > ><mailto:conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx>>>
> > Sent: Wednesday, October 11, 2023 2:18 PM
> > To: CONDUIT Users List
> > Cc: Unidata CONDUIT Support
> > Subject: Re: [conduit] NCO Conduit Transition to RHEL8
> >
> > Hello CONDUIT Users,
> >
> > The start date of the transition of NCO's Conduit servers to RHEL8
> > was pushed back by one day due to Critical Weather Day on Tuesday 10/10 .
> > Therefore our migration to the new systems began today, Wednesday
> > October 11th. The transition will still take place over three days
> > with our operational servers now planned to migrate tomorrow,
> > Thursday October 12th at ~1300z.
> >
> > Changes on your end are not necessary and we are anticipating a
> > seamless transition. If however you do encounter issues that are not
> > resolved by an LDM restart, please report them to NCO Dataflow (
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>>>) as
> > soon as possible with a subject line that includes the text "CONDUIT Issue".
> >
> > Thank you
> > --
> > NCO Dataflow Team
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>>>
> >
> >
> > On Fri, Oct 6, 2023 at 12:44???PM Dustin Sheffler - NOAA Federal <
> > dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx><mailto:
> > dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx>><mailto:
> > dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx><mailto:
> > dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx>>>> wrote:
> > Attention CONDUIT Users,
> >
> > Beginning Tuesday, October 10th, 2023 at 1300z, NCO Dataflow will
> > begin the migration of our CONDUIT systems to RHEL 8. The transition
> > will take place over three days with our operational servers
> > migrating on Wednesday, October 11th at 1300z. Changes on your end
> > are not necessary and we are anticipating a seamless transition. If
> > however you do encounter issues that are not resolved by an LDM
> > restart, please report them to NCO Dataflow (
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>>>) as
> > soon as possible with a subject line that includes the text "CONDUIT Issue".
> >
> > Thank you
> > --
> > NCO Dataflow Team
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>>>
> >
> > To sign up for the NOMADS/FTPPRD Users List:
> > https://urldefense.com/v3/__https://www.lstsrv.ncep.noaa.gov/mailman
> > /listinfo/ncep.list.nomads-ftpprd__;!!K-Hz7m0Vt54!jO7oxor844lf1mx6eZ
> > 1lLm9IiXb2tQwuasmf7fpm4zYAovUgKafvlTkGMqh8bT4HAUS2ZVJnKh6djGcW_Q$
> > <tel:%28301%29%20683-1400>
> > _______________________________________________
> > NOTE: All exchanges posted to Unidata maintained email lists are
> > recorded in the Unidata inquiry tracking system and made publicly
> > available through the web. Users who post to any of the lists we
> > maintain are reminded to remove any personal information that they
> > do not want to be made public.
> >
> >
> > conduit mailing list
> > conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx><mailto:
> > conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx>>
> > For list information or to unsubscribe, visit:
> > https://urldefense.com/v3/__https://www.unidata.ucar.edu/mailing_lis
> > ts/__;!!K-Hz7m0Vt54!jO7oxor844lf1mx6eZ1lLm9IiXb2tQwuasmf7fpm4zYAovUg
> > KafvlTkGMqh8bT4HAUS2ZVJnKh5KsIwTTg$
> >
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they do
> not want to be made public.
>
>
> conduit mailing list
> conduit@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://urldefense.com/v3/__https://www.unidata.ucar.edu/mailing_lists
> /__;!!K-Hz7m0Vt54!jO7oxor844lf1mx6eZ1lLm9IiXb2tQwuasmf7fpm4zYAovUgKafv
> lTkGMqh8bT4HAUS2ZVJnKh5KsIwTTg$
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are recorded in
the Unidata inquiry tracking system and made publicly available through the
web. Users who post to any of the lists we maintain are reminded to remove any
personal information that they do not want to be made public.
conduit mailing list
conduit@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
https://www.unidata.ucar.edu/mailing_lists/
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are recorded in
the Unidata inquiry tracking system and made publicly available through the
web. Users who post to any of the lists we maintain are reminded to remove any
personal information that they do not want to be made public.
conduit mailing list
conduit@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
https://www.unidata.ucar.edu/mailing_lists/
