Hi All,
Today we migrated our system that feeds LDM on our CONDUIT servers. We ran
into an unforeseen issue with connections, and are in the middle of
resolving the issue. Currently we are seeing successful inserts into the
LDM queue. Please let us know if the issues persist.
Regards,
William
On Tue, Oct 17, 2023 at 7:23 PM Sebenste, Gilbert <sebensteg@xxxxxxx> wrote:
> Unidata is definitely seeing issues:
>
>
> https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+conduit.unidata.ucar.edu
>
> Gilbert Sebenste
> Meteorology Support Analyst
>
>
>
> -----Original Message-----
> From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx> On Behalf Of Tyle, Kevin
> R
> Sent: Tuesday, October 17, 2023 1:37 PM
> To: David Ovens <ovens@xxxxxxxxxxxxxxxxxxxx>; Mike Zuranski <
> zuranski.wx@xxxxxxxxx>
> Cc: ncep.pmb.dataflow@xxxxxxxx; CONDUIT Users List <
> conduit@xxxxxxxxxxxxxxxx>; Unidata CONDUIT Support <
> support-conduit@xxxxxxxxxxxxxxxx>
> Subject: [External] Re: [conduit] NCO Conduit Transition to RHEL8
>
> CAUTION: This email originated from outside of COD’s system. Do not click
> links, open attachments, or respond with sensitive information unless you
> recognize the sender and know the content is safe.
>
>
> I'm seeing "similar but different" incomplete GFS 12Z reception. Here is
> what we received for the 0.5 degree grids (we pull every 6 forecast hours):
>
> 161091504 Oct 17 16:42 GFS_231017_12_066
> 161020526 Oct 17 16:41 GFS_231017_12_060
> 161532465 Oct 17 16:40 GFS_231017_12_030
> 160720347 Oct 17 16:39 GFS_231017_12_006
> 161437760 Oct 17 16:38 GFS_231017_12_024
> 161427788 Oct 17 16:38 GFS_231017_12_012
> 160771248 Oct 17 16:37 GFS_231017_12_054
> 149524822 Oct 17 16:34 GFS_231017_12_000
>
> _________________________________________________
>
> Kevin Tyle, M.S.; Manager of Departmental Computing NSF ACCESS Campus
> Champion Dept. of Atmospheric & Environmental Sciences UAlbany ETEC Bldg -
> Harriman Campus
> 1220 Washington Avenue, Room 419
> Albany, NY 12222
> ktyle@xxxxxxxxxx | 518-442-4578 | @nywxguy | he/him/his
> _________________________________________________
>
> -----Original Message-----
> From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx> On Behalf Of David Ovens
> via conduit
> Sent: Tuesday, October 17, 2023 2:16 PM
> To: Mike Zuranski <zuranski.wx@xxxxxxxxx>
> Cc: ncep.pmb.dataflow@xxxxxxxx; CONDUIT Users List <
> conduit@xxxxxxxxxxxxxxxx>; Unidata CONDUIT Support <
> support-conduit@xxxxxxxxxxxxxxxx>
> Subject: Re: [conduit] NCO Conduit Transition to RHEL8
>
> Hello everyone,
>
> Our problems receiving CONDUIT data have continued to show up
> sporadically. Today's 12Z GFS run produced only these files in our
> /home/disk/data/conduit/gfs.20231017/12/atmos/ directory:
>
> size time (PDT) filename
> 441024023 Oct 17 09:28 gfs.t12z.pgrb2.0p25.anl
> 504326579 Oct 17 09:27 gfs.t12z.pgrb2.0p25.f000
> 538762730 Oct 17 09:34 gfs.t12z.pgrb2.0p25.f003
> 543944330 Oct 17 09:29 gfs.t12z.pgrb2.0p25.f006
> 544617905 Oct 17 09:30 gfs.t12z.pgrb2.0p25.f015
> 546797452 Oct 17 09:28 gfs.t12z.pgrb2.0p25.f018
> 547155169 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f030
> 543889075 Oct 17 09:30 gfs.t12z.pgrb2.0p25.f033
> 546571525 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f036
> 548007247 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f042
> 544924791 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f045
> 546482726 Oct 17 09:29 gfs.t12z.pgrb2.0p25.f048
> 544314349 Oct 17 09:29 gfs.t12z.pgrb2.0p25.f051
> 544234794 Oct 17 09:35 gfs.t12z.pgrb2.0p25.f054
> 543829161 Oct 17 09:35 gfs.t12z.pgrb2.0p25.f057
> 541267075 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f069
> 538371335 Oct 17 09:34 gfs.t12z.pgrb2.0p25.f075
> 543307726 Oct 17 09:33 gfs.t12z.pgrb2.0p25.f078
> 539918825 Oct 17 09:35 gfs.t12z.pgrb2.0p25.f084
> 540995533 Oct 17 09:36 gfs.t12z.pgrb2.0p25.f087
> 537682133 Oct 17 09:36 gfs.t12z.pgrb2.0p25.f090
>
> As you can see, we are missing multiple forecast hours. And, typical
> arrival times would be nearly one hour earlier for these files (we got the
> 0p25.anl file at 08:34 PDT yesterday). Are others seeing similar delays
> and missing data?
>
> Cheers,
>
> David
> --
> David Ovens e-mail: ovens@xxxxxxxxxxxxxxxxxxxx
> Research Meteorologist phone: (206) 685-8108
> Dept of Atm. Sciences plan: Real-time WRF forecasting for the
> Box 351640 Pacific Northwest
> University of Washington https://atmos.uw.edu/wrfrt
> Seattle, WA 98195 Weather Graphics and Loops
> https://atmos.uw.edu/~ovens/loops
>
>
> On Sun, Oct 15, 2023 at 08:56:21PM -0500, Mike Zuranski wrote:
> > On Sun, Oct 15, 2023 at 8:38???PM Herzmann, Daryl E [AGRON] <
> > akrherz@xxxxxxxxxxx> wrote:
> >
> > > Hi Mike,
> > >
> > > Thanks, that was the clue-by-four I needed to be hit with. I
> > > adjusted my LDM request and got the other half of the NDFD products
> > > now being transmitted!
> > >
> >
> > Awesome, glad you're getting everything!
> >
> >
> >
> > > PS, that screenshot is ultra fancy, is it available for us mere
> > > mortals to use?
> > >
> > > daryl
> > >
> >
> > Hey Thanks! :D
> >
> > I'm afraid not yet, it's far from ready for primetime and isn't always
> > running attm. But I've got some irons in the fire for interfaces that
> > would be more useful to you mere mortals our beloved & valued users.
> > There's a bit for us to juggle right now so I can't say much in terms
> > of timing, other than *I'm Working On It *(tm).
> >
> >
> > Best,
> > -Mike
> >
> > *Mike Zuranski*
> > Data Engineer II
> > Unidata Program Center
> > University Corporation for Atmospheric Research
> >
> >
> >
> > > ________________________________________
> > > From: Mike Zuranski <zuranski.wx@xxxxxxxxx>
> > > Sent: Sunday, October 15, 2023 3:18 PM
> > > To: Herzmann, Daryl E [AGRON]
> > > Cc: CONDUIT Users List; ncep.pmb.dataflow@xxxxxxxx; Unidata CONDUIT
> > > Support
> > > Subject: Re: [conduit] NCO Conduit Transition to RHEL8
> > >
> > > Daryl,
> > >
> > > Looks like it's there to me, all under the
> > > "/common/data/products/NDFD/grib2/..." header. 20Z coming in now
> actually:
> > >
> > > [image.png]
> > >
> > > Are you not seeing these?
> > >
> > > -Mike
> > >
> > >
> > > Mike Zuranski
> > > Data Engineer II
> > > Unidata Program Center
> > > University Corporation for Atmospheric Research
> > >
> > >
> > > On Sun, Oct 15, 2023 at 1:00???PM Herzmann, Daryl E [AGRON] <
> > > akrherz@xxxxxxxxxxx<mailto:akrherz@xxxxxxxxxxx>> wrote:
> > > Hi Mike,
> > >
> > > Thanks, but I am not seeing either of those two. Do you have
> > > examples from today?
> > >
> > > daryl
> > >
> > > ________________________________________
> > > From: Mike Zuranski
> > > <zuranski.wx@xxxxxxxxx<mailto:zuranski.wx@xxxxxxxxx>>
> > > Sent: Sunday, October 15, 2023 11:52 AM
> > > To: Herzmann, Daryl E [AGRON]
> > > Cc: CONDUIT Users List; ncep.pmb.dataflow@xxxxxxxx<mailto:
> > > ncep.pmb.dataflow@xxxxxxxx>; Unidata CONDUIT Support
> > > Subject: Re: [conduit] NCO Conduit Transition to RHEL8
> > >
> > > Hi Daryl,
> > >
> > > For some unknown amount of time, there has been duplicate NDFD data
> > > on CONDUIT that would come with two separate product IDs, though the
> > > data was identical. Here is an example of the differences; the two
> > > products were otherwise bytewise identical:
> > >
> > > /common/data/products/NDFD/grib2/YCUZ98_KWBN_121351.grib2
> > > !grib2/nwstg/NWS_0/#000/202310121400F058/SPED/10 m HGHT! 000042
> > > data2/ndfd/YCUZ98_KWBN_121351.grib2
> > > !grib2/nwstg/NWS_0/#000/202310121400F058/SPED/10 m HGHT! 000042
> > >
> > > I have been in talks with NCEP about this and I know they were
> > > looking into it; while I wasn't aware of any plans, it would appear
> > > they have consolidated/rectified the different product IDs with this
> > > update. From my records it looks like the latter pattern stopped
> > > around 10/12 14Z, the former pattern continues to be used.
> > >
> > > If you search using the former product ID pattern, do you see all
> > > the expected NDFD products then?
> > >
> > > -Mike
> > >
> > >
> > > Mike Zuranski
> > > Data Engineer II
> > > Unidata Program Center
> > > University Corporation for Atmospheric Research
> > >
> > >
> > > On Sun, Oct 15, 2023 at 10:58???AM Herzmann, Daryl E [AGRON] <
> > > akrherz@xxxxxxxxxxx<mailto:akrherz@xxxxxxxxxxx><mailto:akrherz@iasta
> > > te.edu <mailto:akrherz@xxxxxxxxxxx>>> wrote:
> > > Greetings,
> > >
> > > Since this transition, I am noticing that NDFD data is not flowing
> > > over CONDUIT. Anybody else able to verify?
> > >
> > > daryl
> > >
> > > ________________________________________
> > > From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx<mailto:
> > > conduit-bounces@xxxxxxxxxxxxxxxx><mailto:conduit-bounces@xxxxxxxxxxx
> > > r.edu <mailto:conduit-bounces@xxxxxxxxxxxxxxxx>>> on behalf of NCEP
> > > Dataflow via conduit
> > > <conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx
> > > ><mailto:conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx>>>
> > > Sent: Wednesday, October 11, 2023 2:18 PM
> > > To: CONDUIT Users List
> > > Cc: Unidata CONDUIT Support
> > > Subject: Re: [conduit] NCO Conduit Transition to RHEL8
> > >
> > > Hello CONDUIT Users,
> > >
> > > The start date of the transition of NCO's Conduit servers to RHEL8
> > > was pushed back by one day due to Critical Weather Day on Tuesday
> 10/10 .
> > > Therefore our migration to the new systems began today, Wednesday
> > > October 11th. The transition will still take place over three days
> > > with our operational servers now planned to migrate tomorrow,
> > > Thursday October 12th at ~1300z.
> > >
> > > Changes on your end are not necessary and we are anticipating a
> > > seamless transition. If however you do encounter issues that are not
> > > resolved by an LDM restart, please report them to NCO Dataflow (
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>>>) as
> > > soon as possible with a subject line that includes the text "CONDUIT
> Issue".
> > >
> > > Thank you
> > > --
> > > NCO Dataflow Team
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>>>
> > >
> > >
> > > On Fri, Oct 6, 2023 at 12:44???PM Dustin Sheffler - NOAA Federal <
> > > dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx><mailto:
> > > dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx>><mailto:
> > > dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx><mailto:
> > > dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx>>>> wrote:
> > > Attention CONDUIT Users,
> > >
> > > Beginning Tuesday, October 10th, 2023 at 1300z, NCO Dataflow will
> > > begin the migration of our CONDUIT systems to RHEL 8. The transition
> > > will take place over three days with our operational servers
> > > migrating on Wednesday, October 11th at 1300z. Changes on your end
> > > are not necessary and we are anticipating a seamless transition. If
> > > however you do encounter issues that are not resolved by an LDM
> > > restart, please report them to NCO Dataflow (
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>>>) as
> > > soon as possible with a subject line that includes the text "CONDUIT
> Issue".
> > >
> > > Thank you
> > > --
> > > NCO Dataflow Team
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx><mailto:
> > > ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>>>
> > >
> > > To sign up for the NOMADS/FTPPRD Users List:
> > > https://urldefense.com/v3/__https://www.lstsrv.ncep.noaa.gov/mailman
> > > /listinfo/ncep.list.nomads-ftpprd__;!!K-Hz7m0Vt54!jO7oxor844lf1mx6eZ
> > > 1lLm9IiXb2tQwuasmf7fpm4zYAovUgKafvlTkGMqh8bT4HAUS2ZVJnKh6djGcW_Q$
> > > <tel:%28301%29%20683-1400>
> > > _______________________________________________
> > > NOTE: All exchanges posted to Unidata maintained email lists are
> > > recorded in the Unidata inquiry tracking system and made publicly
> > > available through the web. Users who post to any of the lists we
> > > maintain are reminded to remove any personal information that they
> > > do not want to be made public.
> > >
> > >
> > > conduit mailing list
> > > conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx><mailto:
> > > conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx>>
> > > For list information or to unsubscribe, visit:
> > > https://urldefense.com/v3/__https://www.unidata.ucar.edu/mailing_lis
> > > ts/__;!!K-Hz7m0Vt54!jO7oxor844lf1mx6eZ1lLm9IiXb2tQwuasmf7fpm4zYAovUg
> > > KafvlTkGMqh8bT4HAUS2ZVJnKh5KsIwTTg$
> > >
>
> > _______________________________________________
> > NOTE: All exchanges posted to Unidata maintained email lists are
> > recorded in the Unidata inquiry tracking system and made publicly
> > available through the web. Users who post to any of the lists we
> > maintain are reminded to remove any personal information that they do
> > not want to be made public.
> >
> >
> > conduit mailing list
> > conduit@xxxxxxxxxxxxxxxx
> > For list information or to unsubscribe, visit:
> > https://urldefense.com/v3/__https://www.unidata.ucar.edu/mailing_lists
> > /__;!!K-Hz7m0Vt54!jO7oxor844lf1mx6eZ1lLm9IiXb2tQwuasmf7fpm4zYAovUgKafv
> > lTkGMqh8bT4HAUS2ZVJnKh5KsIwTTg$
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are recorded
> in the Unidata inquiry tracking system and made publicly available through
> the web. Users who post to any of the lists we maintain are reminded to
> remove any personal information that they do not want to be made public.
>
>
> conduit mailing list
> conduit@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are recorded
> in the Unidata inquiry tracking system and made publicly available through
> the web. Users who post to any of the lists we maintain are reminded to
> remove any personal information that they do not want to be made public.
>
>
> conduit mailing list
> conduit@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
--
Dataflow Team Lead (Acting)
NCEP Central Operations
College Park, Maryland 20740
(240)-736-8261
