Sorry, included an echo you don't need. Should be:
--- cut here ---
#!/bin/bash
#assessing the next hour based upon 10 minute look forward as described by
NOAA
datahome="${1}"
datafile="${2}"
minutes="${3}"
mkdir -p ${datahome}
dateonly=`echo "${datafile}" | grep -E -o "[0-9]*"`
if [ "${#dateonly}" -eq "8" ]; then
outputdate=`date -d "20${datafile:0:6} ${datafile:6:2}:${minutes}:00 Z
+10minutes" -u "+%y%m%d%H"`
elif [ "${#dateonly}" -eq "10" ]; then
outputdate=`date -d "${datafile:0:8} ${datafile:8:2}:${minutes}:00 Z
+10minutes" -u "+%Y%m%d%H"`
else
exit
fi
outputfile=`echo "${datafile}" | sed -e "s|${dateonly}|${outputdate}|g"`
cat >> ${datahome}/${outputfile}
--- cut here ---
-
Stonie Cooper, PhD
Software Engineer III
NSF Unidata
cooper@xxxxxxxx
On Wed, Jan 29, 2025 at 11:50 AM Stonie Cooper <cooper@xxxxxxxx> wrote:
> Using Pete's example as the template:
>
> DDS|IDS ^S[AP].* .... ([0-3][0-9])([0-2][0-9])
> FILE /usr/local/ldm/data/weather/(\1:yy)(\1:mm)\1\2.METAR
>
> Which works for everything but the new messages, for the new 99 headed
> data, *an* option:
>
> DDS|IDS ^S[AP]US99 .... ([0-3][0-9])([0-2][0-9])([0-5][0-9])
> PIPE /usr/local/ldm/util/saus99_binning.sh
> /usr/local/ldm/data/weather/ (\1:yy)(\1:mm)\1\2.METAR \3
>
> Where /usr/local/ldm/util/saus99_binning.sh *could* be something like:
>
> --- cut here ---
> #!/bin/bash
> #assessing the next hour based upon 10 minute look forward as described by
> NOAA
> datahome="${1}"
> datafile="${2}"
> minutes="${3}"
>
> mkdir -p ${datahome}
> dateonly=`echo "${datafile}" | grep -E -o "[0-9]*"`
> echo "${#dateonly}"
>
> if [ "${#dateonly}" -eq "8" ]; then
> outputdate=`date -d "20${datafile:0:6} ${datafile:6:2}:${minutes}:00 Z
> +10minutes" -u "+%y%m%d%H"`
> else
> outputdate=`date -d "${datafile:0:8} ${datafile:8:2}:${minutes}:00 Z
> +10minutes" -u "+%Y%m%d%H"`
> fi
>
> outputfile=`echo "${datafile}" | sed -e "s|${dateonly}|${outputdate}|g"`
>
> cat >> ${datahome}/${outputfile}
> --- cut here ---
>
> If you do this, you will want to edit your original pattern to exclude the
> S[AP]US99 . . . for everything else.
>
> Or you can pipe everything through the script.
> -
> Stonie Cooper, PhD
> Software Engineer III
> NSF Unidata
> cooper@xxxxxxxx
>
>
> On Wed, Jan 29, 2025 at 11:07 AM Mike Zuranski <zuranski.wx@xxxxxxxxx>
> wrote:
>
>> Morning all,
>>
>> I received a response to my inquiry:
>>
>> First thank you for reaching out to us on this matter. *This was an
>> unintentional change* but I can describe how it happened. The SAUS70 &
>> SAUS80 METAR collectives that were previously disseminated over the
>> SBN/NOAAPort prior to this change would get disseminated every 10 minutes,
>> including the latest observations in that period. For example, many
>> observations are time stamped at 55 after the hour, which are then included
>> in the top of the hour SAUS70 METAR collective. However, SAUS99 METAR
>> collectives are issued upwards of several times a minute containing the
>> latest observation. We get a big benefit here in more timely observational
>> data. In addition to the improved timeliness, SAUS70 & SAUS80 require
>> manual intervention to add sites whereas sites automatically are added to
>> SAUS99. Given these two benefits, *we hope the end-users applications
>> are able to be updated to work with SAUS99*.
>>
>> Feel free to let us know if you have any additional questions or concerns!
>>
>> So there it is, this is the new norm and expected behavior moving forward.
>>
>> If this is negatively impactful to community members enough, my personal
>> opinion is this might be addressed the easiest if Unidata could offer a
>> decoder of sorts to adapt to this new norm. I can't say how widespread any
>> impacts are, all I've heard is what's been said here. And of course I'm
>> sure Unidata staff are hard at work as always and it's not my place to
>> suggest how their energies should be spent. But that's where my head is
>> heading.
>>
>> Best,
>> -Mike
>>
>> On Tue, Jan 28, 2025 at 2:47 PM Sebenste, Gilbert <sebensteg@xxxxxxx>
>> wrote:
>>
>>> This is what I see with the raw NADIN/WMSCR feed, which I’ve had access
>>> to. These are indeed the RAW METARs sent without any processing from the
>>> NWS, which slowed things down and had duplicate observations. The upsides
>>> are that new METARs show up immediately, and assuming the feed to the NWS
>>> is timely, you’ll get them 1-10 minutes faster than before.
>>>
>>>
>>>
>>> Gilbert Sebenste
>>>
>>> Meteorology Support Analyst
>>>
>>>
>>>
>>> *From:* ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> *On Behalf Of *Pete
>>> Pokrandt
>>> *Sent:* Monday, January 27, 2025 2:20 PM
>>> *To:* Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx>;
>>> ldm-users@xxxxxxxxxxxxxxxx; Tyle, Kevin R <ktyle@xxxxxxxxxx>
>>> *Subject:* [External] Re: [ldm-users] The S[AP]US99 update for METARs
>>> just happened
>>>
>>>
>>>
>>> CAUTION: This email originated from outside of COD’s system. Do not
>>> click links, open attachments, or respond with sensitive information unless
>>> you recognize the sender and know the content is safe.
>>>
>>>
>>>
>>> All,
>>>
>>>
>>>
>>> I just noticed a curiosity about the new SAUS99 METAR format that I'm
>>> not immediately sure how to deal with. I am getting the SAUS99 and SPUS99
>>> reports, but the time on the WMO header is different, and that's messing up
>>> my data filing in the LDM.
>>>
>>>
>>>
>>> To save data for the old Pete Neilley weather program (yes, I do still
>>> use that..) I use the following
>>>
>>>
>>>
>>> DDS|IDS ^S[AP].* .... ([0-3][0-9])([0-2][0-9]) FILE
>>> /usr/local/ldm/data/weather/(\1:yy)(\1:mm)\1\2.METAR
>>>
>>>
>>>
>>> Note that it is using the DDMM in the WMO product header (the
>>> ([0-3][0-9])([0-2][0-9]) part ) to determine the day/hour of the file to
>>> put it in.
>>>
>>>
>>>
>>> As we all know, the METAR times are a bit before the actual hour they
>>> are valid for (e.g. the METAR with a time of 1253 is actually the 13 UTC
>>> ob, and should be put in the 13 UTC file.
>>>
>>>
>>>
>>> With the old format (SAUS70/80/90) the time on the WMO header actually
>>> was the correct time (1300) for the 1253 ob.
>>>
>>>
>>>
>>> SAUS80 KWBC 271300 RRB^M^M
>>>
>>> METAR^M^M
>>>
>>>
>>>
>>> KMSN 271253Z 23007KT 200V270 10SM FEW180 FEW250 M07/M14 A2991 RMK^M^M
>>>
>>> AO2 SLP141 T10721144 $=^M^M
>>>
>>>
>>>
>>>
>>>
>>> However, with the new format (SAUS99) the time on the WMO is generally
>>> earlier, so in the example here, the 1753 Ob (which should be in the 18 UTC
>>> file) ends up in the 17 UTC file.
>>>
>>>
>>>
>>> SAUS99 KWBC 271756 RRH^M^M
>>>
>>> METAR^M^M
>>>
>>>
>>>
>>> KMSN 271753Z 25011G23KT 10SM BKN160 BKN250 04/M12 A2973 RMK AO2^M^M
>>>
>>> SLP077 T00391122 10044 21072 58033=^M^M
>>>
>>>
>>>
>>> I think I found a workaround for my purposes, but wanted to put it out
>>> there in case it's affecting anyone else..
>>>
>>>
>>>
>>> Pete
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Pete Pokrandt - System Engineer IV
>>>
>>> UW-Madison Dept of Atmospheric and Oceanic Sciences
>>>
>>> 608-262-3086 - poker@xxxxxxxxxxxx
>>> ------------------------------
>>>
>>> *From:* ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> on behalf of
>>> Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx>
>>> *Sent:* Monday, January 27, 2025 11:27 AM
>>> *To:* ldm-users@xxxxxxxxxxxxxxxx <ldm-users@xxxxxxxxxxxxxxxx>
>>> *Subject:* [ldm-users] The S[AP]US99 update for METARs just happened
>>>
>>>
>>>
>>> Howdy,
>>>
>>> A reminder that this change just happened.
>>>
>>> The NWS National Centers for Environmental Prediction (NCEP) Central
>>> Operations (NCO) creates routine collectives of observational METAR data
>>> in SAUS70 KWBC, SAUS80 KWBC, SAUS90 KWBC and their counterparts for
>>> Special observations SPUS70, SPUS80 and SPUS90. These collectives
>>> require manual intervention to add METAR stations to them. These are
>>> then distributed over the SBN. Another product, SAUS99 KWBC, and the
>>> Special observation counterpart SPUS99 KWBC contain any METAR observation
>>> received every minute. Delivering SAUS99 and SPUS99 over the SBN rather
>>> than the aforementioned collectives will provide more observational data
>>> and more timely data to users of the SBN.
>>>
>>> The replacement with SAUS99 and SPUS99 is expected to occur on or about
>>> 1200 Coordinated Universal Time (UTC), January 27, 2025. If a Critical
>>> Weather Day has been issued, this change will be postponed until 1200 UTC
>>> on the next non-Critical Weather Day.
>>>
>>>
>>> https://urldefense.com/v3/__https://www.weather.gov/media/notification/pdf_2023_24/scn24-99_add_saus99_to_sbn_aaa.pdf__;!!Mak6IKo!K44CJVSMFQBBR4h9RLswmLpN1JShZZJLM-zvWlrCQT26pq7B-6JV6h9ZIAqtSvktkgbOL-pBQRZTgA9LCC8p$
>>>
>>> So far, I have observed 265 SAUS99 products and 114 SPUS99 products
>>> cross the IDD since about 15 Z
>>>
>>> daryl
>>> _______________________________________________
>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>> recorded in the Unidata inquiry tracking system and made publicly
>>> available through the web. Users who post to any of the lists we
>>> maintain are reminded to remove any personal information that they
>>> do not want to be made public.
>>>
>>>
>>> ldm-users mailing list
>>> ldm-users@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe, visit:
>>> https://urldefense.com/v3/__https://www.unidata.ucar.edu/mailing_lists/__;!!Mak6IKo!K44CJVSMFQBBR4h9RLswmLpN1JShZZJLM-zvWlrCQT26pq7B-6JV6h9ZIAqtSvktkgbOL-pBQRZTgL9fV5rk$
>>>
>>> _______________________________________________
>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>> recorded in the Unidata inquiry tracking system and made publicly
>>> available through the web. Users who post to any of the lists we
>>> maintain are reminded to remove any personal information that they
>>> do not want to be made public.
>>>
>>>
>>> ldm-users mailing list
>>> ldm-users@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe, visit:
>>> https://www.unidata.ucar.edu/mailing_lists/
>>>
>>
>>
>> --
>> - Mike Zuranski
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web. Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> https://www.unidata.ucar.edu/mailing_lists/
>>
>
