Re: [ldm-users] volume scan metadata/VCP question

Polling just isn't fast enough for us.  That site is also not always
up-to-date (tends to lag).

My thinking is more along the lines of leveraging the data we already have
(or can get) via LDM.  We already have a good sized fleet of powerful
servers, and being able to leverage the (damn near) real time nature of the
data allows us to yield results and data MUCH more quickly.

Right now, it typically takes less than 5 seconds for us to receive a
volume scan, read out the VCP, and dispatch the messages and push the data
based on what triggers/subscribers are in the database.


On Sat, Apr 26, 2014 at 3:43 PM, Rob Dale <rdale@xxxxxxxxxxxx> wrote:

> Sounds like a neat project. Can't you just check the ROC website every few
> minutes?
>
> http://www.roc.noaa.gov/wsr88d/operations/VCP.aspx
>
> Or ingest the GSM product as I think a new one is issued when the
> radarsite changes modes...
>
> Rob
>
> On Apr 26, 2014, at 4:27 PM, Blair Trosper <
> blair.trosper@xxxxxxxxxxxxxxxxxxx> wrote:
>
> Some of you may recall a past thread similar to this, and I'm sort of
> looking for ideas -- so if anyone has something better than what I
> describe, I would love to know.
>
> We've got a service, which we're about to open up to the world for free,
> that notifies a subscriber via SMS when a RADAR site goes from 31/32 into
> an "active" mode (you can choose the granularity of which VCPs you'd want
> notifications for, just in case you didn't care about VCP 121, for
> example).  It's a very good 'heads up' that things are about to become
> eventful.  (The service can also send emails, push the data to a URL
> endpoint, and even push it to apps.)
>
> To that end, we keep track of the metadata (VCP, timestamp, etc) from all
> the volume scans we ingest.  At present, this requires us to use custom C++
> code to actually decipher the L2 or L3 scans to locate the data.  While
> this generally works, the problem in this method is two fold:
> - About 10% of the scans do not conform to the storage structure outlined
> by the ROC, requiring us to write more kludges than I'd care to admit.
>  (ROC hasn't bothered to return communications regarding bugs we've
> reported in this regard.)
> - This is computationally expensive.  (We're doing more than just pulling
> the metadata, such as plotting the smoothed data over Google Maps, but you
> can imagine than reading the entire L2 file just to find out the VCP is
> inefficient.)
>
> I've always wished there was a free text message type product on IDS/HDS
> that would send the metadata along with each volume scan, but that doesn't
> seem to exist.  (We've requested it, but apparently the demand for this is
> low.)
>
> The closest thing I can find is something similar to this product:
> http://www.rwic.und.edu/weather/text/KMAF/SDUS84.wmo
>
> Its NNN is DPA and follows the WMO ID pattern of SDUS##...however, it's
> only sent out hourly.
>
> Can anyone think of a better way...or perhaps even point out a product I'm
> perhaps not aware of?
>
> ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙
>  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙
> Blair Trosper
>   Updraft Networks / Weather Data
>   NOC:  469-844-5440
>   Early Watch Notifications:  http://twitter.com/weatherwatches
>
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
>


-- 

∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙
 ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙  ∙
Blair Trosper
  Updraft Networks / Weather Data
  NOC:  469-844-5440
  Early Watch Notifications:  http://twitter.com/weatherwatches
  • 2014 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: