Thanks for responding, Sean. Rather than a "push" system (RSS) that
users would have to subscribe to, just having a service that users could
hit ("pull") whenever they want to hit a new dataset before running
their script would be nice.
Don
On 4/11/14 10:43 AM, Sean Arms wrote:
Hi Chris,
This is a very good question, and one we have explored here at Unidata
for quite some time.
In the TDS, we try to wait until the file on disk has not change for a
certain period of time (five or ten minutes...can't recall the exact
time right now) before changing over the url for latest. The reason we
do this is based on how the data delivery system, the LDM, works.
In the LDM, there is no way to know in general when a model run (or
other dataset) has finished coming into a machine and there isn't a
mechanism to calculate the expected file size a priori; for some data
sources, not even the data producer can tell this as data is sent as
it's being generated. Because of these factors, revealing a percent
complete would not be possible either.
What we have talked about is having a subscription based service of
sorts to ping subscribed users when a dataset has been updated (think
rss for data). However, this is only in the discussion stage at this
point and it's not clear this is the direction we will go.
Cheers,
Sean
On 4/11/14, 10:02 AM, Don Murray (NOAA Affiliate) wrote:
Hi Chris-
There really isn't a way to know this from the IDV. I think the TDS
has some
hueristics for when the "latest" link is switched based on the historical
arrival time of data. A service that could be queried to indicate the
% complete
would be useful if it doesn't already exist. Perhaps someone from the TDS
development group could chime in.
Don
On 4/10/14 11:44 PM, Chris Noble wrote:
Greetings,
Is there any way to find out when a run is complete/available in the
Catalog, or alternatively, what do others use as a safe/reliable time to
trigger a bundle to catch the latest run for the NCEP Global 0p5?
Background: I have some scheduled scripts to load bundles and make
images,
but figuring out when a new run is available is somewhat trial and
error.
I have been triggering about 5 hours after the run time, but lately
the
00Z run seems to be a little slower to become available (eg today,
triggering at 5h15m past 00Z still loaded the old 18Z run, but 10min
later
the 00Z was available). I'm wondering if there's an easy way to
investigate
availability times so I can improve/tweak my scheduling.
Appreciate any advice.
Thanks and regards,
Chris.
_______________________________________________
idvusers mailing list
idvusers@xxxxxxxxxxxxxxxx
For list information, to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
_______________________________________________
idvusers mailing list
idvusers@xxxxxxxxxxxxxxxx
For list information, to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
Don Murray
NOAA/ESRL/PSD and CIRES
303-497-3596
http://www.esrl.noaa.gov/psd/people/don.murray/
