[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Updating Aggregations in TDS



Are files also deleted, or only added? Do files ever get appended to or 
modified?

Roy Mendelssohn wrote:
> When a file is added to a directory, a flag is set in a table that says
> it has been added.   We would rather poll that table  (much faster as it
> doesn't need to open a read a slew of netcdf files).  I can have Bob
> write a program in Java to do the required recheck operations if you can
> point us to the appropriate steps to be taken for the netcdf-java library.
> 
> In the near future, both the information to generate our catalog.xml and
> what files are present and/or have been added will be in a database.
> 
> -Roy
> 
> On Oct 30, 2008, at 9:55 AM, John Caron wrote:
> 
>> We are going to add this kind of functionality to 4.0, currently it
>> doesnt exist.
>>
>> How do you detect when data changes?
>>
>> Roy Mendelssohn wrote:
>>> Hi John:
>>>
>>> I am glad we understand how recheckEvery works.  It is consistent with
>>> what we were experiencing.
>>>
>>>> basically you want to do the recheck in the background, and then
>>>> accept whatever is there when a request comes in? Or you want to
>>>> detect whenever the directory changes?
>>>>
>>>
>>> Because other things, such as web page displays, need to be updated when
>>> data is added, we already have a system where we know when data have
>>> been added to a dataset.  What we would like to do is at that point
>>> force a recheck for that dataset, and hopefully only for that dataset,
>>> rather than the entire catalog.  If we can do that, we probably will
>>> turn off the recheckEvery because we know every dataset will be up to
>>> date.
>>>
>>> Does this makes sense?  If need be, we can write a script in Java to do
>>> the above, if we can get advice on the best way to do this.
>>>
>>> Thanks,
>>>
>>> -Roy
>>>
>>> **********************
>>> "The contents of this message do not reflect any position of the U.S.
>>> Government or NOAA."
>>> **********************
>>> Roy Mendelssohn
>>> Supervisory Operations Research Analyst
>>> NOAA/NMFS
>>> Environmental Research Division
>>> Southwest Fisheries Science Center
>>> 1352 Lighthouse Avenue
>>> Pacific Grove, CA 93950-2097
>>>
>>> e-mail: address@hidden (Note new e-mail address)
>>> voice: (831)-648-9029
>>> fax: (831)-648-8440
>>> www: http://www.pfeg.noaa.gov/
>>>
>>> "Old age and treachery will overcome youth and skill."
>>> "From those who have been given much, much will be expected"
> 
> **********************
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> 1352 Lighthouse Avenue
> Pacific Grove, CA 93950-2097
> 
> e-mail: address@hidden (Note new e-mail address)
> voice: (831)-648-9029
> fax: (831)-648-8440
> www: http://www.pfeg.noaa.gov/
> 
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"