Re: [thredds] wget and thredds server

Hi Patrick,

What is the version of the TDS with which you are trying to synch? The
TDS has supported lastModified for quite some time, so it would have to
be a pretty old version to not support it. But that's the first thing
that comes to mind.

You could also try looking at the HTTP "Last-Modified" response headers
you are getting back. You can do this with the wget "--save-headers"
option. Or, in Firefox, by using a plugin like Live HTTP Headers.

John, any chance the file caching in the TDS could be masking a change
to the last modified timestamp?

Ethan

On 5/12/2012 3:13 PM, John Caron wrote:
> On 5/11/2012 7:33 AM, Patrick Brockmann wrote:
>> Hi all,
>>
>> I would like to synchonize some netcdf files available from
>> a THREDDS Data Server with a local repository.
>>
>> I have used the wget command and HTTPServer access from
>> the TDS but it seems that the server does not provide enough
>> or correct informations to use properly the --timestamp option
>> of wget (very useful to synchronize).
>> Read http://www.gnu.org/software/wget/manual/wget.html#Time_002dStamping
>>
>> The method works for an old OPeNDAP server but not with 2 different
>> Thredds Data Servers
>>
>> Try:
>> $ wget -N http://dods.ipsl.jussieu.fr/brocksce/tmp/tyty.txt
>> If you retry you get the message
>> ----> Server file no newer than local file `tyty.txt' -- not retrieving.
>> if the 2 files (input and local) are not the same from date or size, the
>> download is done again.
>> If you modify the file by adding a character, you get
>> ----> The sizes do not match (local 43) -- retrieving.
>> If you apply a touch -t 201106230000 on the file to see what happens
>> ----> Remote file is newer, retrieving.
>>
>> I would like to get the same feature (synchro on date and size) when I
>> download from a TDS/HTTPServer.
>>
>> Any help welcome on this synchronization.
>>
>> Regards
>> Patrick
>>
> 
> Hi Patrick:
> 
> When compression is enabled, which it often is on a TDS, the
> Content-Length header is not sent, since the response is variable length.
> Apparently wget needs that to synch (?) Seems like it should just use
> Last-Modified.
> 
> I dont see a workaround without turning off compression.
> 
> John
> 
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/

-- 
Ethan Davis                                       UCAR Unidata Program
edavis@xxxxxxxxxxxxxxxx                    http://www.unidata.ucar.edu