Re: [netcdfgroup] If a compressed dataset is on a THREDDS server, where is the uncompression done?

  • To: Ansley Manke <ansley.b.manke@xxxxxxxx>
  • Subject: Re: [netcdfgroup] If a compressed dataset is on a THREDDS server, where is the uncompression done?
  • From: Roy Mendelssohn <roy.mendelssohn@xxxxxxxx>
  • Date: Thu, 6 Sep 2012 17:18:45 -0700
Hi Ansley:

I believe the answer is "yes", sort of.  Remember, THREDDS unless you are using 
the netcdf subset service or another experimental service John C. is working 
on, does not deliver netcdf files, it delivers data through OPeNDAP, which does 
not have this compression as part of the description of the protocol.  So the 
data must be uncompressed and translated to be sent.  however, many OpeNDAP 
servers, and I believe this is true of TDS, if the client says it can receive 
zipped files, will zip the OPeNDAP stream and then that will be decompressed 
when it reaches the client.  One question we (ERD) has been experimenting with 
is the read penalty with using netcdf4 files in conjunction with OPeNDAP, and 
what chunking/compression settings have the best tradeoff between performance 
and space

I am probably off on some of this, but I believe it is basically correct.

-Roy


On Sep 6, 2012, at 5:04 PM, Ansley Manke wrote:

> Hi,
> Here's a question from a user of Ferret software, which is built with netcdf4 
> libraries. When opening and reading data from a THREDDS server, where the 
> data is in netcdf-4 format, using compression, where does the decompression 
> takes place:
> 
> - at the THREDDS server and the uncompressed data are delivered over the web,
> - at the local machine, the compressed data are delivered over the web and 
> are uncompressed at the local machine?
> 
> -Ansley
> 
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/ 

**********************
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.



  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: