Correct, thanks Roy.
Anything thats part of the file format is handled on the server.
Standard HTTP deflate (zip) compression is used to send data across the
wire, if client allows. the ncstream protocol is experimenting with more
complicated schemes, but is not ready for general use.
Figuring out optimal chunking in netcdf4 is now the right game to play.
It depends on the subset request, so you want to optimize your common
requests.
the netcdf group would welcome user experience reports and timings.
posting to this list lets us all mull over the issues.
thanks!
John
On 9/6/2012 6:18 PM, Roy Mendelssohn wrote:
Hi Ansley:
I believe the answer is "yes", sort of. Remember, THREDDS unless you are using
the netcdf subset service or another experimental service John C. is working on, does not
deliver netcdf files, it delivers data through OPeNDAP, which does not have this
compression as part of the description of the protocol. So the data must be uncompressed
and translated to be sent. however, many OpeNDAP servers, and I believe this is true of
TDS, if the client says it can receive zipped files, will zip the OPeNDAP stream and then
that will be decompressed when it reaches the client. One question we (ERD) has been
experimenting with is the read penalty with using netcdf4 files in conjunction with
OPeNDAP, and what chunking/compression settings have the best tradeoff between
performance and space
I am probably off on some of this, but I believe it is basically correct.
-Roy
On Sep 6, 2012, at 5:04 PM, Ansley Manke wrote:
Hi,
Here's a question from a user of Ferret software, which is built with netcdf4
libraries. When opening and reading data from a THREDDS server, where the data
is in netcdf-4 format, using compression, where does the decompression takes
place:
- at the THREDDS server and the uncompressed data are delivered over the web,
- at the local machine, the compressed data are delivered over the web and are
uncompressed at the local machine?
-Ansley
_______________________________________________
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
**********************
"The contents of this message do not reflect any position of the U.S. Government or
NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097
e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
_______________________________________________
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
