Nilesh Lahoti a écrit :
...
The netCDF library works great for us. However, I came across with one
particular issue of netCDF and would like to discuss if there are any
solution to this problem or something that can do to make its
performance better. When we process emissions for our three
dimensional grid of size (172 x 172 x 22) for 24 hours time period
having hourly data, the file size is around 1 gigabyte(GB). There are
several cells that have zero values and therefore the floating point
value for pollutants in netCDF file has zero values. When we use gzip
utility on unix to compress this files, the file size become almost 10
MB which saves us 99% of disk space. Now the question arise that if
the netCDF is most compress scientific format, than is it possible to
suppress this zero values of the floating point variable or is there
any switch that can be used to handle zero values and reduce file size
by any chance.
...
Hello,
You can also have a look at this page:
http://snow.cit.cornell.edu/noon/z_netcdf.html
This is a modification of the lowlevel file layer of netcdf library
which allow
compressing and uncompressing data on the fly (in fact the whole netCDF
structure).
The compressed files produced are not readable by the standard library
and it
may be difficult to share those file. However there is a command line
utility
(nczip) which allow converting the compressed file to a 'normal' netCDF
file and any classical netCDF file to a compressed one. The "magic
number" is not the same so there is no risk the library being confused
about the file type (CDF for standard netCDF files CDZ for compressed
ones).
The advantage is that there is no interface modification of the API (except
one attribute, NC_COMPRESS, on opening if you want to create
compressed files directly in your program) and this allow you to wait
until netCDF-4.
The annoyance is that all your programs must be relinked with the modified
library.
I have used this patch with success years ago but I am not sure that it will
work with recent versions of the netCDF library (specially the 64 bits
extension known as file format version 2).
--
Philippe Poilbarbe
CLS - Space Oceanography Division
==============================================================================
To unsubscribe netcdfgroup, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================
