you are using netcdf-4 format i think. this has much more complex storage
objects than what you are used to with netcdf-3 format.
LB wrote:
Hi,
I'm very new to netCDF and I was surprised by the size of files
generated by netCDF.
Here is the header of the netCDF file :
ncdump -h cycle.nc <http://cycle.nc>
netcdf cycle {
dimensions:
Regime = 3 ;
group: Regimes {
variables:
float T41(Regime) ;
float TTR(Regime) ;
float N1(Regime) ;
} // group Regimes
group: Cycles {
dimensions:
Cycle = 5 ;
SousCycle = 11 ;
variables:
byte Type(Cycle) ;
short Nb_lignes(Cycle) ;
short Indice(SousCycle) ;
short Repeat(SousCycle) ;
float Duree(SousCycle) ;
} // group Cycles
group: Simulation {
dimensions:
Temps = UNLIMITED ; // (19168 currently)
variables:
float Temps(Temps) ;
byte Regime(Temps) ;
float T41(Temps) ;
float e_p1(Temps) ;
float Temps_Touche ;
} // group Simulation
}
The generated file is 3.1 Mo and so 50% heavier than I had expected.
Here is how I calculated the expected file size :
name Dimension cell size Nb cell variable size
group: Regimes {
float T41 Regime 32 3 96
float TTR Regime 32 3 96
float N1 Regime 32 3 96
group: Cycles {
byte Type Cycle 8 5 40
short Nb_lignes Cycle 16 5 80
short Indice SousCycle 16 11 176
short Repeat SousCycle 16 11 176
float Duree SousCycle 32 11 352
group: Simulation {
float Temps Temps 32 19168 613376
byte Regime Temps 8 19168 153344
float T41 Temps 32 19168 613376
float e_p1 Temps 32 19168 613376
float Temps_Touche 32 1 32
total size 1994616 Bits
soit 1.99 Mo
Is there anything I forgot ?
Besides, I would like to know how to if there is any efficient way to
reduce this size:
- I've tried to use deflate_level=1 to 9 in nf90_def_var for the
variables in the simulation group but it didn't change the file size.
Is there anything else to do to to let netcdf compress data ? gzip seems
to compress the resulting file quite well.
- Is there a way to store 16bits float to compress data (with loss) in
order to reduce the size of data where we have a limited resolution ?
Best regards,
--
Loic
------------------------------------------------------------------------
_______________________________________________
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
