Hi David,
I think you must use the nc_get_var_*() function matching the storage
type of the variable, these methods are not meant to be used for type
conversions.
Your NetCDF file states that the data type for the phycocyanin variable
is float, so you should use nc_get_var_float to read its values.
On my system float values are stored on 32 bits whereas double values
are stored on 64 bits.
When you ask the library to read the content of the variable as double,
it loads 64 bits from the file and interpret them as a double value.
Since phycocyanin values are stored on 32 bits, it means that you
actually read two float values from the file but interpret their binary
representation as a single double value.
It explains the strange values you get with nc_get_var_double:
- the variable only contains fill values, i.e. -9999.f
- on my machine the hexadecimal representation for -9999.f is 0xc61c3c00
- if you read two adjacent floats with a -9999.f value, you get
0xc61c3c00c61c3c00.
- if you interpret these 64 bits as a double value you get:
-559239646634519513659653226496.000000
In your test program if you print more values of ddata you will also see
that you get the strange value for the 720 first items in the array, and
then starting from the 721st you only get 0: since you actually need two
phycocyanin values for each 64bits item of the array, you only have
enough data to fill the first half of ddata.
Cheers,
Sylvain
On 09/09/2020 00:30, David Pierce via netcdfgroup wrote:
Hello netcdf-ers,
I had a user of the R ncdf4 package alert me to an odd and perplexing
apparent bug in the netcdf library. It is triggered by the netcdf file
you can download here:
http://cirrus.ucsd.edu/~pierce/tmp/mendota_buoy.2018-11-08.nc
This file has the following variable (among others):
float phycocyanin(time) ;
phycocyanin:_FillValue = -9999.f ;
phycocyanin:units = "RFU" ;
phycocyanin:long_name = "Phycocyanin" ;
phycocyanin:_Storage = "chunked" ;
phycocyanin:_ChunkSizes = 1440 ;
phycocyanin:_DeflateLevel = 4 ;
phycocyanin:_Shuffle = "true" ;
phycocyanin:_Endianness = "little" ;
'ncdump' reports all the data as missing:
phycocyanin = _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _,
_, _,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _,
_, (etc.)
When I read the data into a floating point array using the C library
using nc_get_var_float() then I get the expected values of -9999.0.
However when I read it into a double precision array with
nc_get_var_double() then I get strange values. Here is the output of
the little test program that I've appended below:
Sizeof float: 4 double: 8
======== DOUBLE ========
-559239646634519513659653226496.000000
-559239646634519513659653226496.000000
-559239646634519513659653226496.000000
-559239646634519513659653226496.000000
-559239646634519513659653226496.000000
-559239646634519513659653226496.000000
-559239646634519513659653226496.000000
-559239646634519513659653226496.000000
-559239646634519513659653226496.000000
-559239646634519513659653226496.000000
======== FLOAT ========
-9999.000000 -9999.000000 -9999.000000 -9999.000000 -9999.000000
-9999.000000 -9999.000000 -9999.000000 -9999.000000 -9999.000000
This seems like a bug, unless I'm overlooking something obvious in the
test code (always possible). The nc_get_var_double() call is supposed
to convert the netcdf file's values into double precision and store
them in the provided double precision array, right? That's my
understanding anyway. I can't see why they are not -9999.00's in the
resultant array.
Any thoughts are appreciated,
--Dave
Test program:
#include <stdio.h>
#include <stdlib.h>
#include "netcdf.h"
void main( int argc, char *argv[] )
{
int err, ncid, varid;
size_t nt;
double *ddat;
float *fdat;
nt = 1440; /* just hardcode for test */
printf( "Sizeof float: %ld double: %ld\n", sizeof(float),
sizeof(double) );
err = nc_open("mendota_buoy.2018-11-08.nc
<http://mendota_buoy.2018-11-08.nc>", 0, &ncid );
if( err != 0 ) {
printf( "err open = %d\n", err );
exit(-1);
}
err = nc_inq_varid( ncid, "phycocyanin", &varid );
if( err != 0 ) {
printf( "err inq_varid = %d\n", err );
exit(-1);
}
/* Make room for both double and floating dat */
ddat = (double *)malloc( sizeof(double) * nt );
fdat = (float *)malloc( sizeof(float ) * nt );
/* Read into double array. Supposed to convert to double...? */
err = nc_get_var_double( ncid, varid, ddat );
if( err != 0 ) {
printf( "err get_var_double = %d\n", err );
exit(-1);
}
printf( "======== DOUBLE ========\n" );
for( int ii=0; ii<10; ii++ )
printf( "%lf ", ddat[ii] );
printf( "\n" );
/* === Do same thing, but for float === */
err = nc_get_var_float( ncid, varid, fdat );
if( err != 0 ) {
printf( "err get_var_float = %d\n", err );
exit(-1);
}
printf( "======== FLOAT ========\n" );
for( int ii=0; ii<10; ii++ )
printf( "%f ", fdat[ii] );
printf( "\n" );
}
-------------------------------------------------------------------
David W. Pierce Division of Climate, Atmospheric Science, and Physical
Oceanography Scripps Institution of Oceanography (858) 534-8276
(voice) / (858) 534-8561 (fax) dpierce@xxxxxxxx
-------------------------------------------------------------------
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
https://www.unidata.ucar.edu/mailing_lists/
