Re: [thredds] Problem serving datasets with ushort data types via THREDDS OPeNDAP

• To: thredds@xxxxxxxxxxxxxxxx
• Subject: Re: [thredds] Problem serving datasets with ushort data types via THREDDS OPeNDAP
• From: "dmh@xxxxxxxx" <dmh@xxxxxxxx>
• Date: Wed, 27 Apr 2016 16:10:35 -0600

The fundamental problem is that java does not support unsigned types.
So, it should be the case that the bit pattern of the signed short should
match the unsigned bit pattern. You will note the _Unsigned=true

this:
    int x = (int) (signedvalue) & 0xffff)
=Dennis Heimbigner
  Unidata


On 4/27/2016 8:52 AM, John Kerfoot wrote:

Hello,

I've run into a problem when trying to serve NetCDF files that contain ushort datatypes. Here's an example:

http://opendap.oceanobservatories.org:8090/thredds/catalog/ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/catalog.html?dataset=ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc

When I go to the OPeNDAP Access form for this dataset:

http://opendap.oceanobservatories.org:8090/thredds/dodsC/ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc.html

and look at the description of the variable 'external_temp_raw', I see that it is a 16-bit Unsigned Integer, though the _FillValue == -1, which is not possible. If I request the first 10 records of this variable:

http://opendap.oceanobservatories.org:8090/thredds/dodsC/ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc?external_temp_raw[0:1:10]

I get values that are within the range of the ushort data type (0-65535)

Here's where things get strange: If I use ncdump to inspect the data setvia the opendap url:

$ ncdump -v external_temp_raw http://opendap.oceanobservatories.org:8090/thredds/dodsC/ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc | grep external_temp_raw

short external_temp_raw(obs) ;
external_temp_raw:_FillValue = -1s ;

external_temp_raw:comment = "Temperature external to the instrument measured in counts" ;

external_temp_raw:units = "counts" ;
external_temp_raw:coordinates = "time lat lon" ;
external_temp_raw:long_name = "Raw External Temperature, counts" ;
external_temp_raw:_Unsigned = "true" ;
external_temp_raw:_ChunkSizes = 9689 ;

external_temp_raw = -24937, -24937, -24933, -24931, -24939, -24937, -24937,

The data type is shown as short and the first 7 values of the variable are returned as negative values. THREDDS is treating these datatypes as shorts instead of ushorts.

If I download the NetCDF file via HTTPServer and run the same ncdump command, here's what I get:

$ ncdump -v external_temp_raw ~/Downloads/deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc <http://deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc> | grep external_temp_raw

        ushort external_temp_raw(obs) ;
external_temp_raw:_FillValue = 65535US ;

external_temp_raw:comment = "Temperature external to the instrument measured in counts" ;

external_temp_raw:units = "counts" ;
external_temp_raw:coordinates = "time lat lon" ;
external_temp_raw:long_name = "Raw External Temperature, counts" ;

external_temp_raw = 40599, 40599, 40603, 40605, 40597, 40599, 40599, 40599,

So, the actual file shows the datatype as ushort and a _FillValue == 65335US and the data values are within the expected range for that datatype.

Any ideas as to why THREDDS appears to be casting the ushort datatype to short and changing the actual values of that variable when it is accessed via OPeNDAP?

Thanks,
John





_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/

• References:
  • [thredds] Problem serving datasets with ushort data types via THREDDS OPeNDAP
    • From: John Kerfoot