Hi Chris:
netcdf-java has several levels of API. The lowest level (NetcdfFile) doesnt
touch the data, so you just get whatever is in there. this is the level
that users of the C library are used to.
the next highest level of the netcdf-java API (NetcdfDataset) process
missing data, scale/offset, CF conventions, and other things. At this layer
the various attribute conventions are used to identify missing data.
John
On Wed, Apr 15, 2015 at 10:27 AM, Chris Barker <chris.barker@xxxxxxxx>
wrote:
> As I read those docs, and as I've always interpreted netcdf, those "rules"
> are there to inform the user of the data -- I don't expect client software
> reading the data to replace values in the the arrays for me -- at least not
> low-level generic data reading libraries.
>
> What were you expecting, and why?
>
> Higher level libs that represent a data model _may_ do some QA/QC on the
> data, I suppose, but even then, I'd want to clearly specify what I wanted
> it to do -- certainly not silently toss out values because the were outside
> the limits specified in the meta-data.
>
> -Chris
>
>
>
> On Wed, Apr 15, 2015 at 9:18 AM, John Caron <caron@xxxxxxxx> wrote:
>
>> The netcdf-java library does indeed ignore that rule. We should revisit
>> the manual and clarify things i guess.
>>
>> On Wed, Apr 15, 2015 at 9:33 AM, Russ Rew <russ@xxxxxxxxxxxxxxxx> wrote:
>>
>>> Hi Phil,
>>>
>>> I think that's a misinterpretation of the Users Guide attribute
>>> conventions. Under the description for _FillValue, the Guide states:
>>>
>>> The fill value ... is normally outside the valid range and therefore
>>> treated as missing when read by generic applications. It is legal (but not
>>> recommended) for the fill value to be within the valid range.
>>>
>>>
>>> The last sentence implies that the valid range is not determined by the
>>> _Fill_Value.
>>>
>>> I think it's intended that the rule about _Fill_Value under the
>>> description for valid_range only applies in case none of valid_min,
>>> valid_max, or valid_range are specified, so it wouldn't apply to your
>>> example:
>>>
>>> If neither valid_min, valid_max nor valid_range is defined then generic
>>> applications should define a valid range as follows. ...
>>>
>>>
>>> However, I suspect that the rule is confusing enough that writers of
>>> generic clients might just ignore it, even in that case.
>>>
>>> --Russ
>>>
>>> On Wed, Apr 15, 2015 at 8:55 AM, Bentley, Philip <
>>> philip.bentley@xxxxxxxxxxxxxxxx> wrote:
>>>
>>>> Hi folks,
>>>>
>>>> I was hoping someone can clarify for me the correct use of the
>>>> valid_min, valid_max and valid_range attributes by *well-behaved* netCDF
>>>> clients.
>>>>
>>>> Given the netCDF file shown below (in CDL form), and considering the
>>>> rules for handling the aforementioned attributes, as defined in Appendix A
>>>> of the NetCDF user guide (see
>>>> http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html),
>>>> I *think* I'd expect conforming netCDF clients to represent the data for
>>>> each of the variables var1..var3 as an array with values (_, -272, 0, 100,
>>>> 9999, _), where _ signifies missing data values.
>>>>
>>>> ---
>>>> netcdf mditest {
>>>>
>>>> dimensions:
>>>> dim1 = 6;
>>>>
>>>> variables:
>>>> float var1(dim1);
>>>> var1:long_name = "var1";
>>>> var1:units = "1";
>>>> var1:valid_min = -273.0f;
>>>> var1:valid_max = 1.0e5f;
>>>>
>>>> float var2(dim1);
>>>> var2:long_name = "var2";
>>>> var2:units = "1";
>>>> var2:valid_min = -273.0f;
>>>> var2:_FillValue = 1.0e5f; // constrains valid_max
>>>>
>>>> float var3(dim1);
>>>> var3:long_name = "var3";
>>>> var3:units = "1";
>>>> var3:_FillValue = -273.0f; // constrains valid_min
>>>> var3:valid_max = 1.0e5f;
>>>>
>>>> // global attributes
>>>> :Conventions = "CF-1.0";
>>>>
>>>> data:
>>>> var1 = -300, -272, 0, 100, 9999, 1e6 ;
>>>> var2 = -300, -272, 0, 100, 9999, 1e6 ;
>>>> var3 = -300, -272, 0, 100, 9999, 1e6 ;
>>>> }
>>>> ---
>>>>
>>>> However, having tried several different netCDF clients - some C, some
>>>> Java, some Python - none of them appear to adhere consistently to the
>>>> aforementioned rules for handling the valid min/max/range attributes. The
>>>> python-based clients, in particular, only seem to honour the _FillValue
>>>> attribute, reflecting, I believe, the current behaviour of the
>>>> netcdf4-python module.
>>>>
>>>> Am I perhaps misinterpreting the nc attribute-handling conventions?
>>>>
>>>> Regards,
>>>>
>>>> --
>>>> Phil Bentley PhD, Climate Science IT Apps Group
>>>> Met Office Hadley Centre, FitzRoy Road, Exeter, EX1 3PB, United Kingdom
>>>> Tel: +44 (0)1392 886881
>>>> Email: philip.bentley@xxxxxxxxxxxxxxxx Web:
>>>> http://www.metoffice.gov.uk/
>>>>
>>>>
>>>> _______________________________________________
>>>> netcdfgroup mailing list
>>>> netcdfgroup@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe, visit:
>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>
>>>>
>>>
>>> _______________________________________________
>>> netcdfgroup mailing list
>>> netcdfgroup@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe, visit:
>>> http://www.unidata.ucar.edu/mailing_lists/
>>>
>>
>>
>> _______________________________________________
>> netcdfgroup mailing list
>> netcdfgroup@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris.Barker@xxxxxxxx
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
