Re: [netcdfgroup] Bad file ??

On Wed, Oct 2, 2024 at 1:44 PM Dave Allured - NOAA Affiliate via
netcdfgroup <netcdfgroup@xxxxxxxxxxxxxxxx> wrote:

> This file was probably written by a pre-2013 version of the netcdf-C
> library,
>

Ahh, that may explain it.


> Modern libraries choke on the type mismatch, as seen here.
>
...

> However IMO, libraries should also support problem-free writes to such
> existing files.
>

Is this worth an issue on the netCDF C lib?

-CHB





> It may be possible to "fix" files like this, by replacing or deleting the
> offending _FillValue attributes with lower level or capable software, such
> as the HDF5 library.
>
>
> On Wed, Oct 2, 2024 at 1:58 PM Chris Barker via netcdfgroup <
> netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
>
>> I was wondering what the NaNf meant :-)
>>
>> But yeah, that's probably it -- now to track down whoever wrote this file
>> --
>> But I do wonder what software allowed this?
>>
>> -CHB
>>
>>
>> On Wed, Oct 2, 2024 at 12:53 PM Benjamin Root <ben.v.root@xxxxxxxxx>
>> wrote:
>>
>>> I think the issue is that the _FillValue of "NaN" is probably a float64
>>> while the variable's dtype is float32, which is probably what it means by a
>>> type mismatch. When I have a NaN FillValue for a float32 variable, it
>>> prints "NaNf"
>>>
>>> ```
>>> float sum_tp_mm(time, latitude, longitude) ;
>>>    sum_tp_mm:_FillValue = NaNf ;
>>> ```
>>>
>>
>
> On Wed, Oct 2, 2024 at 12:05 PM Chris Barker via netcdfgroup <
> netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
>
>> Thanks Roy, at least it's not just us, and that error did give me a hint,
>> look at this from ncdump-ing the header:
>>
>>  float uo(time, depth, latitude, longitude) ;
>> string uo:units = "m s-1" ;
>> uo:_FillValue = NaN ;
>> string uo:standard_name = "eastward_sea_water_velocity" ;
>> string uo:long_name = "Eastward velocity" ;
>>
>> There's no type for the uo:_FillValue attribute.
>>
>> How in the world did that happen?
>>
>> At least now I have a specific thing to point the file creators to.
>>
>> -CHB
>>
>>
>> On Wed, Oct 2, 2024 at 10:58 AM Roy Mendelssohn - NOAA Federal <
>> roy.mendelssohn@xxxxxxxx> wrote:
>>
>>> I can reproduce the error.  I tried to copy the file using mccoy,  got
>>> the following:
>>>
>>> nccopy cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc test.nc
>>> NetCDF: Not a valid data type or _FillValue type mismatch
>>> Location: file ?; fcn ? line 1408
>>>
>>> hope this helps.
>>>
>>> -Roy
>>>
>>>
>>> > On Oct 2, 2024, at 10:46 AM, Chris Barker via netcdfgroup <
>>> netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
>>> >
>>> > I have a user unable to use a netcdf file with our code.
>>> >
>>> > It turns out that the file works just fine when read, but if you try
>>> to open it for writing, it chokes:
>>> >
>>> > (this is the netCDF4 Python lib, version 1.6.5, which seems to be the
>>> latest release)
>>> >
>>> > I get:
>>> > In [3]: netCDF4.Dataset('
>>> cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc', 'r+')
>>> >
>>> ---------------------------------------------------------------------------
>>> > OSError                                   Traceback (most recent call
>>> last)
>>> > Cell In[3], line 1
>>> > ----> 1 netCDF4.Dataset('
>>> cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc', 'r+')
>>> >
>>> > File src/netCDF4/_netCDF4.pyx:2469, in
>>> netCDF4._netCDF4.Dataset.__init__()
>>> >
>>> > File src/netCDF4/_netCDF4.pyx:2028, in
>>> netCDF4._netCDF4._ensure_nc_success()
>>> >
>>> > OSError: [Errno -103] NetCDF: Can't write file: '
>>> cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc'
>>> >
>>> > Any idea how I can get more info as to what's wrong with it?
>>> >
>>> > Again, we have had no problems reading the file.
>>> >
>>> > Thanks,
>>> > - CHB
>>> >
>>> > File is 1.2MB, (not huge by today's standards?) so I've attached it
>>> here if anyone wants to take a look.
>>> > --
>>> > Christopher Barker, Ph.D.
>>> > Oceanographer
>>> >
>>> > Emergency Response Division
>>> > NOAA/NOS/OR&R            (206) 526-6959   voice
>>> > 7600 Sand Point Way NE   (206) 526-6329   fax
>>> > Seattle, WA  98115       (206) 526-6317   main reception
>>> >
>>> > Chris.Barker@xxxxxxxx
>>
>> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> https://www.unidata.ucar.edu/mailing_lists/
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@xxxxxxxx
  • 2024 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: