Actually now that I think of it, since you can read the file a better solution
is to just create a new file with the proper attributes and then item by item
copy the values ( i just checked that at least in Python you can read the
values and they appear to be properly masked). A pain, but once written should
work for any similar fils from cmem.
-Roy
> On Oct 2, 2024, at 3:52 PM, Roy Mendelssohn - NOAA Federal
> <roy.mendelssohn@xxxxxxxx> wrote:
>
> To edit right now I think a hex editor will work. Even in a good text editor
> FillValue shows up, it is just not clear how to reset it without seeing the
> hex.. But a hex editor should be able to get at it.
>
> -Roy
>
>
>> On Oct 2, 2024, at 3:31 PM, Dave Allured - NOAA Affiliate via netcdfgroup
>> <netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
>>
>> Yes in my opinion, this is worth issues in both the netcdf-c and
>> netcdf-python libraries, but at low priority. I think this is a relatively
>> obscure case. The hope is that if fixed in the libraries, it will avoid
>> more debugging efforts by future users.
>>
>>
>> On Wed, Oct 2, 2024 at 3:10 PM Chris Barker <chris.barker@xxxxxxxx> wrote:
>> On Wed, Oct 2, 2024 at 1:44 PM Dave Allured - NOAA Affiliate via netcdfgroup
>> <netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
>> This file was probably written by a pre-2013 version of the netcdf-C library,
>>
>> Ahh, that may explain it.
>>
>> Modern libraries choke on the type mismatch, as seen here.
>> ...
>> However IMO, libraries should also support problem-free writes to such
>> existing files.
>>
>> Is this worth an issue on the netCDF C lib?
>>
>> -CHB
>>
>>
>> It may be possible to "fix" files like this, by replacing or deleting the
>> offending _FillValue attributes with lower level or capable software, such
>> as the HDF5 library.
>>
>>
>> On Wed, Oct 2, 2024 at 1:58 PM Chris Barker via netcdfgroup
>> <netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
>> I was wondering what the NaNf meant :-)
>>
>> But yeah, that's probably it -- now to track down whoever wrote this file --
>>
>> But I do wonder what software allowed this?
>>
>> -CHB
>>
>>
>> On Wed, Oct 2, 2024 at 12:53 PM Benjamin Root <ben.v.root@xxxxxxxxx> wrote:
>> I think the issue is that the _FillValue of "NaN" is probably a float64
>> while the variable's dtype is float32, which is probably what it means by a
>> type mismatch. When I have a NaN FillValue for a float32 variable, it prints
>> "NaNf"
>>
>> ```
>> float sum_tp_mm(time, latitude, longitude) ;
>> sum_tp_mm:_FillValue = NaNf ;
>> ```
>>
>>
>> On Wed, Oct 2, 2024 at 12:05 PM Chris Barker via netcdfgroup
>> <netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
>> Thanks Roy, at least it's not just us, and that error did give me a hint,
>> look at this from ncdump-ing the header:
>>
>> float uo(time, depth, latitude, longitude) ;
>> string uo:units = "m s-1" ;
>> uo:_FillValue = NaN ;
>> string uo:standard_name = "eastward_sea_water_velocity" ;
>> string uo:long_name = "Eastward velocity" ;
>>
>> There's no type for the uo:_FillValue attribute.
>>
>> How in the world did that happen?
>>
>> At least now I have a specific thing to point the file creators to.
>>
>> -CHB
>>
>>
>> On Wed, Oct 2, 2024 at 10:58 AM Roy Mendelssohn - NOAA Federal
>> <roy.mendelssohn@xxxxxxxx> wrote:
>> I can reproduce the error. I tried to copy the file using mccoy, got the
>> following:
>>
>> nccopy cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc test.nc
>> NetCDF: Not a valid data type or _FillValue type mismatch
>> Location: file ?; fcn ? line 1408
>>
>> hope this helps.
>>
>> -Roy
>>
>>
>>> On Oct 2, 2024, at 10:46 AM, Chris Barker via netcdfgroup
>>> <netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
>>>
>>> I have a user unable to use a netcdf file with our code.
>>>
>>> It turns out that the file works just fine when read, but if you try to
>>> open it for writing, it chokes:
>>>
>>> (this is the netCDF4 Python lib, version 1.6.5, which seems to be the
>>> latest release)
>>>
>>> I get:
>>> In [3]:
>>> netCDF4.Dataset('cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc',
>>> 'r+')
>>> ---------------------------------------------------------------------------
>>> OSError Traceback (most recent call last)
>>> Cell In[3], line 1
>>> ----> 1
>>> netCDF4.Dataset('cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc',
>>> 'r+')
>>>
>>> File src/netCDF4/_netCDF4.pyx:2469, in netCDF4._netCDF4.Dataset.__init__()
>>>
>>> File src/netCDF4/_netCDF4.pyx:2028, in netCDF4._netCDF4._ensure_nc_success()
>>>
>>> OSError: [Errno -103] NetCDF: Can't write file:
>>> 'cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc'
>>>
>>> Any idea how I can get more info as to what's wrong with it?
>>>
>>> Again, we have had no problems reading the file.
>>>
>>> Thanks,
>>> - CHB
>>>
>>> File is 1.2MB, (not huge by today's standards?) so I've attached it here if
>>> anyone wants to take a look.
>>> --
>>> Christopher Barker, Ph.D.
>>> Oceanographer
>>>
>>> Emergency Response Division
>>> NOAA/NOS/OR&R (206) 526-6959 voice
>>> 7600 Sand Point Way NE (206) 526-6329 fax
>>> Seattle, WA 98115 (206) 526-6317 main reception
>>>
>>> Chris.Barker@xxxxxxxx
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web. Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> netcdfgroup mailing list
>> netcdfgroup@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> https://www.unidata.ucar.edu/mailing_lists/
>
> **********************
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new street address***
> 110 McAllister Way
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: Roy.Mendelssohn@xxxxxxxx www: https://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>
**********************
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Mendelssohn@xxxxxxxx www: https://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
