OK -- issue posted:
https://github.com/Unidata/netcdf-c/issues/3038
I only did the C lib, 'cause it's showing up in C and python -- is Fortran
a different code base? or does it use the same C lib under hood?
As for how high priority it is -- that's not up to me -- not a high enough
priority for me to write a PR :-)
Thanks,
-CHB
-
On Wed, Oct 2, 2024 at 4:02 PM Roy Mendelssohn - NOAA Federal via
netcdfgroup <netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
> Actually now that I think of it, since you can read the file a better
> solution is to just create a new file with the proper attributes and then
> item by item copy the values ( i just checked that at least in Python you
> can read the values and they appear to be properly masked). A pain, but
> once written should work for any similar fils from cmem.
>
> -Roy
>
> > On Oct 2, 2024, at 3:52 PM, Roy Mendelssohn - NOAA Federal <
> roy.mendelssohn@xxxxxxxx> wrote:
> >
> > To edit right now I think a hex editor will work. Even in a good text
> editor FillValue shows up, it is just not clear how to reset it without
> seeing the hex.. But a hex editor should be able to get at it.
> >
> > -Roy
> >
> >
> >> On Oct 2, 2024, at 3:31 PM, Dave Allured - NOAA Affiliate via
> netcdfgroup <netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
> >>
> >> Yes in my opinion, this is worth issues in both the netcdf-c and
> netcdf-python libraries, but at low priority. I think this is a relatively
> obscure case. The hope is that if fixed in the libraries, it will avoid
> more debugging efforts by future users.
> >>
> >>
> >> On Wed, Oct 2, 2024 at 3:10 PM Chris Barker <chris.barker@xxxxxxxx>
> wrote:
> >> On Wed, Oct 2, 2024 at 1:44 PM Dave Allured - NOAA Affiliate via
> netcdfgroup <netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
> >> This file was probably written by a pre-2013 version of the netcdf-C
> library,
> >>
> >> Ahh, that may explain it.
> >>
> >> Modern libraries choke on the type mismatch, as seen here.
> >> ...
> >> However IMO, libraries should also support problem-free writes to such
> existing files.
> >>
> >> Is this worth an issue on the netCDF C lib?
> >>
> >> -CHB
> >>
> >>
> >> It may be possible to "fix" files like this, by replacing or deleting
> the offending _FillValue attributes with lower level or capable software,
> such as the HDF5 library.
> >>
> >>
> >> On Wed, Oct 2, 2024 at 1:58 PM Chris Barker via netcdfgroup <
> netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
> >> I was wondering what the NaNf meant :-)
> >>
> >> But yeah, that's probably it -- now to track down whoever wrote this
> file --
> >> But I do wonder what software allowed this?
> >>
> >> -CHB
> >>
> >>
> >> On Wed, Oct 2, 2024 at 12:53 PM Benjamin Root <ben.v.root@xxxxxxxxx>
> wrote:
> >> I think the issue is that the _FillValue of "NaN" is probably a float64
> while the variable's dtype is float32, which is probably what it means by a
> type mismatch. When I have a NaN FillValue for a float32 variable, it
> prints "NaNf"
> >>
> >> ```
> >> float sum_tp_mm(time, latitude, longitude) ;
> >> sum_tp_mm:_FillValue = NaNf ;
> >> ```
> >>
> >>
> >> On Wed, Oct 2, 2024 at 12:05 PM Chris Barker via netcdfgroup <
> netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
> >> Thanks Roy, at least it's not just us, and that error did give me a
> hint, look at this from ncdump-ing the header:
> >>
> >> float uo(time, depth, latitude, longitude) ;
> >> string uo:units = "m s-1" ;
> >> uo:_FillValue = NaN ;
> >> string uo:standard_name = "eastward_sea_water_velocity" ;
> >> string uo:long_name = "Eastward velocity" ;
> >>
> >> There's no type for the uo:_FillValue attribute.
> >>
> >> How in the world did that happen?
> >>
> >> At least now I have a specific thing to point the file creators to.
> >>
> >> -CHB
> >>
> >>
> >> On Wed, Oct 2, 2024 at 10:58 AM Roy Mendelssohn - NOAA Federal <
> roy.mendelssohn@xxxxxxxx> wrote:
> >> I can reproduce the error. I tried to copy the file using mccoy, got
> the following:
> >>
> >> nccopy cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc
> test.nc
> >> NetCDF: Not a valid data type or _FillValue type mismatch
> >> Location: file ?; fcn ? line 1408
> >>
> >> hope this helps.
> >>
> >> -Roy
> >>
> >>
> >>> On Oct 2, 2024, at 10:46 AM, Chris Barker via netcdfgroup <
> netcdfgroup@xxxxxxxxxxxxxxxx> wrote:
> >>>
> >>> I have a user unable to use a netcdf file with our code.
> >>>
> >>> It turns out that the file works just fine when read, but if you try
> to open it for writing, it chokes:
> >>>
> >>> (this is the netCDF4 Python lib, version 1.6.5, which seems to be the
> latest release)
> >>>
> >>> I get:
> >>> In [3]: netCDF4.Dataset('
> cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc', 'r+')
> >>>
> ---------------------------------------------------------------------------
> >>> OSError Traceback (most recent call
> last)
> >>> Cell In[3], line 1
> >>> ----> 1 netCDF4.Dataset('
> cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc', 'r+')
> >>>
> >>> File src/netCDF4/_netCDF4.pyx:2469, in
> netCDF4._netCDF4.Dataset.__init__()
> >>>
> >>> File src/netCDF4/_netCDF4.pyx:2028, in
> netCDF4._netCDF4._ensure_nc_success()
> >>>
> >>> OSError: [Errno -103] NetCDF: Can't write file: '
> cmems_mod_nws_phy_anfc_0.027deg-3D_P1D-m_1727700325546.nc'
> >>>
> >>> Any idea how I can get more info as to what's wrong with it?
> >>>
> >>> Again, we have had no problems reading the file.
> >>>
> >>> Thanks,
> >>> - CHB
> >>>
> >>> File is 1.2MB, (not huge by today's standards?) so I've attached it
> here if anyone wants to take a look.
> >>> --
> >>> Christopher Barker, Ph.D.
> >>> Oceanographer
> >>>
> >>> Emergency Response Division
> >>> NOAA/NOS/OR&R (206) 526-6959 voice
> >>> 7600 Sand Point Way NE (206) 526-6329 fax
> >>> Seattle, WA 98115 (206) 526-6317 main reception
> >>>
> >>> Chris.Barker@xxxxxxxx
> >> _______________________________________________
> >> NOTE: All exchanges posted to Unidata maintained email lists are
> >> recorded in the Unidata inquiry tracking system and made publicly
> >> available through the web. Users who post to any of the lists we
> >> maintain are reminded to remove any personal information that they
> >> do not want to be made public.
> >>
> >>
> >> netcdfgroup mailing list
> >> netcdfgroup@xxxxxxxxxxxxxxxx
> >> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
> >
> > **********************
> > "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> > **********************
> > Roy Mendelssohn
> > Supervisory Operations Research Analyst
> > NOAA/NMFS
> > Environmental Research Division
> > Southwest Fisheries Science Center
> > ***Note new street address***
> > 110 McAllister Way
> > Santa Cruz, CA 95060
> > Phone: (831)-420-3666
> > Fax: (831) 420-3980
> > e-mail: Roy.Mendelssohn@xxxxxxxx www: https://www.pfeg.noaa.gov/
> >
> > "Old age and treachery will overcome youth and skill."
> > "From those who have been given much, much will be expected"
> > "the arc of the moral universe is long, but it bends toward justice"
> -MLK Jr.
> >
>
> **********************
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new street address***
> 110 McAllister Way
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: Roy.Mendelssohn@xxxxxxxx www: https://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK
> Jr.
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@xxxxxxxx
