I really consider NetCDF4 as a standard that should be followed by users
AND devs. ;)
1. Sometimes devs should push users forward to achieve benefits, a good
example, albeit heavily debated and different issue, is the python project
(going from 2 to 3 really did a lot of damage for many users) which in the
long run might/will be regarded as a bold, but important move. It is still
slowly being transitioned, you seem to have reached the doorstep of
actually crossing to python 3, or whether you should stick with python 2. :)
2. In my own projects using 3 was restricting my ideas and thus the
capabilities of the code I produced. Since I have adopted 4 I have been
able to easily extend my code to areas that would have been cumbersome
using 3.
2a. Yes I restrict my user base to rely on 4, but it was my decision to
make.
2b. I also think that in the long run this will heavily benefit the users.
:)
3. Data representation/saving for users is also a very good reason to
change. Instead of saving 5 files which have some connection, we can put
all in one file and use groups. This is not to say that there aren't
downsides to the bigger file, but users will (in my experience) quickly
adopt a file format if it "makes more sense" and/or is structured better
than the previous.
4. Huge files are more than often seen and here 4 provides several
capabilities that any user would welcome:
4a. Creation of files can be done without compression to speed up the
application just as in 3 (or even in parallel)
4b. Compression can easily be (un)applied via command line tools
4c. Even if compression have been applied all utilities compiled against a
4 library will be able to read it, without having to unzip it (at least the
utilities I have used), (at least using fortran 90 format, I am not too
sure about c)
4d. For huge and many files, it seems better to have them intrinsically
compressed without restricting access to them, while for 3 you will have to
zip to reduce disk footprint, and every time you need to read it, you need
to unzip.... meeeh.... :(
5. You can always create 3 from 4 (expanding groups into files)
In regards of what unidata should do and shouldn't do, I will have no say.
However Jeff Whitaker have done a good job of easing the utilisation of
NetCDF files in python (it is now under unidata on github, so I presume it
has been fully adopted). Thanks for that! Many thanks indeed!
PS. I do not come from the atmospheric community so maybe I am just way off
target? :)
2014-11-06 16:52 GMT+01:00 Cathy Smith (NOAA Affiliate) <
cathy.smith@xxxxxxxx>:
> NetCDF'ers
> We've changed many of our files from netCDF3 to netCDF4 both to take
> advantage of internal packing but also because they work better in some
> software (NCL). We have heard from users of problems reading the newer
> format. Often they don't have their applications or netCDF library upgraded
> to the latest versions but sometimes, software they relied on isn't
> available for netCDF4. For example, EXCEL does not have a netCDF4 interface
> and a package called "FAN" probably doesn't. Should we be assuming users
> and software developers ought to be netCDF4 ready or is netCDF4 more of a
> language for advanced users and there is no expectation that users use it
> as opposed to netCDF3? Should unidata be facilitating the update of
> packages to netCDF4? I'm thinking more of local software such as on this
> page http://www.unidata.ucar.edu/software/netcdf/Contrib.html but also
> packages like EXCEL.
> We do suggest users can convert the files back to netCDF3 if possible but
> there are downsides to that for the users.
>
> Thanks,
> Cathy Smith
>
>
> --
> ----------------------------------------------
> NOAA/ESRL PSD and CU CIRES
> 303-497-6263
> http://www.esrl.noaa.gov/psd/people/cathy.smith/
>
> Emails about data/webpages may get quicker responses from emailing
> esrl.psd.data@xxxxxxxx
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
--
Kind regards Nick
