Re: [netcdfgroup] nccopy should use 1 as default-chunksize for unlimited dimension

Hi,

sorry that I have not been clearer, but my main complaint is that

>     NetCDF: Invalid argument
>     Location: file nccopy.c; line 637

isn't a useful feedback to a user trying to chunk some dimensions.


I think it is caused by nccopy using 0 (NC_UNLIMITED) as default for unlimited dimension which doesn't work for chunking. 1 or anything else would be much, much better than 0.

In my world of operational meteorology, file-formats change very slowly - a default of 1 is very similar to what netcdf-3 has been doing with unlimited dimensions and it won't surprise users as a default.


Best regards,

Heiko


On 2014-10-23 16:22, Ed Hartnett wrote:
This gives very poor performance when the number of timesteps in the
file is large.

On Thu, Oct 23, 2014 at 5:25 AM, Heiko Klein <Heiko.Klein@xxxxxx
<mailto:Heiko.Klein@xxxxxx>> wrote:

    Hi,

    when chunking files with unlimited dimension, the unlimited
    dimension must be given explicitly in nccopy, and will usually be
    set to one.

    In a file with time as unlimited dimension, and X and Y as
    dimensions, it is currently required to use

    $ nccopy -k 4 -c "time/1,X/100,Y/100" in.nc <http://in.nc> out.nc
    <http://out.nc>


    When running without time, it does not work:

    $ nccopy -k 4 -c "X/100,Y/100" in.nc <http://in.nc> out.nc
    <http://out.nc>
    NetCDF: Invalid argument
    Location: file nccopy.c; line 637


    Only for unlimited dimensions, one needs to give the dimension
    explicitly, for all other dimensions a useful default (full
    dim-size) is used. I think a useful default for unlimited dimensions
    is 1.


    Heiko



    --
    Dr. Heiko Klein                              Tel. + 47 22 96 32 58
    <tel:%2B%2047%2022%2096%2032%2058>
    Development Section / IT Department          Fax. + 47 22 69 63 55
    <tel:%2B%2047%2022%2069%2063%2055>
    Norwegian Meteorological Institute http://www.met.no
    P.O. Box 43 Blindern  0313 Oslo NORWAY

    _________________________________________________
    netcdfgroup mailing list
    netcdfgroup@xxxxxxxxxxxxxxxx <mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
    For list information or to unsubscribe,  visit:
    http://www.unidata.ucar.edu/__mailing_lists/
    <http://www.unidata.ucar.edu/mailing_lists/>



--
Dr. Heiko Klein                              Tel. + 47 22 96 32 58
Development Section / IT Department          Fax. + 47 22 69 63 55
Norwegian Meteorological Institute           http://www.met.no
P.O. Box 43 Blindern  0313 Oslo NORWAY



  • 2014 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: