Re: [netcdfgroup] libnetcdf 4.0.0 and 4.1.1 read speed

Jennifer Adams <jma@xxxxxxxxxxxxx> writes:

> Hi, Denis -- 
> Additional follow up questions ... 
> What did your ncks command look like -- i.e. what part of the file did you
> 'convert'? 
> Did any part of the ncdump output change after you ran ncks? 
> Are you opening the file with sdfopen or are you using a descriptor file
> with templating? 
> --Jennifer
> On Aug 5, 2010, at 6:15 PM, Russ Rew wrote:
>
>   Hi Denis,
>
>  I have compiled grads with netcdf 4.1.1 lately.
>
>  When I read a file created from netcdf 4.0.0 it takes more than  5
>
>  seconds to open.
>
>  When I read the same file converted by ncks using netcdf 4.1.1,  it
>
>  opens almost instantaneously.
>
>  Deflation and chunking are the same in both files.  Same 'grads'
>
>  software opening each file (compiled with 4.1.1).
>
>  Is there something in netcdf.4.1.1 library that makes the file being
>
>  read much quicker than when files are created with netcdf 4.0.0
>  library?
>

I can think of two reasons:

1 - 4.0 picked poor default chunksizes for large variables.

2 - 4.1.1 turns on default caching which dramatically improves
performance.

The way to test which of these is the case is to open the same file and
check the chunk sizes with nc_inq_var_chunking. Are the chunk sizes
different?

Thanks,

Ed

-- 
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx



  • 2010 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: