Re: [netcdf-java] I/O perfomance

Antonio,

If you're just reading the entire variable, try using the read() call with
no parameters, which will read all of the data. You can also pass null for
origin and shape.

Ryan

On Wed, Sep 21, 2016 at 2:07 PM, Antonio Rodriges <antonio.rrz@xxxxxxxxx>
wrote:

> Hello,
>
> I have written a simple test that opens NetCDF3 file (50.2 MB), reads its
> whole array time x lat x lon = 1460 x 94 x 192
>
> Open file: 838 ms
> Find var: 12 ms
> Read array: 2393 ms
> (code is below)
>
> Numbers are for SSD with approx 222 MB/sec speed
>
> This is almost 3 sec while during the same time C++ code (NCO, CDO) may do
> a lot more.
>
> My questions are:
> - could this be improved? (e.g. link to native lib or smth else)
> - is this the fastest way to read array (code below)?
>
> Thanks
>
>         long millis = System.currentTimeMillis();
>
>         NetcdfFile FILE = NetcdfFile.open(filepath, null);
>         System.out.println("Open file: " + (System.currentTimeMillis() -
> millis));
>
>         millis = System.currentTimeMillis();
>         Variable var = FILE.findVariable("uwnd");
>         System.out.println("Find var: " + (System.currentTimeMillis() -
> millis));
>
>         int[] origin ;
>         int[] shape ;
>
>         millis = System.currentTimeMillis();
>         shape = var.getShape();
>         System.out.println("Get shape: " + (System.currentTimeMillis() -
> millis));
>
>         origin = new int[shape.length];
>
>         millis = System.currentTimeMillis();
>         Array values = var.read(origin, shape);
>         System.out.println("Read array: " + (System.currentTimeMillis() -
> millis));
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>



-- 
Ryan May, Ph.D.
Software Engineer
UCAR/Unidata
Boulder, CO
  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: