Hi Antonio,
I just checked something similar with the netcdf-c library and I manage
to read > 1600MB/s (the file is cached, and memory bandwidth is 6GB/s).
Your 20MB/s are far too low.
That said, java virtual machines are beasts concerning performance
measurements due to JIT compilation and object-handling. As a rule of
thumb, to get realistic performance of a java routine, run it first 1000
times, then wait for the compiler to finish and then start measuring
your routine. Otherwise, you only measure java-overhead.
Heiko
On 2016-09-21 22:40, Ryan May wrote:
> Antonio,
>
> If you're just reading the entire variable, try using the read() call
> with no parameters, which will read all of the data. You can also pass
> null for origin and shape.
>
> Ryan
>
> On Wed, Sep 21, 2016 at 2:07 PM, Antonio Rodriges <antonio.rrz@xxxxxxxxx
> <mailto:antonio.rrz@xxxxxxxxx>> wrote:
>
> Hello,
>
> I have written a simple test that opens NetCDF3 file (50.2 MB),
> reads its whole array time x lat x lon = 1460 x 94 x 192
>
> Open file: 838 ms
> Find var: 12 ms
> Read array: 2393 ms
> (code is below)
>
> Numbers are for SSD with approx 222 MB/sec speed
>
> This is almost 3 sec while during the same time C++ code (NCO, CDO)
> may do a lot more.
>
> My questions are:
> - could this be improved? (e.g. link to native lib or smth else)
> - is this the fastest way to read array (code below)?
>
> Thanks
>
> long millis = System.currentTimeMillis();
>
> NetcdfFile FILE = NetcdfFile.open(filepath, null);
> System.out.println("Open file: " +
> (System.currentTimeMillis() - millis));
>
> millis = System.currentTimeMillis();
> Variable var = FILE.findVariable("uwnd");
> System.out.println("Find var: " +
> (System.currentTimeMillis() - millis));
>
> int[] origin ;
> int[] shape ;
>
> millis = System.currentTimeMillis();
> shape = var.getShape();
> System.out.println("Get shape: " +
> (System.currentTimeMillis() - millis));
>
> origin = new int[shape.length];
>
> millis = System.currentTimeMillis();
> Array values = var.read(origin, shape);
> System.out.println("Read array: " +
> (System.currentTimeMillis() - millis));
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx <mailto:netcdf-java@xxxxxxxxxxxxxxxx>
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
> <http://www.unidata.ucar.edu/mailing_lists/>
>
>
>
>
> --
> Ryan May, Ph.D.
> Software Engineer
> UCAR/Unidata
> Boulder, CO
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
