On Sun, 4 Oct 2009, Esteban Wagner wrote:
Robb,
thanks for the code, it worked great!.
I have a question regarding the perfomance of the reading:
With the code you sent me, reading a grib2 file with 11 parameters and
31 times each for only one location takes: 80 seconds.
Is there any way to improve this time?
Esteban,
The best method to improving performance is to limit the number of calls
to readDataSlice() even if you have to get more data then you need, you
can always reduce it locally. For example, get all the times for a
variable by using -1 in the time index position, this should be a huge
saving.
Array data = pwd.readDataSlice( -1, -1, idx[1], idx[0]);
As I want to gather values for several locations, say more than 100
locations its a lot of time.(80sec * 100 = 8000 sec = 2.2 hours).
Also, you might want to look at GridCoordSystem methods to see if you can
make a bounding box, or some other method to expand your coverage instead of
getting just one grid point. Look at makeSubset() method to create a new
GridDatatype might be another option.
I would review the javaDocs again, there's probably some other methods
that could improve performance.
Robb...
* Perfomance measures were obtained with the following:
* Hardware: Amd Turion 64 X2 TL-58 1.9GHz , 3Gb Ram, Windows Vista 32,
5400Rpm HD.
* Grib2 file:
ftp://polar.ncep.noaa.gov/pub/waves/develop/multi_1.latest_run/multi_1.glo_30m.all.grb2
* Library versions:(NetCDF:Version 4.0.16; Grib:Version 6.0.16)
* Code :
---------------
import java.io.File;
import java.util.Date;
import ucar.ma2.Array;
import ucar.ma2.IndexIterator;
import ucar.nc2.dt.GridCoordSystem;
import ucar.nc2.dt.GridDatatype;
import ucar.nc2.dt.grid.GridDataset;
public class ReadLatLonFromGrib {
public static void main(final String[] args) {
final File file = new File(
"src/test/resources/multi_1.glo_30m.all.grb2");
final float lat = 39;
final float lon = 179;
final long startTime = new Date().getTime();
// Read the values for the given location
readValuesForLocation(file, lat, lon);
final long endTime = new Date().getTime();
// Calculate reading elapsed time(seconds).
final long elapsedTime = (endTime - startTime) / 1000;
System.out.println("Time elapsed for reading "
+ parameters.values().length
+ " parameters, for 31 times, in one location:
" + elapsedTime
+ " secs.");
}
private static enum parameters {
Direction_of_swell_waves, Direction_of_wind_waves,
Mean_period_of_swell_waves, Mean_period_of_wind_waves,
Primary_wave_direction, Primary_wave_mean_period,
Significant_height_of_combined_wind_waves_and_swell,
Significant_height_of_swell_waves, Significant_height_of_wind_waves,
Wind_direction_from_which_blowing, Wind_speed
};
private static void readValuesForLocation(final File file, final float
lat,
final float lon) {
System.out.println("Reading grib values for lat: " + lat + " and
lon: "
+ lon);
try {
// For each parameter
for (final Enum parameter : parameters.values()) {
final GridDataset gridDataSet =
GridDataset.open(file
.getAbsolutePath());
// For each time
for (int i = 0; i < 31; i++) {
final GridDatatype pwd = gridDataSet
.findGridDatatype(parameter.name());
final GridCoordSystem pwdGcs =
pwd.getCoordinateSystem();
final int[] result = null;
// Get index value for lat/lon
final int[] idx =
pwdGcs.findXYindexFromLatLon(lat, lon,
result);
// Reads values for the given
(location,parameter,time)
final Array data = pwd.readDataSlice(i,
-1, idx[1], idx[0]);
final IndexIterator iter =
data.getIndexIterator();
while (iter.hasNext()) {
// Obtain the value
final float val =
iter.getFloatNext();
}
}
}
System.out.println("Finish reading values");
} catch (final Exception exc) {
exc.printStackTrace();
}
}
}
Thanks.
Regards.
Esteban.
-------------------------------------------------------------------------------------------------
On Fri, Oct 2, 2009 at 4:14 PM, Robb Kambic <rkambic@xxxxxxxxxxxxxxxx> wrote:
On Tue, 29 Sep 2009, Esteban Wagner wrote:
Hi,
I have a grb2 file containing global Primary_wave_direction forecasts. I
would like to read the
values it contains.
By the moment I was able to read the file and obtain a GridDataset using
the following code:
------------
(NetCDF:Version 4.0.16; Grib:Version 6.0.16)
final File file = new
File("src/test/resources/multi_1.glo_30m.DIRPW.grb2");
final GridDataset gridDataSet = GridDataset.open(file.getAbsolutePath());
----------
the file can be downloaded from:
ftp://polar.ncep.noaa.gov/pub/waves/develop/multi_1.latest_run/multi_1.glo_30m.DIRPW.grb2
----------
Now I would like if someone can tell me which are the methods from
gridDataSet I should use in order
to obtain the different forecast values of the Primary_wave_direction for
a given grid point.
Thanks,
Regards.
Esteban.
Esteban,
I'll reference the JavaDocs first:
http://www.unidata.ucar.edu/software/netcdf-java/v4.1/javadoc/ucar/nc2/dt/GridDataset.html
http://www.unidata.ucar.edu/software/netcdf-java/v4.1/javadoc/ucar/nc2/dt/GridDatatype.html
http://www.unidata.ucar.edu/software/netcdf-java/v4.1/javadoc/ucar/nc2/dt/GridCoordSystem.html
I'm assuming by grid point you mean Lat/Lon
One needs to be at the GridCoordSystem level to be able to convert the
Lat/Lon point into indices. Then use the indices information in the
GridDatatype readDatSlice routine to get the value. Here's a short sample
program.
public class GetDataFromLatLon {
public static void main(String[] args) {
try {
final File file = new
File("C:/data/grib/multi_1.glo_30m.DIRPW.grb2");
final GridDataset gridDataSet =
GridDataset.open(file.getAbsolutePath());
GridDatatype pwd = gridDataSet.findGridDatatype(
"Primary_wave_direction");
System.out.println( pwd.getInfo());
GridCoordSystem pwdGcs = pwd.getCoordinateSystem();
int[] result = null;
// Get index value for Lat 30.0 and Lon 179
int[] idx = pwdGcs.findXYindexFromLatLon( 30.0, 179.0, result );
//int[] idx = pwdGcs.findXYindexFromLatLon( 90.0, 0.0, result );
// Extract data value for time 0, no Vert index, Lat index, Lon index
Array data = pwd.readDataSlice( 0, -1, idx[1], idx[0]);
// Another option: read all data values for timeIndex = 0
//Array data = pwd.readVolumeData( 0 );
IndexIterator iter = data.getIndexIterator();
while(iter.hasNext()) {
float val = iter.getFloatNext();
System.out.println( "Primary_wave_direction = "+ val);
}
System.out.println( "Success");
} catch (Exception exc) {
exc.printStackTrace();
}
}
}
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric
Research
rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/
===============================================================================
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/
===============================================================================
