Re: [netcdf-java] RunTime coordinate axis

Thank you John.

I know that the Feature collection is not perfect and needs some extra work. But I think it stills offers a neat solution for huge collection of files. I have another question/request but I will made it in another mail.


But I'm coming back to my original question. Forget about how this dataset has been generated and look the attached NCML. The problem is that the Total_precipitation_surface variable it's no been recognized as a Grid data type. If I permute dimensions on the time variable then the variable is recognized as Grid datatype.

I don't think the Unidata's _Coordinate convention forces any order on dimension, and therefore defining as first dimension the RunTime axis on the time 2D coordinated axis it's a programming limitation.

Looking at:
ucar.nc2.dt.grid.GridCoordSys.isGridCoordSys(Formatter, CoordinateSystem, VariableEnhanced)

which it appears to be the responsible for deciding if a Variable is a Grid datatype

It appears that it could be easily fixed if we modified this part:
      if (!rt1D.getDimension(0).equals(t.getDimension(0))) {
        if (sbuff != null) {
sbuff.format("%s: Time axis must use RunTime dimension%n", cs.getName());
        }
        return false;
      }


coding also the check for the 2nd dimension:
if (!rt1D.getDimension(0).equals(t.getDimension(0)) && !rt1D.getDimension(0).equals(t.getDimension(1))) {
        if (sbuff != null) {
sbuff.format("%s: Time axis must use RunTime dimension%n", cs.getName());
        }
        return false;
      }

and also another code hacking at :
ucar.nc2.dt.grid.GridCoordSys.makeTimeAxisForRun(int)

and more precisely tAxis.slice, which is assuming the runtime dimension in the 1st position

private CoordinateAxis1DTime makeTimeAxisForRun(int run_index) {
    VariableDS section;
    try {
      section = (VariableDS) tAxis.slice(0, run_index);
      return CoordinateAxis1DTime.factory(ds, section, null);
    } catch (InvalidRangeException e) {
      e.printStackTrace();
    } catch (IOException e) {
      e.printStackTrace();
    }
    return null;
  }


Do you think this will fix the issue, or I'm missing something else?

By the way, I can make those changes, but how can I push them into the Thredds Git repository?

Regards

Antonio

--
Antonio S. Cofiño
Grupo de Meteorología de Santander
Dep. de Matemática Aplicada y
        Ciencias de la Computación
Universidad de Cantabria
http://www.meteo.unican.es

El 11/03/2013 14:51, John Caron escribió:
Hi Antonio:

The problem is that we dont reliably know the runtime. Many (most?) GRIB encodings use the "reference time" as the runtime. However not all do. We probably need a way to configure the TDS to assume reference time = runtime.

I apologize, I havent had time to work on the solution for this. I will bump it up the list.

John


On 3/8/2013 11:21 AM, "Antonio S. Cofiño" wrote:
John,

The file/path structure is the following:

 * One folder per runtime year
 * Inside of this folder one file per variable and forecast time step.
This file include 12 hourly runtimes for the whole year (720 runtimes)

These are file names and paths for couple of years (1979 and 1980) and variables (142.128 and 143.128). The last number before the .grb are the step time in hours (3, 6, 9, 12, 15, 18, 21, 24, 30 and 36):
./1979/INTERIM_1979_SFC_142.128_12.grb
./1979/INTERIM_1979_SFC_142.128_15.grb
./1979/INTERIM_1979_SFC_142.128_18.grb
./1979/INTERIM_1979_SFC_142.128_21.grb
./1979/INTERIM_1979_SFC_142.128_24.grb
./1979/INTERIM_1979_SFC_142.128_30.grb
./1979/INTERIM_1979_SFC_142.128_36.grb
./1979/INTERIM_1979_SFC_142.128_3.grb
./1979/INTERIM_1979_SFC_142.128_6.grb
./1979/INTERIM_1979_SFC_142.128_9.grb
./1979/INTERIM_1979_SFC_143.128_12.grb
./1979/INTERIM_1979_SFC_143.128_15.grb
./1979/INTERIM_1979_SFC_143.128_18.grb
./1979/INTERIM_1979_SFC_143.128_21.grb
./1979/INTERIM_1979_SFC_143.128_24.grb
./1979/INTERIM_1979_SFC_143.128_30.grb
./1979/INTERIM_1979_SFC_143.128_36.grb
./1979/INTERIM_1979_SFC_143.128_3.grb
./1979/INTERIM_1979_SFC_143.128_6.grb
./1979/INTERIM_1979_SFC_143.128_9.grb
./1980/INTERIM_1980_SFC_142.128_12.grb
./1980/INTERIM_1980_SFC_142.128_15.grb
./1980/INTERIM_1980_SFC_142.128_18.grb
./1980/INTERIM_1980_SFC_142.128_21.grb
./1980/INTERIM_1980_SFC_142.128_24.grb
./1980/INTERIM_1980_SFC_142.128_30.grb
./1980/INTERIM_1980_SFC_142.128_36.grb
./1980/INTERIM_1980_SFC_142.128_3.grb
./1980/INTERIM_1980_SFC_142.128_6.grb
./1980/INTERIM_1980_SFC_142.128_9.grb
./1980/INTERIM_1980_SFC_143.128_12.grb
./1980/INTERIM_1980_SFC_143.128_15.grb
./1980/INTERIM_1980_SFC_143.128_18.grb
./1980/INTERIM_1980_SFC_143.128_21.grb
./1980/INTERIM_1980_SFC_143.128_24.grb
./1980/INTERIM_1980_SFC_143.128_30.grb
./1980/INTERIM_1980_SFC_143.128_36.grb
./1980/INTERIM_1980_SFC_143.128_3.grb
./1980/INTERIM_1980_SFC_143.128_6.grb
./1980/INTERIM_1980_SFC_143.128_9.grb

Therefore I'm using a GRIB feature collection per time step.:
<featureCollection name="INTERIM EuroAfrica075 GRIB 03H" featureType="GRIB" path="interimEuroAfrica075Grib_03">
    <metadata inherited="true">
      <serviceName>all</serviceName>
      <dataFormat>GRIB-1</dataFormat>
    </metadata>
<collection spec="/oceano/gmeteo/DATA/ECMWF/INTERIM/EuroAfrica075/SFC_FC/**/INTERIM_.*_3\.grb$"/>
    <update startup="true" trigger="allow"/>
    <gribConfig datasetTypes="Best Latest Files" />
</featureCollection>

There are one of this per time step.

I'm interested on the 2D time coordinate, but the GRIB collection doesn't provide this datasetType. I have tried the FMRC feature collection, but it treats each file as one runtime and every time inside as a step time.

This why I'm using one dataset per time step.

What I'm trying to do now it's to "aggregate" all of them on the time dimension:
  <aggregation dimName="frctime" type="joinNew">
<variableAgg name="Stratiform_precipitation_Large-scale_precipitation_surface"/>
    <variableAgg name="Convective_precipitation_surface"/>
    <variableAgg name="time"/>
    <variableAgg name="time1"/>
    <variableAgg name="time1_bounds"/>
<netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_03/best"/> <netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_06/best"/> <netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_09/best"/> <netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_12/best"/> <netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_15/best"/> <netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_18/best"/> <netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_21/best"/> <netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_24/best"/> <netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_30/best"/> <netcdf location="dods://localhost:13180/tds6/dodsC/interimEuroAfrica075Grib_36/best"/>
  </aggregation>

But, the aggregation it's not been recognized as GRID datatype, because the dimension order in the frctime variable it's no the right one.

Regards and thank you

Antonio


--
Antonio S. Cofiño
Grupo de Meteorología de Santander
Dep. de Matemática Aplicada y
        Ciencias de la Computación
Universidad de Cantabria
http://www.meteo.unican.es

El 08/03/2013 18:33, John Caron escribió:
On 3/2/2013 2:44 PM, "Antonio S. Cofiño" wrote:
Dear John,

(I'm using the netcdf-java 4.3.15)
As you can see in the attached ncml,I'm defining a RunTime dimension in the Total_precipitation_surface inner than the time dimension. I'm doing this because I need to do the outer aggregation for the forecast time dimension. Also the time variable is been aggregated.

The problem is that the Total_precipitation_surface variable it's no been recognized as a Grid data type. If I permute dimensions on the time variable then the variable is recognized as Grid datatype.

This is right? or I'm forced to define as first dimension the RunTime axis on the time 2D coordinated axis?

Looking at:
ucar.nc2.dt.grid.GridCoordSys.isGridCoordSys(Formatter, CoordinateSystem, VariableEnhanced)

which it appears to be the responsible for deciding if a Variable is a Grid datatype

It appears that it could be easily fixed if we modified this part:
      if (!rt1D.getDimension(0).equals(t.getDimension(0))) {
        if (sbuff != null) {
sbuff.format("%s: Time axis must use RunTime dimension%n", cs.getName());
        }
        return false;
      }

coding also the check for the 2nd dimension:
if (!rt1D.getDimension(0).equals(t.getDimension(0)) && !rt1D.getDimension(0).equals(t.getDimension(1))) {
        if (sbuff != null) {
sbuff.format("%s: Time axis must use RunTime dimension%n", cs.getName());
        }
        return false;
      }

It's correct?

Antonio



Hi Antonio:

A Grib collection should do the correct thing, so im confused as to how this dataset was generated. You are using Grib Collection or FMRC?

John




  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: