Re: [netcdf-java] Storing forecast ensembl model data in netCDF

Hi Don,

I'm not quite fan of the FMRC. Basically at the beginning they are cool and in many cases are enough, functional and efficient.

The problem that I usually face, with FMRC, is when I wan a fine grain control on how data it's been aggregated, and I'm finish building my ad-hoc ncml aggregations.

With respect to time coordinates with "special" calendars, the netcdf-java it's good handling them:

https://docs.unidata.ucar.edu/netcdf-java/5.2/javadoc/ucar/nc2/time/Calendar.html

but sometimes the TDS has issues with those kind on "non-standard" calendars. I think FMRC is one of that cases, but is a guess.

Regards

Antonio




On 24/4/20 16:41, Don Murray via netcdf-java wrote:
Thanks again, Antonio.  The ECMWF link is very helpful.

Do you know if the FMRC can handle a calendar attribute on the time? I'm dealing with monthly data and would like to "use months since" with a 360day calendar (I know months since is frowned upon, but with a 360day calendar, it's a fixed 30day period).

Thanks again for your help.  I'll let you know if I have more questions.

Don

On 4/23/20 2:09 PM, Antonio S. Cofiño wrote:
Hi Don,

Yes you need to register into the system, because some datasets are restricted. If you want to access please sign-up

http://meteo.unican.es/udg-tap/

and let me know. I can provide you a temporal access.

Although the dataset it's encoded in GRIB1 format, it's been exposed as one "netcdf" entity, with all variables and coordinates. Attached you can find the resulting CDL for the "netcdf" entity.

For this dataset the realization coordinate was not recognized by the netcdf-java I have had to impose it and create all aggreagtions, also on the runtime coordinate.

If you want a Guide about encoding products you may be find interesting this guide:

https://confluence.ecmwf.int/display/COPSRV/Guide+to+NetCDF+encoding+for+C3S+providers

Let me know if that helps

Regards

Antonio



On 22/4/20 22:28, Don Murray via netcdf-java wrote:
Thanks, Antonio.  With GRIB it seems straight forward.  I have netCDF files that I'm trying to use and am tryint to figure out how to structure them.  Are any of the datasets on your site backed by netCDF instead of GRIB? Looks like I need a user/pass to get in to see the details.

Don

On 4/22/20 1:17 PM, Antonio S. Cofiño wrote:
Hi Don,

On 4/22/20 5:24 PM, Don Murray via netcdf-java wrote:
Hi All-

I'm looking at storing forecast model data (NMME and GEFS) in netCDF format and am wondering if anyone out there is doing that and using netcdf-java to access and subset.

Yes. You can take a look at:

https://meteo.unican.es/trac/wiki/udg/ecoms/dataserver/catalog

In particular System4, CFSv2 and glosea5 datasets which corresponds to different seasonal forecasts systems.


For GRIB data, netcdf-java/THREDDS uses a 2D time dimension with dimensions (reftime, time, ens, lat, lon) where time is stored as time(reftime, time). I'm looking for examples that use netCDF as the storage format, particularly if they are served up via THREDDS.  I'm using the ucar.nc2.dt.grid classes to read netCDF files at the present.

In particular we are using netcdf-java for accessing data from client side and TDS from server side to aggregate data in common datasets.

In the attached files you can find the opendap info for on of the datasets which includes "realization" coordinate and 2D time, among level, lat and lon coordinates (level it's a particular case).

https://www.unidata.ucar.edu/software/netcdf-java/v4.3/v4.2/javadocAll/ucar/nc2/ft/grid/Grid.html#makeSubset(ucar.ma2.Range,%20ucar.ma2.Range,%20ucar.ma2.Range,%20ucar.ma2.Range,%20ucar.ma2.Range,%20ucar.ma2.Range)

The client use de makeSubset method from the Grid object which allows to subset along the realization coordinate and 2D time:

https://www.unidata.ucar.edu/software/netcdf-java/v4.3/v4.2/javadocAll/ucar/nc2/ft/grid/Grid.html

The dataset are the original GRIB1 files aggregated using netcdf-java ncml aggregation and published it by TDS.

This is been used from a different R packages we developed, but in particular the central part it's the loadeR which interacts directly with the netcdf-java API (by rJava):

https://github.com/SantanderMetGroup/loadeR

You have some academic reference and use cases there.

Please, let me know if you need more info.

Regards

Antonio


--
Antonio S. Cofiño
Dep. de Matemática Aplicada y
         Ciencias de la Computación
Universidad de Cantabria
http://www.meteo.unican.es


_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: https://www.unidata.ucar.edu/mailing_lists/



_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: https://www.unidata.ucar.edu/mailing_lists/




  • 2020 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: