Hi John,
Referring to the major email thread (one email attached) that has been
going on regarding ensemble aggregations (and forecast aggregations!):
The proposals have been for a new (5th) dimension in CF that would be
the ensemble axis. The assumption is that the "coordinates" associated
with this axis would often be an array of strings. For example, there
might be a coordinate variable called "institution" that contained
"UCAR", "JPL", "PMEL", ... (I am glossing over the nuisance details
that netCDF3 doesn't support a true String datatype. That detail does
not really effect the current discussion.)
Now, though most of the discussions on the email seem to imply that the
ensemble is in a single "file" I doubt that in practice this would often
be the case. Far more manageable would be if an aggregation server such
as TDS were capable of creating the ensemble axis from a collection of
individual files. This would imply presumably that global attributes of
the individual files (say a global attribute "institution") should be
promoted to become arrays in the aggregation.
I was wondering if you had been planning (or already have implemented)
such a capability in TDS. There have been many years of conversation on
this topic in the OPeNDAP world.
- Steve
================================================================
-------- Original Message --------
Subject: Re: [CF-metadata] Getting back to ensembles
Date: Tue, 28 Nov 2006 13:59:46 +0000
From: Francisco Doblas-Reyes <Francisco.Doblas-Reyes@xxxxxxxxx>
Reply-To: Francisco.Doblas-Reyes@xxxxxxxxx
Organization: ECMWF
To: cf-metadata@xxxxxxxxxxxx
CC: Antje Weisheimer <Antje.Weisheimer@xxxxxxxxx>
References: <20061123000327.GA24633@xxxxxxxxxxxxxxxxx>
Dear all,
I have to confess that I'm slightly lost after the exchange of messages
in the last few weeks (time during which I was away and unable to
respond). However, I'll try to make some comments on the proposed
structures and send an example of what I've understood.
Using "realization" as the dimension to include ensemble data in the
file seems a good option to me, although I went for a dimension name
different (see below). String variables with this dimension should be
able to do the job to include the metadata describing the simulations.
What is not so clear to me is whether attributes such as "institution"
or "source" or the one mentioned by Jamie "experiment_id" would be
allowed as auxiliary variables, "realization" being for me the
coordinate variable. Their inclusion in the list of accepted standard
names would be the best for me, but having them as standard_metadata or
external_vocabulary is also acceptable. However, the use of external
dictionaries poses certain problems, as discussed. We can maintain at
ECMWF those describing the simulations performed with all the models
that are run and archived here (several European forecast models), but
nothing guarantees that the format will be similar to the vocabularies
kept at, say, NCEP. Reaching an agreement on the external dictionaries
might take again a few months and contacting people who are not part of
the CF list.
Below there is an example of multi-forecast system file which is close
to what is being discussed here. It's been constructed from a set of
multi-model seasonal forecasts for the year 2001 (started on the first
of February, May, August and November). The variable is geopotential
height. Each single model (there are 7) contributes with a 9-member
6-month simulation, where each member has been produced with slightly
different initial conditions. You'll see that "ensemble" is a dimension,
the label/coordinate variable (dimensioned with ensemble) being
"realization", which is not monotonic (one of the alternatives discussed
in Jonathan's message). Realization is a number because they correspond
to the production order in the original single-model ensemble. Please,
note the use of "reftime" and "leadtime" and let me know if you diagree
with something. I included values for the variables "source" and
"institution" (this last one being the institution providing the data),
as well as for "experiment_id". Note that "experiment_id" is necessary
because several experiments can be carried out with the same value of
source, realization and institution. Keeping all these variables
separate (instead of a long character chain) makes life easier for data
handling and plot labelling.
I could read and plot the file with ncBrowse (both as a stand-alone file
and as a request to a Thredds server) and I'm in the process of doing
the same with the NCO functions.
Finally, as data providers we can rewrite the files once a final
decision has been reached. In the meantime, we can serve the data with
metadata similar to those below, but expressing that they can be changed
in the near future.
Best regards,
Paco
netcdf MM_129_mon_2001 {
dimensions:
longitude = 144 ;
latitude = 71 ;
level = 3 ;
time = 24 ;
time_bnd = 2 ;
ensemble = 63 ;
string4 = 4 ;
string15 = 15 ;
string50 = 50 ;
variables:
float longitude(longitude) ;
longitude:data_type = "float" ;
longitude:units = "degrees_east" ;
longitude:axis = "X" ;
longitude:standard_name = "longitude" ;
longitude:topology = "circular" ;
longitude:modulo = 360 ;
longitude:valid_min = 0. ;
longitude:valid_max = 359. ;
float latitude(latitude) ;
latitude:data_type = "float" ;
latitude:units = "degrees_north" ;
latitude:axis = "Y" ;
latitude:standard_name = "latitude" ;
latitude:valid_min = -89. ;
latitude:valid_max = 89. ;
float reftime(time) ;
reftime:units = "days since 1950-01-01 00:00:00" ;
reftime:standard_name = "forecast_reference_time" ;
reftime:long_name = "forecast reference time" ;
int leadtime(time) ;
leadtime:units = "days since 1950-01-01 00:00:00" ;
leadtime:standard_name = "forecast_period" ;
leadtime:long_name = "Time elapsed since the start of
the forecast" ;
leadtime:bounds = "time_bnd" ;
int time_bnd(time, time_bnd) ;
time_bnd:units = "days since 1950-01-01 00:00:00" ;
int realization(ensemble) ;
realization:standard_name = "realization" ;
realization:long_name = "Number of the simulation in
the ensemble" ;
char experiment_id(ensemble, string4) ;
experiment_id:standard_name = "experiment_id" ;
experiment_id:long_name = "Experiment identifier" ;
char source(ensemble, string50) ;
source:standard_name = "source" ;
source:long_name = "Method of production of the data" ;
char institution(ensemble, string15) ;
institution:standard_name = "institution" ;
institution:long_name = "Institution responsible for
the forecast system" ;
float level(level) ;
level:data_type = "float" ;
level:units = "hPa" ;
level:axis = "Z" ;
level:standard_name = "air_pressure" ;
level:positive = "up" ;
float geopotential(ensemble, time, level, latitude, longitude) ;
geopotential:data_type = "float" ;
geopotential:units = "m2 s-2" ;
geopotential:unit_long = "square_meter_per_square_second" ;
geopotential:standard_name = "geopotential" ;
geopotential:long_name = "geopotential" ;
geopotential:cell_methods = "leadtime: mean (interval 1
day)" ;
geopotential:_FillValue = 1.e+12f ;
// global attributes:
:Conventions = "CF-1.0" ;
:Generator = "SeasPy v1.1" ;
:Created = "Fri Nov 10 15:09:50 2006" ;
:References =
"http://www.ecmwf.int/research/demeter/index.html"; ;
:Comment = "Data interpolated from original model grid
into a regular grid. Data restrictions: none" ;
data:
skip lat, lon and level
reftime = 18659, 18659, 18659, 18659, 18659, 18659, 18748, 18748,
18748, 18748, 18748, 18748, 18840, 18840, 18840, 18840, 18840, 18840,
18932, 18932, 18932, 18932, 18932, 18932 ;
leadtime = 18673, 18701, 18732, 18762, 18793, 18823, 18762, 18793,
18823, 18854, 18885, 18915, 18854, 18885, 18915, 18946, 18976, 19007,
18946, 18976, 19007, 19038, 19066, 19097 ;
skip time_bnd
realization = 0, 1, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2, 3, 4, 5, 6, 7, 8, 0,
1, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2, 3, 4, 5, 6,
7, 8, 0, 1, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2, 3, 4, 5, 6, 7, 8 ;
experiment_id
"cnrm",
"cnrm",
"cnrm",
"cnrm",
"cnrm",
"cnrm",
"cnrm",
"cnrm",
"cnrm",
"crfc",
"crfc",
"crfc",
"crfc",
"crfc",
"crfc",
"crfc",
"crfc",
"crfc",
"lody",
"lody",
"lody",
"lody",
"lody",
"lody",
"lody",
"lody",
"lody",
"scnr",
"scnr",
"scnr",
"scnr",
"scnr",
"scnr",
"scnr",
"scnr",
"scnr",
"scwf",
"scwf",
"scwf",
"scwf",
"scwf",
"scwf",
"scwf",
"scwf",
"scwf",
"smpi",
"smpi",
"smpi",
"smpi",
"smpi",
"smpi",
"smpi",
"smpi",
"smpi",
"ukmo",
"ukmo",
"ukmo",
"ukmo",
"ukmo",
"ukmo",
"ukmo",
"ukmo",
"ukmo" ;
source
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, ARPEGE3/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/OPA8.2, System 0, Method 1 ",
"DEMETER, ECHAM4/OPA8.2, System 0, Method 1 ",
"DEMETER, ECHAM4/OPA8.2, System 0, Method 1 ",
"DEMETER, ECHAM4/OPA8.2, System 0, Method 1 ",
"DEMETER, ECHAM4/OPA8.2, System 0, Method 1 ",
"DEMETER, ECHAM4/OPA8.2, System 0, Method 1 ",
"DEMETER, ECHAM4/OPA8.2, System 0, Method 1 ",
"DEMETER, ECHAM4/OPA8.2, System 0, Method 1 ",
"DEMETER, ECHAM4/OPA8.2, System 0, Method 1 ",
"DEMETER, ECHAM4/OPA8.2, System 0, Method 1 ",
"DEMETER, IFS23R4/HOPE-E, System 0, Method 1 ",
"DEMETER, IFS23R4/HOPE-E, System 0, Method 1 ",
"DEMETER, IFS23R4/HOPE-E, System 0, Method 1 ",
"DEMETER, IFS23R4/HOPE-E, System 0, Method 1 ",
"DEMETER, IFS23R4/HOPE-E, System 0, Method 1 ",
"DEMETER, IFS23R4/HOPE-E, System 0, Method 1 ",
"DEMETER, IFS23R4/HOPE-E, System 0, Method 1 ",
"DEMETER, IFS23R4/HOPE-E, System 0, Method 1 ",
"DEMETER, IFS23R4/HOPE-E, System 0, Method 1 ",
"DEMETER, ECHAM5/OM1, System 0, Method 1 ",
"DEMETER, ECHAM5/OM1, System 0, Method 1 ",
"DEMETER, ECHAM5/OM1, System 0, Method 1 ",
"DEMETER, ECHAM5/OM1, System 0, Method 1 ",
"DEMETER, ECHAM5/OM1, System 0, Method 1 ",
"DEMETER, ECHAM5/OM1, System 0, Method 1 ",
"DEMETER, ECHAM5/OM1, System 0, Method 1 ",
"DEMETER, ECHAM5/OM1, System 0, Method 1 ",
"DEMETER, ECHAM5/OM1, System 0, Method 1 ",
"DEMETER, GloSea, System 0, Method 1 ",
"DEMETER, GloSea, System 0, Method 1 ",
"DEMETER, GloSea, System 0, Method 1 ",
"DEMETER, GloSea, System 0, Method 1 ",
"DEMETER, GloSea, System 0, Method 1 ",
"DEMETER, GloSea, System 0, Method 1 ",
"DEMETER, GloSea, System 0, Method 1 ",
"DEMETER, GloSea, System 0, Method 1 ",
"DEMETER, GloSea, System 0, Method 1 " ;
institution
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF ",
"ECMWF " ;
--
________________________________________
Francisco J. Doblas-Reyes
European Centre for Medium-Range
Weather Forecasting (ECMWF)
Shinfield Park, RG2 9AX
Reading, UK
Tel: +44 (0)118 9499 655
Fax: +44 (0)118 9869 450
f.doblas-reyes@xxxxxxxxx
_______________________________________
_______________________________________________
CF-metadata mailing list
CF-metadata@xxxxxxxxxxxx
http://www.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
--
Steve Hankin, NOAA/PMEL -- Steven.C.Hankin@xxxxxxxx
7600 Sand Point Way NE, Seattle, WA 98115-0070
ph. (206) 526-6080, FAX (206) 526-6744
