Report on the NetCDF Users Working Group

In September 1992, a group of interested parties from COMET, Unidata, and FSL (NOAA Forecast Systems Lab) formed to develop conventions for representing atmospheric science data. The goals of this group are to identify a minimum set of variables and attributes required to identify data sets of common use and learn more about writing generic software which does not make assumptions about a particular netCDF definition.

Meetings have been held on an irregular basis. Initially, the group focused on standard definitions such as station identification and time representation. The first standard definition drafted contained a description of surface observation data (see attached definition). The group is just now finishing up the definition for gridded data which was based on GRIB formatted data sets. This definition should be available shortly upon completion of discussion on how to represent navigation information.

General Conventions

The NUWG agreed that all time fields will be represented such to be compatible with the UDUNITS library. We agreed that time should be represented as a double such as:

	double time
		:units = 	
		:long_name =	
		:_FillValue =

Defining time as a double gives the most flexibility to the widest range of data sets.

The variable names chosen for netCDF variables will be up the supplier of the data set. The agreement by this group is that all code written to read netCDF data will NOT be variable name dependent. Code will be written generically using the netCDF inquire functions or will employ some sort of table look-up algorithm.

The group also agreed to attach "long_name" attributes to all variables as a means of self-documenting files. When a user performs an ncdump, "long_name" attributes should help that user understand what is contained in the file.

Any units used must be compatible with the UDUNITS library and consistent for a given variable (ie. all temperatures written out in degrees C, even though they are reported in F or C). The "units" attribute becomes critical to generic code which performs numerical computations.

In general, the netCDF definitions for a data set that we create are considered a minimum set of variables for data exchange. The data provider is free to add additional fields as needed for quality control, further identification, or other internal uses.

Station Data

Many of the early meetings focused on some basic conventions that pertained to identifying observed data. Common problems with all types of station include station identification and reporting the time of the observation.

To identify fixed location stations, we agreed that the numeric WMO id would be the identifier of choice. As mentioned above, data suppliers could add an internal id in addition to the WMO id. To fully identify stations, a station name, the latitude, longitude and elevation of the station will be required. (See the SAO definition which follows for an example). Again, the "units" attributes are required for these variables so that users, for example, can determine if latitude is in degrees north or south.

Gridded Data

Grids presented a particular problem as we all wanted to store more that one model run in a file, requiring that time be the unlimited dimension. But, the problem with grids is that there are two times, a reference time (T0) of the grid and the valid (or forecast) time. In our grid proposal, we introduced the concept of a "record" to be the unlimited dimension. The idea is to define referential attributes of a variable such as:

	float u(record, z, x, y);
		u:record = "valtime, reftime";

	double valtime(record);
		valtime:long_name = "valid time of model";
		valtime:units = "minutes since (1993-1-1 00:00:00.0)";

	double reftime(record);
		reftime:long_name = "reference time of model";
		reftime:units = "minutes since (1993-1-1 00:00:00.0)";

This indicates that the "nth" record is defined by the "nth" value in both the "valtime" and "reftime" variables. They contain the actual times for the grid. This allows us to refer to both valid and reference time as a single unlimited dimension.

The referential concept is also useful in the second problem with hybrid grids which contain a relative vertical coordinate "z". An example is where a vertical level of a grid point value may be related to pressure, in variable "p", or to the virtual potential temperature in variable "vpt". So the referential attribute can be used as follows:

	float u(record, z, x, y);
		u:z = "vpt, p";

This defines for grid value u(1,2,1,1) that the value for the "z" dimension of "u" can be found in p(1,2,1,1) or vpt(1,2,1,1).

In a simpler case, where the grid is defined for set levels, we can define such referential attributes as:

	dimensions:
		z = 3;

	variables:
		float u(record,z,x,y);
		      u:z = "levels";

		float levels(z);

	data:
		levels = 1000.0, 850.0, 500.0;

Definitions for grid navigation are still under discussion. The group has investigated creating navigation attributes which are defined by strings in a syntax that can be understood by UDUNITS. Variables would reference such attributes as follows:

	float u(record,z,x,y);
 	      u:_Navigation_var = "grid1";

	char grid1(len);
	     grid1:projection = "polar stereographic";
	     grid1:La1 = "45.0 degrees_N";
	     grid1:Lo1 = "110.0 degrees_W";
	     grid1:Dx = "60 km";
	 	.
		.
		.

	data:

	grid1 = "Polar Stereographic Grid, GRIB Edition 1";

Participation

NUWG is open to all interested parties. The irregular meetings are announced on and supplemented by some discussion on the NUWG mailing list (nuwg@comet.ucar.edu). Any interested party is encouraged to join the mailing list by sending an add request to nuwg-request@comet.ucar.edu. Unidata is also planning to support a directory for the maintenance of netCDF conventions documents on discipline-specific conventions.