Re: draft cdl for goes magnetic field data

• To: Dan Wilkinson <Daniel.C.Wilkinson@xxxxxxxx>
• Subject: Re: draft cdl for goes magnetic field data
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Thu, 26 Jul 2007 16:46:03 -0600

Dan Wilkinson wrote:

John,
Ah, you found our other bone of contention.

GOES are geosynchronous and traditionally only the longitude is needed for most purposes. Some users, however, are interested in very slight variations and will want to know the exact inclination which is nominally zero, but in practicality can be a small fraction of a degree. Those users may also want the time and longitude for the daily northward crossing of the equator. I thought that the most efficient means of providing this information would be in an external file. Eventually we will also create files for GOES X-rays and energetic particles that will also be able to access them.

making files self contained is a pretty important goal.

OTOH, currently your records are only 36 bytes long. adding lat/lon/alt floats 
would add 12 more bytes, a significant increase. It wouldnt matter as much if 
you plan to have a lot more fields in there.

also i forgot to mention that you want to add some global attributes like the 
ones here:

http://www.unidata.ucar.edu/software/netcdf-java/formats/DataDiscoveryAttConvention.html

The data providers were pretty particular about the lengthy names for the magnetic vector components. What are the issues in that regard? If there is a hard limit I can see if they would be willing to meet those restrictions.

There are no hard limits, "long_name" is typically used to display choices or 
plot labels. longer descriptions are absolutely vital, but you could use a different 
attribute name. but leave them if you need to.

Regards,
Dan


I thought that the best

John Caron wrote:

Hi Dan

A few comments:

1. the cdl had
        float t2_filtered_counts(record);

t1_filtered_counts: long_name = "Magnetic torquer 2 counts smoothed by one of 2 algorithms.";

            t1_filtered_counts: units = "counts";
            t1_filtered_counts: valid_range = -525.0f, 525.0f;
            t1_filtered_counts: missing_value = 9999.0f;

instead of

        float t2_filtered_counts(record);

t2_filtered_counts: long_name = "Magnetic torquer 2 counts smoothed by one of 2 algorithms.";

            t2_filtered_counts: units = "counts";
            t2_filtered_counts: valid_range = -525.0f, 525.0f;
            t2_filtered_counts: missing_value = 9999.0f;


2. the long names should be shorter, eg i would change

Hp: long_name= "magnetic field vector component, points northward, perpendicular to orbit plane which for a zero degree inclination orbit is parallel

                      to Earth's spin axis.";

to something like

    Hp: long_name= "magnetic field vector component (northward)";

Hp: description= "magnetic field vector component, points northward, perpendicular to orbit plane which for a zero degree inclination orbit is parallel

                      to Earth's spin axis.";

3) the units seem ok, except for count should be "" (empty string). I might add display_units = "counts";

4) the main thing you are missing is geolocation info; is thats available?.

Dan Wilkinson wrote:

Hi John,

I'm writing to take you up on your offer to review our NetCDF file format for GOES Magnetometer data. I've attached a CDL file with our latest draft. The format uses what seems to be the most popular method of storing event time which looks like this "milliseconds since 1970-01-01 00:00:00.0 UTC." We choose milliseconds because the data samples are taken at half second intervals. Is it correct to assume that the various APIs will make it unnecessary for users to ever have convert to and from milliseconds and calendar type dates? I'm sure the answer must be true but since this a very important issue I just wanted to double check.

yes, udunits will convert to calendar dates.

The units for the magnetic flux values are nano Tesla, and udunits gives two ways to represent this. 1) Gamma, which means the same thing but some feel that is is becoming an obsolete term. 2) Tesla is supported along with the scale prefix nano, which can be combined as nT -- this is what we would like to do. This use of a scale prefix isn't supported by the CF conventions, however, I heard in a recent Unidata seminar that the CF conventions are more suited towards model output and not so much for observational data.

CF/udunits supports nano = 10-9, alias "n", see section 3.1

Finally, the NetCDF-java documentation mentions structures as a future means of organizing record oriented data, and I like that idea. What is your opinion on using structures at this point even though they are not universally supported within the APIs?

You are already using the "record structure" with the unlimited dimension, which is the only option you have with netcdf-3 files.

Thanks in advance for taking time to review our CDL. Whatever we decide at this point will likely be propagated to many other datasets so we want to be careful with our choices.

your welcome, good luck!

Regards,
Dan Wilkinson
303-497-137

==============================================================================
To unsubscribe netcdf-java, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================