[cf-satellite] How to store lightning detection products in a CF compliant manner ?

• To: cf-satellite@xxxxxxxxxxxxxxxx, cf-metadata@xxxxxxxxxxxx
• Subject: [cf-satellite] How to store lightning detection products in a CF compliant manner ?
• From: Randy Horne <rhorne@xxxxxxxxxxxxxxxxx>
• Date: Fri, 10 Feb 2012 10:23:34 -0500

(this is long winded….sorry)

The next generation GOES program will be providing operational level 2 
lightning detection products.

These products have the following essential characteristics.

(1)

There will be a variable number of lightning observations in a NetCDF product 
file.   These lightning observations are referred to as “flashes”.

(2)

Each flash contains a variable number of “groups”.  A flash is composed of 
groups that satisfy specific spatial and temporal window/envelope constraints.

(3)

Each group contains a variable number of “events”.  A group is composed of 
events that satisfy specific spatial and temporal window/envelope constraints.

(4)

Flashes, groups, and events all have data elements that are not related to 
coordinate variables.  For example: a flash has optical energy, areal extent, 
and flags,; a group has optical energy, area extent, and quality flags and; an 
event has optical energy and quality data.

(5)

Flashes, groups, and events all have lat/lon and time coordinates.  However, 
from a user access point of view, the lat/lon and time coordinates associated 
with flashes are what is most important.

(6)

There is wide variability (approximately 2 orders of magnitude) in the number 
of events per group.  There is wide variability (approximately 1 order of 
magnitude) in the number of groups per flash.

 

I view this product as being closest to a discrete sampling geometry “point” in 
the CF metadata document because the flashes have no implied coordinate 
relationship to other points.  However, because of a flash’s composition (two 
nested levels of variable length records), “point” doesn’t work very well.

If it were compliant to use the discrete sampling geometry “contiguous ragged 
array representation” with points, there is still an issue because of the two 
nested levels of variable length records.  Also, the implicit need for the 
contiguous ragged arrays to be composed of scalars causes problems because all 
three levels, flashes, groups, and events have multiple data elements.

Without being constrained by the current set of CF conventions, it would seem 
the best technical option is to use variable length arrays (VLEN), and 
structures (compound).  Something like ….

types:

  compound events_t {             // type for a single event

    int   optical_energy ;

    int   area_extent ;

    int   quality ;

  }

  event_t   some_event_t (*) ; // type for some event

  compound group_t                // type for a single group

    int   optical_energy ;

    int   area_extent ;

    int   quality ;

    some_event_t  event ;

  }

  group_t   some_group_t (*) ; // type for some group

  compound flash_t                   // type for a single flash

    int   optical_energy ;

    int   area_extent ;

    int   quality ;

    some_group_t  group ;

  }

dimensions:

  num_flashes = Unlimited

variables:

  flash_t   flash (num_flashes)  // here is the data variable

 

**Although not shown, there are lat, lon, and time coordinate variables.

Any thoughts on how to proceed here ?