NOTE: The cf-satellite mailing list is no longer active. The list archives are made available for historical reasons.
Hi Randy... My read on the "Flags" in Section 3.5 of the CF Conventions is that the intended use is that a variable contains "flag values" which are described by the various "flag_" attributes (flag_values, flag_meanings, etc). While the implication is that the variable contains only "flag" values, I'm not certain that is essential. When you say your variables have "10s of millions of data points where each has a distinct quality flag", I am not sure if you are saying the variables _contain_ the quality flag value(s) or somehow they are in a different variable. For example, you might have actual data values in the range of 0:100, but if the data value at a point is "101" then it has a special meaning. If that is the case, could you not simply use the "flag_values" to indicate "101" is a flag value? Unless the netCDF library handles variables with these attributes in a special way, it would be up to the application that reads the data to pick up the "flag_" attributes and do something with the values that match. If this is the case, I don't see that your file size would increase substantially...of course, I could be mis-interpreting what you said, too. tom On Mon, Oct 31, 2011 at 11:12 AM, Randy Horne <rhorne@xxxxxxxxxxxxxxxxx> wrote: > In the application I am working on, there are specific types of products > where where the same quality flags apply to multiple variables. These > products will be contained in a single NetCDF file. > > The current CF conventions dictate that quality flags are attached to > specific variables. The implication is that comforming with CF conventions > would require the same quality flags to be stored multiple times in our > NetCDF product files. This is potentially problematic due to the resulting > increase in product file size. Our variables have on the order of 10s of > millions of data points where each has a distinct quality flag. > > This issue has come up before, and any design to optimize flags has been > dismissed due to the overriding objective of having variables that are > self-describing. > > Product file compression is a potential answer, but this presents its own set > of performance issues. > > Maybe flags could be handled in a similar manner as coordinates to allow > sharing among variables ? > > > > ..............End of Message ...............................--> > > > > > > _______________________________________________ > cf-satellite mailing list > cf-satellite@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > http://www.unidata.ucar.edu/mailing_lists/ > -- Tom Whittaker University of Wisconsin-Madison Space Science & Engineering Center (SSEC) Cooperative Institute for Meteorological Satellite Studies (CIMSS) 1225 W. Dayton Street Madison, WI 53706 USA ph: +1 608 262 2759
cf-satellite archives: