NOTE: The cf-satellite mailing list is no longer active. The list archives are made available for historical reasons.
Jim, > I'm confused about a few things. If you don't mind spending a bit of > time clarifying, I'd appreciate it. Oops, sorry, it looks like I'm the one who is confused! > Here are my confused questions. > > * Isn't the _FillValue attribute the one that maps to '_'? (See the > discussion of _FillValue and missing_value here > <http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/Attribute-Conv > entions.html>.) Yes, you're right of course. Sorry for mixing up "_FillValue" and "missing_value". The latter was at one time deprecated, but now that deprecation has been removed from the latest CF conventions standard. > * Is the flag_values attribute considered applicable in the case of > floating point variables? That was my understanding. CF 1.5 says: ... The flag_values attribute is the same type as the variable to which it is attached, and contains a list of the possible flag values. I can't see anywhere that restricts flag_values to integers. Of course the "flag_masks" attribute wouldn't have much meaning for floating-point values, but I wasn't suggesting using "flag_masks". > * In what sense is this suggestion making a change to the > interpretation of the missing_value attribute? If "this suggestion" means extending missing_value to permit multiple values, then the only change it makes is changing a one-to-one mapping into a one-to-many mapping. Since my example of breaking ncdump/ncgen backward compatibility doesn't apply to the "missing_value" attribute, that may be harmless, unless there is other existing software that depends on "missing_value" having a single value. So I withdraw my confused objection to using "missing_value". However, I still think "flag_values" would also work for this purpose, especially if there were other reasons for special values than that they indicated a special type of missing value. Also "missing_values" might be more descriptive than "missing_value" for a multiple-value attribute, following the example of "flag_values", but I'm also in favor of keeping the additions of new CF attributes to a minimum, which argues for just using "missing_value". --Russ _____________________________________________________________________ Russ Rew UCAR Unidata Program russ@xxxxxxxxxxxxxxxx http://www.unidata.ucar.edu > Grace and peace, > > Jim > > On 7/15/2011 3:09 PM, Russ Rew wrote: > > Hi Jim, > > > >> In the ESIP Federation CF-Satellite session yesterday (July 14, 2011), I > >> suggested that it would be good to have a standard scheme for handling > >> multiple marker values in satellite data that indicate missing data. > >> There is confusion in the current documentation (between netCDF and CF) > >> about the proper use of the missing_value attribute, in particular about > >> whether or not it is acceptable for it to be a vector attribute. I > >> think there is good reason to allow it to be a vector attribute, and > >> that allowing the attribute to be a vector quantity is useful beyond the > >> satellite / remote sensing arena. The problem that arises with having a > >> vector missing_value attribute is the assignment of meanings to the > >> different values. > >> > >> One way to deal with this is to have a missing_value_meanings > >> attribute. The missing_value_meanings attribute would be formed in the > >> same way as the flag_meanings attribute. There would be one string of > >> non-whitespace characters per element of the missing_value attribute, > >> separated by spaces. Each string would be a human-readable phrase > >> (delineating the words of the phrase using underscores or camel-casing) > >> that explained the meaning associated with the corresponding > >> missing_value element. Here's an example fragment of cdl. > >> > >> float foo(points); > >> > >> foo:missing_value = -999.1, -999.2; > >> foo:missing_value_meanings = "InputsOutOfRange > >> SolutionDidNotConverge"; > > I agree that it would be very useful to have an attribute for multiple > > special values for satellite data as well as other kinds of data. IIRC, > > you gave an example of 10 different kinds of missing values for an NPP > > sensor. > > > > I'm a little concerned about breaking backward compatibility by > > extending the "missing_value" attribute for this purpose, because there > > may be software that depends on a one-to-one mapping between a > > missing_value for a variable and some alternate representation for it, > > such as the '_' notation ncdump already uses. The fact that the ncdump > > and ncgen utilities are inverses of each other, converting netCDF data > > to CDL and back to netCDF without changing the data, requires a single > > value for the missing value to avoid loss of information when converting > > from '_' back to the corresponding value. > > > > I suggest just using the existing CF flag conventions for this purpose > > instead, using the variable attributes "flag_values", and > > "flag_meanings" for any number of special values for a variable. > > > > If one of those values is a missing value, the associated flag_meaning > > could just be "missing_value", but flag_meanings can also specify an > > arbitrary number of other nuanced kinds of missing values or various > > other kinds of special values. > > > > http://cfconventions.org/documents/cf-conventions/1.5/cf-conventions.htm > l#flags > > > > --Russ > > _____________________________________________________________________ > > > > Russ Rew UCAR Unidata Program > > russ@xxxxxxxxxxxxxxxx http://www.unidata.ucar.edu > > -- > Jim Biard > > Government Contractor, STG Inc. > Remote Sensing and Applications Division (RSAD) > National Climatic Data Center > 151 Patton Ave. > Asheville, NC 28801-5001 > > jim.biard@xxxxxxxx > 828-271-4900 > > > --Boundary_(ID_xMNVEAaJBvpOih/JfomxKw) > Content-type: text/html; charset=ISO-8859-1 > Content-transfer-encoding: 7BIT > > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> > <html> > <head> > <meta content="text/html; charset=ISO-8859-1" > http-equiv="Content-Type"> > </head> > <body text="#000000" bgcolor="#ffffff"> > Russ,<br> > <br> > I'm confused about a few things. If you don't mind spending a bit > of time clarifying, I'd appreciate it.<br> > <br> > Here are my confused questions.<br> > <ul> > <li>Isn't the _FillValue attribute the one that maps to '_'? (See > the discussion of _FillValue and missing_value <a > href="http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/Attribute-Conve > ntions.html">here</a>.)<br> > </li> > <li>Is the flag_values attribute considered applicable in the case > of floating point variables?</li> > <li>In what sense is this suggestion making a change to the > interpretation of the missing_value attribute?</li> > </ul> > Grace and peace,<br> > <br> > Jim<br> > <br> > On 7/15/2011 3:09 PM, Russ Rew wrote: > <blockquote cite="mid:10685.1310756965@xxxxxxxxxxxxxxxx" type="cite"> > <pre wrap="">Hi Jim, > > </pre> > <blockquote type="cite"> > <pre wrap="">In the ESIP Federation CF-Satellite session yesterday (J > uly 14, 2011), I > suggested that it would be good to have a standard scheme for handling > multiple marker values in satellite data that indicate missing data. > There is confusion in the current documentation (between netCDF and CF) > about the proper use of the missing_value attribute, in particular about > whether or not it is acceptable for it to be a vector attribute. I > think there is good reason to allow it to be a vector attribute, and > that allowing the attribute to be a vector quantity is useful beyond the > satellite / remote sensing arena. The problem that arises with having a > vector missing_value attribute is the assignment of meanings to the > different values. > > One way to deal with this is to have a missing_value_meanings > attribute. The missing_value_meanings attribute would be formed in the > same way as the flag_meanings attribute. There would be one string of > non-whitespace characters per element of the missing_value attribute, > separated by spaces. Each string would be a human-readable phrase > (delineating the words of the phrase using underscores or camel-casing) > that explained the meaning associated with the corresponding > missing_value element. Here's an example fragment of cdl. > > float foo(points); > > foo:missing_value = -999.1, -999.2; > foo:missing_value_meanings = "InputsOutOfRange > SolutionDidNotConverge"; > </pre> > </blockquote> > <pre wrap=""> > I agree that it would be very useful to have an attribute for multiple > special values for satellite data as well as other kinds of data. IIRC, > you gave an example of 10 different kinds of missing values for an NPP > sensor. > > I'm a little concerned about breaking backward compatibility by > extending the "missing_value" attribute for this purpose, because there > may be software that depends on a one-to-one mapping between a > missing_value for a variable and some alternate representation for it, > such as the '_' notation ncdump already uses. The fact that the ncdump > and ncgen utilities are inverses of each other, converting netCDF data > to CDL and back to netCDF without changing the data, requires a single > value for the missing value to avoid loss of information when converting > from '_' back to the corresponding value. > > I suggest just using the existing CF flag conventions for this purpose > instead, using the variable attributes "flag_values", and > "flag_meanings" for any number of special values for a variable. > > If one of those values is a missing value, the associated flag_meaning > could just be "missing_value", but flag_meanings can also specify an > arbitrary number of other nuanced kinds of missing values or various > other kinds of special values. > > <a class="moz-txt-link-freetext" href="http://cfconventions.org/documents/c > f-conventions/1.5/cf-conventions.html#flags">http://cfconventions.org/documen > ts/cf-conventions/1.5/cf-conventions.html#flags</a> > > --Russ > _____________________________________________________________________ > > Russ Rew UCAR Unidata Program > <a class="moz-txt-link-abbreviated" href="mailto:russ@xxxxxxxxxxxxxxxx";>russ@ > unidata.ucar.edu</a> <a class="moz-txt-link-freetext" hre > f="http://www.unidata.ucar.edu";>http://www.unidata.ucar.edu</a> > </pre> > </blockquote> > <br> > <pre class="moz-signature" cols="72">-- > Jim Biard > > Government Contractor, STG Inc. > Remote Sensing and Applications Division (RSAD) > National Climatic Data Center > 151 Patton Ave. > Asheville, NC 28801-5001 > > <a class="moz-txt-link-abbreviated" href="mailto:jim.biard@xxxxxxxx";>jim.biar > d@xxxxxxxx</a> > 828-271-4900 > </pre> > </body> > </html> > > --Boundary_(ID_xMNVEAaJBvpOih/JfomxKw)--
cf-satellite archives: