^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mary Jo Brodzik, Special Projects Lead, 303-492-8263
NSIDC/CIRES, Univ. of Colo. at Boulder, 449 UCB, Boulder, CO 80309-0449
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
---------- Forwarded message ----------
Date: Tue, 25 Oct 2011 10:14:55 +0200
From: Peter Miu <address@hidden>
To: 'Martin Raspaud' <address@hidden>
Cc: "address@hidden" <address@hidden>
Subject: Re: [cf-satellite] cf-satellite Digest, Vol 14, Issue 1
Morning Martin,
Sorry to hear that you were at the user conference but were not able to
attend the presentation I gave.
I've attached the presentation for information, and I welcome feedback on
the subject matter
from subscribers of this mailing list.
The southMostLine, eastMostPixel, northMostLine, westMostPixel global
attributes are defined
to inform users who are used to using these parameters of the subarea
ordered.
This is particularly useful when users order a lot of data as it reminds
them
of the contents of the data. Yes I agree that this is not essential but
adding meta-data like
this even though it may only be useful to some users adds value to the
data.
Knowing the needs of your core users will help determine what type of
meta-data to specify
and your input is very useful.
I agree that implementing the HRV in 2 arrays is a good way to organise the
data.
This avoids a lot of cells being 'empty' which would be the case if the
data was stored in one array.
We will be revising the MSG15 NetCDF data set to include more data
stored in the native format so this will give us an opportunity to
implement the HRV channel
as well.
Regarding your lat lon comment, Unidata provides Standard Coordinate
Transforms to
Geo-locating data onto a earth projection, see below:
http://www.unidata.ucar.edu/software/netcdf-java/reference/StandardCoordinateTransforms.html
This removes the need to store the lat and lon arrays in your data if you
are using Unidata
tools to visualise the data. For users who don't use these tools, it would
require them to
map these transforms to lat and lon i.e. they need to do extra programming
to visualise
the data. As I've mentioned before, usability of the data for all users is
very important
in our development. So even though the lat and lon uses space in the
NetCDF file, it is
provided for usability reasons.
If you take a look at the presentation, we proposed the concept of
tailoring NetCDF data sets
to the user's needs. Whether the lat and lon arrays, coordination
transform or any other
method of geo-locating the data is included in the NetCDF file or not can
be addressed here.
We just need to get enough users requesting the implementation of tailored
NetCDF files (hint :o).
Regarding the band discussion and taking Tom's reinforcing Email into
account (Thanks Tom ... might see you next year
in Boulder if you are there to discuss more CF conventions with you) I
think I understand what bands are now.
Creating a 3D array indexed by a band or channel with lat and lon is not a
problem. For me, I think the
channel/band should be an coordinate variable so that it can be treated
like 'level', "altitude" or "pressure level"
by the Unidata tools for visualisation. I'm also in favour of the
standard_name="channel" rather than "band"
as this is clearer to me, of course this is only my input to this
discussion.
I'm don't know who you have been talking to at EUMETSAT regarding bands but
now I know this is a user need,
I can address this in our Format Advisory Group. Thanks for drawing my
attention to this.
In case anyone reading this thread have problems in understanding some of
the term we are using.
I've found the following tutorial to help understanding the basic NetCDF
concepts and I hope this is useful.
http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-tutorial.pdf
Regards,
Pete.
-----Original Message-----
From: Martin Raspaud [mailto:address@hidden]
Sent: Monday, October 24, 2011 5:08 PM
To: Peter Miu
Cc: address@hidden
Subject: Re: [cf-satellite] cf-satellite Digest, Vol 14, Issue 1
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 24/10/11 15:35, Peter Miu wrote:
Hi Martin,
Hi Pete,
Thanks for the fast feedback. Before I answer your questions, I would
just like to let you know that the EUMETSAT Data Centre is currently
planning the implementation NetCDF formats as the common delivery format
for all orderable data sets from the Archive.
A pilot has already been performed for the level 1B and 2 products
from the Metop-A ASCAT instrument and these are available for user
feedback under the following URL:
http://gsics.eumetsat.int/thredds/ascat.html
The ASCAT NetCDF products were also presented to users in this year's
EUMETSAT user conference held in Oslo.
The development of usable NetCDF formats is our top priority as we
want to ensure that all users (novice and expert) can immediately use
our data so it is very important that guidelines regarding best
practises, filename conventions, CF-Conventions and standards are
followed. A Format Advisory Group has been set up at EUMETSAT to
discuss these guidelines with a view to publishing them so that users
are aware how our data is organised and also to propose updates to the
CF-conventions for satellite data as work is needed here (hence the
existence of this mailing list! :o)
I was at the EUMETSAT conference this year :)
Here are the answers to your questions:
- Why are southMostLine, eastMostPixel, northMostLine, westMostPixel, and
numberOfChannels dimensions and not attributes ?
Apart from the HRV channel, these values remain constant for all
channels in the data sets so we can be specify them as global
attributes. If the channel arrays store different 'sub-area' then we
would need these value to be stored as variable attributes for each
channel array. This is not the case for our NetCDF data set i.e. all
channel arrays hold the same area for the same time.
Note, the NetCDF file follows the Unidata recommendation for
identifying the coordinates of the data in the channel arrays
(Coordinate System Object Model - Current Encodings). Longitude and
latitude arrays exist in the data set and their contents are used to
geo-locating the pixel counts for each channel onto a projection.
See: http://www.unidata.ucar.edu/software/netcdf-java/CDM/
Ok. I don't really understand the interest of having these
dimensions/attributes though, since I guess you will have to define
lon/lats anyway for each different resolution...
- - How would you deal with the inclusion of the HRV channel ?
We have not implemented the HRV channel in our NetCDF data sets as GSICS
does not use this channel for calibration.
To implement this channel in our NetCDF data sets, we would just need
to add another longitude and latitude array specifying the location of
each HRV pixel counts and a 2D array holding the HRV pixel counts.
Sounds good, but the HRV has 2 sub-areas: would you have a different
variable for each or would you group the in one big channel ? (We
implemented the latter, since the netcdf compression works well in this
case)
- - Are the longitudes and latitudes values different from what a
"vertical perspective" with right parameters would provide ? If they do,
why ? If they don't, why not include a grid-mapping definition ?
Not sure what you mean here, perhaps the following explain it:
For the pixel count in ch1( y, x ), it is located at latitude lat( y, x )
and longitude lon( y, x ).
By assigning the lat and lon as the coordinates system for any channel
array, applications know how to geo-locate its contents.
float lat(y,x);
float lon(y,x);
int ch1(y,x);
ch1:coordinates="lat lon";
int ch2(y,x);
ch2:coordinates="lat lon";
My question was if the lon/lats differ from what one would get from running
proj.4 with parameters like
+proj=geos +lon0=0 +a=... etc.
If one would get the same lon/lat data, then I would recommend to record
also these parameters in a grid mapping variable. In this case, the
lon/lats arrays would even become optional.
- - On the cf-satellite list, we have long been discussing "band"s. Did
you consider using those ?
Sorry, I don't know what bands are.
Well, this is a bit sad :(. We have been working quite a lot on this list
toward having band variables that would correctly describe satellite
channels (or bands). My colleagues and I have tried to drag Eumetsat's
attention to this list and the work done here on several occasions. Tom
Whittaker was also at the Eumetsat conference last year in Cordoba to
present CF conventions and the possibility to apply these to satellite. But
apparently that was not enough...
You can read more on the band variable and related questions here:
http://www.unidata.ucar.edu/mailing_lists/archives/cf-satellite/2011/thread.html
- - Did you consider grouping channels in 3d arrays instead of having them
separated ?
I'm not sure what value a 3D arrays would give to the data.
Applications like the Unidata can plot the arrays on top of each other if
needed.
As mentioned above, we want our data to be immediately usable by all
users and by following guidelines, we add values to the data as the
contents can be examined and plotted by existing free NetCDF applications
without any further processing by the users.
- - Could you provide in the file calibration coefficients needed to
convert radiances to reflectances and brightness temperatures ?
Yes we can add any variable attribute to the channel arrays to support the
needs of our users.
The power of NetCDF is as long as we don't remove or reorganise any
attributes and/or arrays, existing users and applications will not
'break' by the change. The only concern here is we need to ensure
that a standard convention is followed for representing measurements
that can be stored in different units (e.g. temperature can be stored as
Celsius, Fahrenheit, Kelvin, ... ). What unit is used should be based on
the needs of the 'core' users and this unit with a standard name should be
included in the CF-conventions.
That would be nice :) Thanks a lot.
Best regards,
Martin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Red Hat - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJOpX9pAAoJEBdvyODiyJI4mC0IAK/6m0+TWMtaVcRChVGLJ9VX
OY4OsSZQ+dceW4+YWGymBRUInUWChURdvy4eLCX9lPiz2TgOXbJkyvVcuK2iULtp
6oYTX+Fn3g3LhDM118rlYe5CApgFcg0bA62hqwXoM0ksrdQ6cbt1tz6aSV1kpkbL
4tCJrmLLen8mcYF3fSsH1xpKli5NUXEOwcf4jDWg9ieQAwic4RuD8bky1rshK3NS
DDJeVjkKETE9VBMsseupWmptuWWG7zzFH9qsMatHjn8tYixTfezwaURu3C8n00Fm
ugwHLO1uUkCb+jK7Ga1Kikftg7hFlATMTgLd1WRpZHyO0LqbbrgiLmvRHbsVYV0=
=waU8
-----END PGP SIGNATURE-----
