Dear all,
Thanks Ed for this overview of Copernicus! I’m also very excited about the data
this programme is producing.
Just to be a little legalistic, Copernicus is actually not an ESA programme –
in fact it’s from the European Commission, and that’s one of the reasons that
EUMETSAT is running a few of the Copernicus missions. As the CGMS liaison with
the CF community, I would also be happy to hear any input that you have
concerning Copernicus data (even if it’s not directly from EUMETSAT, which is
the case for Sentinel-1 and -2, as well as some of the Sentinel-3 products).
What do you like, what can we do better? Particularly in the area of data and
metadata formatting standards, we’re trying to make sure that the data is
well-formatted and easily accessible for users.
Christopher, I’ve also noted the issues that you’re talking about – although I
wish the issue weren’t existent, it is gratifying to see that this is relevant
for users and not just data formatting purists like myself. Thanks for the
valuable input.
Best regards,
Daniel
From: netcdfgroup-bounces@xxxxxxxxxxxxxxxx
<netcdfgroup-bounces@xxxxxxxxxxxxxxxx> On Behalf Of Lynnes, Christopher S.
(GSFC-5860)
Sent: Wednesday 17 October 2018 16:03
To: Ed Hartnett <edwardjameshartnett@xxxxxxxxx>
Cc: NetCDF Group List <netcdfgroup@xxxxxxxxxxxxxxxx>; John Caron
<jcaron1129@xxxxxxxxx>; Ethan Davis <edavis@xxxxxxxxxxxxxxxx>
Subject: Re: [netcdfgroup] ESA Copernicus program and netCDF-4
Ed,
Thanks for sending this out. As it happens, I have been experimenting with the
readability of some Sentinel Level 2 netCDF data in my go-to netCDF tool,
Panoply.
In general, it's pretty readable by Panoply, EXCEPT for the handling of
auxiliary coordinates for geolocation.
For instance, Sentinel 1A OCN IW (Ocean Winds) contains auxiliary coordinates
owiLat and owiLon (yay!), but does not include the :coordinates attribute.
I've seen this shortcoming enough that it's the first thing I check, but the
average science user may be clueless in this area. Also, it takes me only 30
seconds to fix it with ncatted (thanks, NCO!), which again, the average user
might not have in their toolbox.
The second case was Sentinel 3A Level 2 OLCI (Veg Index, etc.). Here, the
auxiliary coordinates are actually in a separate file. The Sentinel Toolbox
knows to look in that file, but other off-the-shelf netCDF software may not.
Again, NCO to the rescue to add the coordinates into the data file, plus add
the :coordinates attribute.
I'm actually going to bring this up nex tweek at an upcoming meeting of the
international WGISS group, but as a community, we should think about how to
handle these cases where very minor omissions or deviations make the data
appear unreadable (at least in geographic context). (I have also seen cases in
NASA where including commas in the :coordinates attribute ("lat, lon") rendered
them undiscoverable by the software.)
On Oct 17, 2018, at 07:22, Ed Hartnett
<edwardjameshartnett@xxxxxxxxx<mailto:edwardjameshartnett@xxxxxxxxx>> wrote:
Fellow NetCDFers,
While on leave, I recently had a chance to stop by two science events from the
European Space Agency Copernicus program.
I learned a lot, and thought I would share it. I very much welcome correction
or elaboration from anyone involved with the Copernicus program.
Starting with the Sentinel-3 spacecraft, all Copernicus ground data, from level
1 up, are in NetCDF-4.
I was not previously aware of the scope and ambition of the Copernicus system.
There a snappy high-level description here:
https://www.youtube.com/watch?v=xcflQZJ5n88.
Here are some notes:
* Sentinel-1 and sentinel-2 missions are successful and on-going. But who
cares, since they don't use netCDF!
* NetCDF-4 was selected for all data starting with the Sentinel-3 mission,
because of the ubiquity of netCDF combined with built-in data compression.
(Everyone loves that built-in data compression!)
* There are currently two Sentinel-3 spacecraft in polar orbit, 2 more
being built on the ground as replacements, so that there will always be 2
operational, for 15-30 years. (And this will be the case for all Sentinel
missions.)
* The sentinel-4 mission involves two instruments on the EUMESATs 3rd
generation satellites. There are two spacecraft in geocentric orbit and more
replacements standing by. (These are the EUMETSAT equivalent to NOAA's GOES
satellites.) These include a *sweet* hyper-spectral spectrometer.
* Sentinel-5P (for precursor) is a bonus spacecraft to fill a potential
instrument gap between existing instruments and the sentinel-5 satellites,
which are being built now for 2021 launch.
* Sentinel 6-10 missions are also planned, as is a gap-filling Sentinel-6P
mission.
* All Copernicus data are freely available in near real-time.
* As is generally the case, each generation of instrument sends lots more
data than the previous generation. So most(?) of their data holdings are
currently in netcdf-4.
* The Copernicus data stream is fast enough, and of high-enough resolution,
that it is being used in natural disaster situations (for example to show which
specific neighborhoods and houses are flooded).
* The data are also used in law enforcement, for environmental pollution
detection.
Copernicus is a very impressive program for Earth observation. The quality and
capability of the instruments, combined with the long-term support for the
program, will guarantee that Copernicus will become the backbone of a lot of
science.
I would very much welcome any comments from any Copernicus data users or
programmers about how netCDF-4 is working for you, and what are the pain-points
if any.
I would also be happy to share information on high-performance computing with
netCDF data, which can involved other packages like pnetcdf and PIO, to give
good parallel I/O performance.
Please contact me if you are using Copernicus data, and tell me about what you
are doing with it.
Thanks,
Ed Hartnett
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
—
Christopher Lynnes NASA/GSFC 301-614-5185
"Perfection is achieved, not when there is nothing more to add, but when there
is nothing left to take away.” A. de St. Exupery
Any email message from EUMETSAT is sent in good faith but shall neither be
binding nor construed as constituting a commitment by EUMETSAT, except where
provided for in a written agreement or contract or if explicitly stated in the
email. Please note that any views or opinions presented in this email are
solely those of the sender and do not necessarily represent those of EUMETSAT.
This message and any attachments are intended for the sole use of the
addressee(s) and may contain confidential and privileged information. Any
unauthorised use, disclosure, dissemination or distribution (in whole or in
part) of its contents is not permitted. If you received this message in error,
please notify the sender and delete it from your system.
