At EUMETSAT, we plan to use an external CharLS filter to compress our
disseminated L1C image data from the future MTG/FCI instrument. We get a
significant amount of extra compression over the in-built netCDF methods that
allows us to maximise the data we send over the available bandwidth.
We originally had to use a modified version of NetCDF to allow access to the
external filters for writing the data (the decompression by end-users could
still be accomplished fairly simply by having the filter plugin available to
the HDF-5 libraries). The latest versions of the NetCDF C library that support
use of external filters directly is a highly appreciated update from our
perspective. Although not all NetCDF implementations currently support the
use of external filters, hopefully this will evolve over time.
Regards
Tim Patterson
Dr. Tim Patterson
Instrument Functional Chain Team Data and Tools Co-ordinator
GEO Division
EUMETSAT
Eumetsat-Allee 1
64295 Darmstadt
Germany
Tel: +49 6151 807 487
E-mail: timothy.patterson@xxxxxxxxxxxx
Web: www.eumetsat.int
-----Original Message-----
From: netcdfgroup-bounces@xxxxxxxxxxxxxxxx
[mailto:netcdfgroup-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Thompson, Matt
(GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Sent: 19 November 2018 16:23
To: netcdfgroup@xxxxxxxxxxxxxxxx
Subject: [netcdfgroup] Using Fancy Compressors (zfp, etc) in netCDF
All,
This is a bit of a general question inspired by going to the "Compression for
Scientific Data" tutorial at SC18 last week. Namely, is there currently an easy
way to test out and try new compressors like zfp and SZ in netCDF?
I know they need to be compiled and registered and all with HDF5 (which I could
probably get to happen), but as my shop is a netCDF (Fortran) shop and not an
HDF5 shop, I can't quite use them until they can be accessed through netCDF.
A search online shows that perhaps once there was a "compress" branch of
netCDF-C, but it seems stale.
Is there any effort in Unidata to start supporting the HDF5 extra compressors?
Or are they supported and I just don't know enough advanced netCDF to know how
to use them (always possible!)?
Thanks,
Matt
--
Matt Thompson, SSAI, Sr Scientific Programmer/Analyst
NASA GSFC, Global Modeling and Assimilation Office
Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
Phone: 301-614-6712 Fax: 301-614-6246
http://science.gsfc.nasa.gov/sed/bio/matthew.thompson
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are recorded in
the Unidata inquiry tracking system and made publicly available through the
web. Users who post to any of the lists we maintain are reminded to remove any
personal information that they do not want to be made public.
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
Any email message from EUMETSAT is sent in good faith but shall neither be
binding nor construed as constituting a commitment by EUMETSAT, except where
provided for in a written agreement or contract or if explicitly stated in the
email. Please note that any views or opinions presented in this email are
solely those of the sender and do not necessarily represent those of EUMETSAT.
This message and any attachments are intended for the sole use of the
addressee(s) and may contain confidential and privileged information. Any
unauthorised use, disclosure, dissemination or distribution (in whole or in
part) of its contents is not permitted. If you received this message in error,
please notify the sender and delete it from your system .
