OK perhaps I may have figured it out...
With ZFP you can invoke the accuracy parameter option in two ways, one that
is intuitive (floating point value) and one that is not. Of course I
require the non-intuitive one:
*H5Pset_zfp_accuracy_cdata(acc, cd_nelmts, cd_values);*
*cd_values index:*
* 0 1 2 3 4 5*
accuracy: 3 unused *accA accB* unused unused
*accA* and *aacB* are "high/low 32-bit words of a double"
Ah hah! So I think I just need to mangle my 64 bit floating point accuracy
value and chop it into high and low parts (cast as unsigned int or
something), and pass them as options 2 and 3 into the 6 element cd_values
array?
I believe then those would be the arguments passed to
nc_def_var_filter(nc_file_id,nc_variable,H5Z_FILTER_ZFP,cd_nelmts,
*cd_values*);
That is what I will try...
Leigh
On Tue, Mar 23, 2021 at 12:52 PM Leigh Orf <leigh.orf@xxxxxxxxx> wrote:
> Joe,
>
> Thanks, when I am completely in HDF5 land, I can do the ZFP stuff!
>
> I don't know how to twiddle the ZFP settings in netcdf API land... and I
> don't know how to make netCDF4 files in HDF5 API land...
>
> On Tue, Mar 23, 2021 at 12:47 PM H. Joe Lee <hyoklee@xxxxxxxxxxxx> wrote:
>
>> Please check this out:
>>
>> https://github.com/HDFGroup/hdf5_plugins/
>>
>> - Joe
>> ---
>> Deploy XaaS quickly using HDF in Distributed Cloud.
>>
>>
>>
>>
>> On Tue, Mar 23, 2021 at 12:03 PM Dennis Heimbigner <dmh@xxxxxxxx> wrote:
>>
>>> If you have the HDF5 ZFP plugin available, then you can certainly
>>> use it in netcdf.
>>> Some documentation on netcdf filter support is here:
>>> >https://github.com/Unidata/netcdf/blob/master/NUG/filters.md
>>>
>>> =Dennis Heimbigner
>>> Unidata
>>>
>>>
>>> On 3/23/2021 10:52 AM, Leigh Orf wrote:
>>> > Hello,
>>> >
>>> > I'm a long time user of HDF and netcdf. I create lots of data in my
>>> > research (severe weather simulation) and ZFP compression in HDF5 files
>>> > has been great. I'd like to be able to create netcdf4 files
>>> > (preferably using the netcdf API) with 3D fields compressed using the
>>> > lossy ZFP compression algorithm. I know that netcdf4 is HDF5 so it's
>>> > certainly possible.
>>> >
>>> > I'm wondering whether anyone could provide any pointers. I see 3
>>> > options really.
>>> >
>>> > 1. Create ZFP compressed netcdf4 files using the normal netCDF-C API
>>> > (my code is mostly C)
>>> > 2. Create ZFP compressed netcdf4 files using the HDF5 API (including
>>> > all the HDF5 special sauce that makes netcdf4 netcdf4)
>>> > 3. Use a version of nccopy that understands ZFP
>>> >
>>> > Are any of these options possible right now? If not where are we on
>>> > efforts to include ZFP into netCDF? I did grab the latest code on
>>> > github and saw references to ZFP but no filter code.
>>> >
>>> > Leigh Orf
>>> > UW-Madison
>>> >
>>> > _______________________________________________
>>> > NOTE: All exchanges posted to Unidata maintained email lists are
>>> > recorded in the Unidata inquiry tracking system and made publicly
>>> > available through the web. Users who post to any of the lists we
>>> > maintain are reminded to remove any personal information that they
>>> > do not want to be made public.
>>> >
>>> >
>>> > netcdfgroup mailing list
>>> > netcdfgroup@xxxxxxxxxxxxxxxx
>>> > For list information or to unsubscribe, visit:
>>> https://www.unidata.ucar.edu/mailing_lists/
>>>
>>> _______________________________________________
>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>> recorded in the Unidata inquiry tracking system and made publicly
>>> available through the web. Users who post to any of the lists we
>>> maintain are reminded to remove any personal information that they
>>> do not want to be made public.
>>>
>>>
>>> netcdfgroup mailing list
>>> netcdfgroup@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe, visit:
>>> https://www.unidata.ucar.edu/mailing_lists/
>>>
>>
