Another recent possibility that supports parallel compression is Peter
Lindstrom's zfp
<http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6876024>.
On Thu, Jan 8, 2015 at 4:16 PM, Jim Edwards <jedwards@xxxxxxxx> wrote:
> pigz is only shared memory parallel and decompression is still serial so
> it's not really a solution.
>
> On Thu, Jan 8, 2015 at 4:10 PM, Orion Poplawski <orion@xxxxxxxxxxxxx>
> wrote:
>
>> On 01/07/2015 02:52 AM, Heiko Klein wrote:
>> > Hi Russ,
>> >
>> > thanks for your answer and the explanation from the hdfgroup.
>> >
>> > For our purposes, it is not the parallel IO, which is important, but the
>> > bottleneck is in many cases zlib. Are there other ways to parallelize
>> the
>> > 'zlib' portion of netcdf/hdf?
>> >
>> > Heiko
>>
>> Perhaps looking at the pigz code would be useful for this.
>>
>>
>> --
>> Orion Poplawski
>> Technical Manager 303-415-9701 x222
>> NWRA, Boulder/CoRA Office FAX: 303-415-9702
>> 3380 Mitchell Lane orion@xxxxxxxx
>> Boulder, CO 80301 http://www.nwra.com
>>
>> _______________________________________________
>> netcdfgroup mailing list
>> netcdfgroup@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
>
> --
> Jim Edwards
>
> CESM Software Engineer
> National Center for Atmospheric Research
> Boulder, CO
>
