Re: distribution musings - install three libraries to get netcdf-4?

• To: Elena Pourmal <epourmal@xxxxxxxxxxxxx>
• Subject: Re: distribution musings - install three libraries to get netcdf-4?
• From: Russ Rew <russ@xxxxxxxxxxxxxxxx>
• Date: Thu, 04 Dec 2003 12:59:42 -0700

Hi Elena,

> When the new HDF5 version comes out, who is responsible for testing
> netcdf-4? How do we coordinate the two libraries release schedule?
> 
> I am sure that there are some other maintenance/support issues. It would
> be nice if we can talk about them before the NetCDF users start to bombard
> our helpdesk with email messages :-)

Just to clarify, we don't expect to have a netCDF-4 available for
distribution to users, even in test form, until possibly late 2004 or
more likely early 2005.  

What Ed has now is a netCDF-4 prototype that is a proof of concept for
implementing the netCDF-3 interface on the current HDF5 library.  We
will make it available to developers at NCSA to help with testing and
further HDF5 development, but the netCDF to HDF5 mapping it uses to
represent the netCDF data model is not yet one we want to support.  We
expect it will take several iterations before we get this mapping
right, so data written using the current prototype mapping would
probably not be accessible through a later released netCDF-4 library.

There may be a few users who would want to try the current prototype,
understanding that it won't be supported, but we want to discourage
storing any important data through this interface until later, when we
have something ready for release.

So we agree that we should start thinking about and discussing some of
these release and maintenance issues now, but not focus too much
effort on finalizing them until we are closer to having something we
can support.

"Robert E. McGrath" <mcgrath@xxxxxxxxxxxxx> wrote:

> Obviously, it's never too soon to start thinking about how we want
> to do this, and we will need at least some sort of process for the
> early releases.

Yes, we'll be working out this process with our prototype releases.

--Russ


>From owner-netcdf-hdf@xxxxxxxxxxxxxxxx 15 2003 Dec -0700 03:48:58 
Message-ID: <ullpe8ig5.fsf@xxxxxxxxxx>
Date: 15 Dec 2003 03:48:58 -0700
From: ed@xxxxxxxxxxxxxxxx
To: netcdf-hdf@xxxxxxxxxxxxxxxx
Subject: HDF5 chunking questions...
Received: (from majordo@localhost)
        by unidata.ucar.edu (UCAR/Unidata) id hBFAhx8k002613
        for netcdf-hdf-out; Mon, 15 Dec 2003 03:43:59 -0700 (MST)
Received: from mailhost.idcomm.com (cranium1.idcomm.com [216.98.192.8])
        by unidata.ucar.edu (UCAR/Unidata) with ESMTP id hBFAhwp2002553
        for <netcdf-hdf@xxxxxxxxxxxxxxxx>; Mon, 15 Dec 2003 03:43:59 -0700 (MST)
Organization: UCAR/Unidata
Keywords: 200312151043.hBFAhwp2002553
Received: from MIKADO (tnt01-ppp-053.idcomm.com [216.98.194.53])
        by mailhost.idcomm.com (Postfix) with ESMTP id 34E41DFB5F3
        for <netcdf-hdf@xxxxxxxxxxxxxxxx>; Mon, 15 Dec 2003 03:43:56 -0700 (MST)
Lines: 12
User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Sender: owner-netcdf-hdf@xxxxxxxxxxxxxxxx
Precedence: bulk

Quincey et. al.,

Given an n-dimensional dataspace, with only one unlimited
(i.e. extendable) dimension, tell me how to select the chunk size for
each dimension to get a good read performance for large data files.

Would you care to suggest any smart algorithms to yeild better
performance for various situations?

Thanks,

Ed

• References:
  • Re: distribution musings - install three libraries to get netcdf-4?
    • From: Elena Pourmal