Russ Rew of Unidata suggested that using nccopy would be faster than
ncdump and ncgen. I changed my script accordingly:
http://www.esrl.noaa.gov/psd/people/dave.allured/data/netcdf/scripts/padsize
It is now faster by a factor of 10. Thanks, Russ!
--Dave
On Wed, Jul 11, 2012 at 4:43 PM, Dave Allured <dave.allured@xxxxxxxx> wrote:
> Phil,
>
> I have been using a kludgy method to manually check the extra padding,
> because there is no direct query in the current netcdf library. The
> method is to regenerate the whole file with ncdump and ncgen, and then
> simply see how much the file size is reduced. If this is of any
> interest, here is a copy of my current shell script:
>
> http://www.esrl.noaa.gov/psd/people/dave.allured/data/netcdf/scripts/padsize
>
> --Dave
>
> On Wed, Jul 11, 2012 at 3:45 AM, Bentley, Philip
> <philip.bentley@xxxxxxxxxxxxxxxx> wrote:
>> Hi folks,
>>
>> My current understanding is that if one increases the length of a netCDF
>> attribute then this will trigger a costly rewrite operation on the whole
>> file since the header section is of fixed size. At least for netCDF-3 files,
>> I think.
>>
>> But does this file rewrite penalty apply if the *net* change in header
>> length arising from a series of attribute changes is zero or less? Put
>> another way, is the basic constraint the overall length of the header
>> section, or the length of the individual attribute that one is planning to
>> change ?
>>
>> Some simple tests using ncatted suggest that it's the former - the length of
>> the header section - though it would be useful to have this confirmed or
>> refuted.
>>
>> For example, I make changes to a series of text attributes such that they
>> are now all shorter in length. This presumably frees up space in the header
>> section. I then increase the length of some other attribute (perhaps a
>> vector of ints). But that increase in length is accommodated by the space
>> freed up earlier. So in this case a file rewrite/copy operation is not
>> triggered, right?
>>
>> Lastly, I notice that it's possible to add some padding to the header
>> section using the h_minfree parameter to the nc__enddef() function. Is it
>> possible therefore to query the current amount of free space in the header
>> section? I couldn't immediately see a function for doing that.
>>
>> TIA,
>>
>> Phil
>>
>>
>> _______________________________________________
>> netcdfgroup mailing list
>> netcdfgroup@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
