Re: [netcdfgroup] What triggers a whole-file copy operation when modifying attributes?

On Wed, Jul 11, 2012 at 10:45:30AM +0100, Bentley, Philip wrote:
> My current understanding is that if one increases the length of a netCDF
> attribute then this will trigger a costly rewrite operation on the whole
> file since the header section is of fixed size. At least for netCDF-3
> files, I think.
> 
> But does this file rewrite penalty apply if the *net* change in header
> length arising from a series of attribute changes is zero or less?  Put
> another way, is the basic constraint the overall length of the header
> section, or the length of the individual attribute that one is planning
> to change ?
> 
> Some simple tests using ncatted suggest that it's the former - the
> length of the header section - though it would be useful to have this
> confirmed or refuted.

Your tests are accurate.  when exiting define mode, if the header is
"too big" it will trigger a rewrite.

> For example, I make changes to a series of text attributes such that
> they are now all shorter in length. This presumably frees up space in
> the header section. I then increase the length of some other attribute
> (perhaps a vector of ints). But that increase in length is accommodated
> by the space freed up earlier. So in this case a file rewrite/copy
> operation is not triggered, right?
> 
> Lastly, I notice that it's possible to add some padding to the header
> section using the h_minfree parameter to the nc__enddef() function.  Is
> it possible therefore to query the current amount of free space in the
> header section? I couldn't immediately see a function for doing that.

I don't know of one either.  You can poke around the internal data
structures if you are comfortable piercing the abstraction layer, but
that's probably not a great idea :>

==rob

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA



  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: