[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #VRU-236841]: Re: Known problem URL for large block size, silent file corruption

Hi Charlie,

> Sounds like you are close to a fix for this nasty bug.
> I'll test your fix on cisl bluefire and mirage, if you want.
> And I'll wait awhile until releasing 4.0.8.

Just to keep you appised of progress, I've checked in a fix to our svn trunk, 
consisting of a 20-line addition to the libsrc/posixio.c code.  The conditions 
for the bug appear to be pretty rare, but are more likely with larger disk 
block sizes.  Examples of the bug with small disk block sizes require 
relatively small files and involve:

  - writing data to a file in nofill mode
  - writing more than one disk-block beyond the end of the file, as might 
    happen in writing the last slice of a multidimensional variable before 
    writing other slices
  - crossing disk-block boundaries with the region to be written
  - having the in-memory buffer in a state in which the region to be written
    corresponds to the upper half of the buffer and recently written data in 
    the lower half of the buffer hasn't been flushed to disk yet.

The last condition makes it difficult to give users an easy way to determine
whether they have been a victim of this problem.  I'm still struggling with
a better description of the conditions under which it might occur, and I still
need to understand why we can duplicate the problem for 4K disk blocks if we
use the double-underbar function nc__create(), but not if we use the more
common nc_create().

When I have that mystery solved, I should be able to send out a netcdfgroup
posting, and maybe create an FAQ or blog entry about the bug with more 
information than people are likely to want to read in an email posting.

--Russ

would 

--Russ


Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: VRU-236841
Department: Support netCDF
Priority: Normal
Status: Closed