[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #AGL-958284]: possible bugs with dim sizes exceeding 2^31

Hi Dave,

> I am attaching a C program that seems to duplicate what I am talking about. 
> It takes awhile to run.
> -dave

OK, thanks, I verified the bug exists in the current version 4.1.3,
but also in version 3.5.0 (from March 2001) and every release since,
so it's been undiscovered for more than 10 years.

I'm testing a fix, which seems to work OK.  You can follow the status
of the bug here, if you're interested:

  https://www.unidata.ucar.edu/jira/browse/NCF-101

I'll try to get it in a snapshot release soon if no problems turn up
in further testing.  I still need to create a small test that
demonstrates the bug using NOFILL mode, so the test will run quickly.
If you need the fix before then, it's pretty trivial, just change the
following line 522 in libsrc/putget.m4 from

                if(*coord > X_INT_MAX)
to
                if(*coord > X_UINT_MAX)

I haven't looked at the other ncdiminq bug you reported yet ...

--Russ

> On Jul 13, 2011, at 9:53 PM, Unidata netCDF Support wrote:
> 
> > Hi Dave,
> >
> >> I have found what I think are some bugs relating to large dimension sizes 
> >> for 64-bit offset NetCDF3 files.
> >>
> >> 1) The old ncdiminq function returns wrong values for dimensions larger 
> >> than 2^31.  This is a minor difficulty that was solved by switching to 
> >> nc_inq_dim.
> >>
> >> 2) Although the FAQ question 'Have all netCDF size limits been 
> >> eliminated?' says:
> >>
> >> The maximum number of records remains 2^32-1
> >>
> >> I am finding that, using nc_put_vara, or the old ncvarputg, records cannot 
> >> be added if  startp[0] is greater than 2^31.  I have a test PyNIO code 
> >> that creates a simple byte type single dimensioned variable. If the 
> >> dimension is fixed size I can write a variable that exceeds 2^31 
> >> (2,500,000,000 bytes is my arbitrary test size). Writing 1,000,000 records 
> >> per invocation of nc_put_vara, the script succeeds until *startp becomes 
> >> 2148000000.  So records above 2^31 are possible, but once *startp exceeds 
> >> 2^31 I get the error return "Index exceeds dimension bound", which does 
> >> not make sense, because it has been doing the same operation repeatedly 
> >> (and successfully) in a loop up to this point.
> >>
> >> This is using NetCDF 4.1.3, compiled with full NetCDF4 support.
> >
> > That sounds pretty serious.  Are the bugs you're noticing in programs that 
> > worked with previous versions of
> > netCDF, for example 4.1.2 or 4.1.1, or are you noticing the bugs in 4.1.3 
> > with new code that you haven't
> > tried with previous versions?
> >
> > I'll try to construct a test to demonstrate the problems from your 
> > description, unless you already have a small
> > test that you can provide so we can reproduce the bugs here.
> >
> > --Russ
> >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: AGL-958284
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> 
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: AGL-958284
Department: Support netCDF
Priority: Normal
Status: Closed