Thank you. That is very useful information.
I am sad to hear that the conscious default is for the library not to throw
an error when the file is truncated (but I understand),
but happy that I can use ucar.nc2.iosp.netcdf3.N3header.
disallowFileTruncation to change this behavior for my work.
On Wed, Nov 2, 2016 at 7:16 AM, Christian Ward-Garrison <cwardgar@xxxxxxxx>
wrote:
> Hi Bob,
>
> Sorry for the delay in responding. I would have expected an exception as
> well, but apparently this is working as intended. Specifically, we're
> trying to match the behavior of the netcdf C library, which allows
> truncated files [1]. I verified this with ncdump, which printed all data
> values without error. For truncated values, ncdump printed '0', which is
> the default for floats in C. NetCDF-Java, on the other hand, used whatever
> values were left in the RandomAccessFile buffer after the last successful
> read for truncated values. Maybe that's a problem.
>
> If you like, it's possible to disallow truncated files by setting
> ucar.nc2.iosp.netcdf3.N3header.disallowFileTruncation to true, in which
> case you'll get the exception you're expecting.
>
> Cheers,
> Christian
>
> [1] http://www.unidata.ucar.edu/software/netcdf/netcdf-4/
> newdocs/netcdf-c/Fill-Values.html#Fill-Values
>
> On Tue, Oct 18, 2016 at 11:56 AM, Bob Simons - NOAA Federal <
> bob.simons@xxxxxxxx> wrote:
>
>> In our system, and I suspect elsewhere, sometimes .nc files don't get
>> completely copied from one place to another. Normally, this is caught and
>> dealt with (e.g., by the script doing the copying), but sometimes it isn't
>> (e.g., a computer is restarted while a script is running), leaving a
>> shortened file in place.
>>
>> I wanted to understand how netcdf-java (v4.6.6) reacted to these shorted
>> files. So I made the attached file by copying only the first 1/6th of a
>> valid file. Then I tried to read it with this code
>>
>> NetcdfFile ncFile = NetcdfFile.open("/git/erddapTe
>> st/nc/invalidShortened.nc");
>> try {
>> Variable var = ncFile.findVariable("MWcdom");
>> //size=[1,1,2321,4001]
>> Array array = var.read();
>> System.out.println("shape=" +
>> Arrays.toString(array.getShape()));
>>
>> array = var.read("0,0,0:2000:1000,0:4000:1000");
>> System.out.println("shape=" +
>> Arrays.toString(array.getShape()));
>>
>> //System.out.println(array.toString());
>> System.out.println("Shouldn't get here!");
>> ncFile.close();
>> } catch (Throwable t) {
>> try {
>> ncFile.close();
>> } catch (Throwable t2) {
>> }
>> System.out.println("caught: " + t.toString());
>> }
>>
>> Note that Variable.read() documentation says that the method throws an
>> IOException.
>>
>> To my surprise, the above code doesn't throw any exception.
>> netcdf-java says it can read the entire var and read the specified
>> section (which gets values from throughout the variable).
>>
>> Is that what you expected?
>> Am I doing something wrong in my test?
>> Doesn't netcdf-java check the validity of the file?
>> Why don't the read()s cause an IOException?
>>
>> Thank you for looking into this.
>>
>> --
>> Sincerely,
>>
>> Bob Simons
>> IT Specialist
>> Environmental Research Division
>> NOAA Southwest Fisheries Science Center
>> 99 Pacific St., Suite 255A (New!)
>> Monterey, CA 93940 (New!)
>> Phone: (831)333-9878 (New!)
>> Fax: (831)648-8440
>> Email: bob.simons@xxxxxxxx
>>
>> The contents of this message are mine personally and
>> do not necessarily reflect any position of the
>> Government or the National Oceanic and Atmospheric Administration.
>> <>< <>< <>< <>< <>< <>< <>< <>< <><
>>
>>
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web. Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> netcdf-java mailing list
>> netcdf-java@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
--
Sincerely,
Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
99 Pacific St., Suite 255A (New!)
Monterey, CA 93940 (New!)
Phone: (831)333-9878 (New!)
Fax: (831)648-8440
Email: bob.simons@xxxxxxxx
The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><
