Re: [netcdf-java] How to create a structure in netcdf4

  • To: Christian Ward-Garrison <cwardgar@xxxxxxxx>
  • Subject: Re: [netcdf-java] How to create a structure in netcdf4
  • From: José Évora <jose.evora@xxxxxxxx>
  • Date: Tue, 24 Nov 2015 08:23:32 +0000
Hi Christian,

Thank you very much, it worked perfectly!. However, I'm trying to replicate
the output as expected for a netcdf reader of a specific kind of data. When
I read a netcdf that is accepted by the specific reader using the netcdf
java library, this is what I obtain (I've have adapted the content to our
toy case, but it can be extrapolated to the real case):

netcdf res/in original.nc {
>   dimensions:

    City = 1;

    Other = UNLIMITED;   // (2 currently)
>     ...
>   variables:

    String City(City=1);



    Structure {
>       char Date1(30);
>       char Date2(53);
>     } CHANGELOG(CHANGELOG=2);
>       :_ChunkSizes = 1; // int
> }


With the code you provided me, I obtain:

 netcdf res/mockSimulation2.nc {
>   dimensions:
>     City = 1;
>     Other-Entry_Dim0 = 1;
>     Other = 1;
>     strlen = 30;
>   variables:
>     String City(City=1);
>     Structure {
>       char Date1(30);
>       char Date2(30);
>     } Other-Entry(Other-Entry_Dim0=1);
>
> }


I only did one change to what you sent me at line 26:

Structure otherVar = (Structure) ncFile.addVariable(root, "Other-Entry",
> STRUCTURE, singletonList(other));


I don't know how I can:

1) Avoid having a dimension strlen
2) Avoid having Other-Entry_Dim0

Thanks in advance!

José Évora Gómez
Instituto Universitario SIANI
Universidad de Las Palmas de GC
@j_evora

2015-11-23 6:59 GMT+00:00 Christian Ward-Garrison <cwardgar@xxxxxxxx>:

> Hi José,
>
> Sorry for the late response—hopefully this is still useful to you .
>
> Writing structures to NetCDF-4 is a recent addition to our library and
> hasn't been tested as comprehensively as we'd like, especially since most
> of our users that are writing NetCDF-4 are only utilizing the classic data
> model (i.e. no structures). Your example uncovered multiple bugs [1][2][3],
> so thank you for that!
>
> That said, it is still possible to make your example work, although the
> admittedly limited documentation makes it tricky to get right. I've
> corrected your NetCdfCreatorLite2 class and posted it here [4]. Notice that
> the "Other" dimension is no longer unlimited and that I'm recalculating the
> structure's size after adding members to it.
>
> Also, for future reference, the latest versions of our library artifacts
> are not available on Maven Central. Instead, grab them from our
> organization's repository, as described here [5].
>
> Cheers,
> Christian
>
> [1] https://github.com/Unidata/thredds/issues/296
> [2] https://github.com/Unidata/thredds/issues/298
> [3] https://github.com/Unidata/thredds/issues/299
> [4] https://gist.github.com/cwardgar/7aa866886468e97c8eea
> [5]
> https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/reference/BuildDependencies.html
>
> On Thu, Nov 5, 2015 at 8:01 AM, José Évora <jose.evora@xxxxxxxx> wrote:
>
>> The dependency is being taken from Maven central. However, I have tried
>> this jar instead of maven:
>>
>> ftp://ftp.unidata.ucar.edu/pub/netcdf-java/v4.6/netcdfAll-4.6.jar
>>
>> And I get exactly the same problems in both java files.
>>
>> José Évora Gómez
>> Instituto Universitario SIANI
>> Universidad de Las Palmas de GC
>> @j_evora
>>
>> 2015-11-05 14:09 GMT+00:00 John Caron <jcaron1129@xxxxxxxxx>:
>>
>>> what repo is
>>>
>>> <dependency>
>>> <groupId>edu.ucar</groupId>
>>> <artifactId>netcdf4</artifactId>
>>> <version>LATEST</version>
>>> </dependency>
>>>
>>> coming from?
>>>
>>>
>>>
>>> On Thu, Nov 5, 2015 at 12:13 AM, José Évora <jose.evora@xxxxxxxx> wrote:
>>>
>>>> When maven resolves the code quoted, this is the library that
>>>> downloads:
>>>>
>>>> Maven: edu.ucar:netcdf4:4.5.5
>>>>
>>>>
>>>> Version seems to be 4.5.5 for the netcdf java library. For the netcdf C
>>>> libraries, I downloaded and installed: netCDF 4.3.3.1. Specifically, this
>>>> file:
>>>>
>>>>
>>>> http://www.unidata.ucar.edu/downloads/netcdf/ftp/netCDF4.3.3.1-NC4-DAP-64.exe
>>>>
>>>> Thanks,
>>>>
>>>> José Évora Gómez
>>>> Instituto Universitario SIANI
>>>> Universidad de Las Palmas de GC
>>>> @j_evora
>>>>
>>>> 2015-11-04 23:33 GMT+00:00 John Caron <jcaron1129@xxxxxxxxx>:
>>>>
>>>>> Hi Jose, what version of netcdf-java and netcdf C libraries are you
>>>>> using? John
>>>>>
>>>>> On Wed, Nov 4, 2015 at 12:54 AM, José Évora <jose.evora@xxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> I'm trying to create a structure (also know compounded type). In this
>>>>>> email I attach two different ways I've tried to do it and their 
>>>>>> respective
>>>>>> error message. This is the dependency I'm using
>>>>>>
>>>>>> <dependency>
>>>>>>> <groupId>edu.ucar</groupId>
>>>>>>> <artifactId>netcdf4</artifactId>
>>>>>>> <version>LATEST</version>
>>>>>>> </dependency>
>>>>>>
>>>>>>
>>>>>> *1) NetCdfCreatorLite.java*
>>>>>>
>>>>>> The execution throws a NullPointerException at
>>>>>>
>>>>>> Structure otherVar = ncFile.addStructure(root,
>>>>>>> root.getParentStructure(), "Other", singletonList(other));
>>>>>>
>>>>>>
>>>>>> Exception in thread "main" java.lang.NullPointerException
>>>>>>> at ucar.nc2.NetcdfFileWriter.addStructure(NetcdfFileWriter.java:584)
>>>>>>> at org.mock.NetCdfCreatorLite.main(NetCdfCreatorLite.java:27)
>>>>>>
>>>>>>
>>>>>> *2) NetCdfCreatorLite2.java*
>>>>>>
>>>>>> According to the java doc, the method addstructure shouldn't be used .
>>>>>> Then, in this second version, I uses a addVariable method to create the
>>>>>> structure:
>>>>>>
>>>>>> Structure otherVar = (Structure) ncFile.addVariable(root, "Other",
>>>>>>> STRUCTURE, singletonList(other));
>>>>>>
>>>>>>
>>>>>> The exeception given this time is:
>>>>>>
>>>>>> Exception in thread "main" java.io.IOException: nc_put_vars: -49:
>>>>>>> NetCDF: Variable not found grpid=65536 objid=1
>>>>>>> at ucar.nc2.jni.netcdf.Nc4Iosp.writeCompoundData(Nc4Iosp.java:2895)
>>>>>>> at ucar.nc2.jni.netcdf.Nc4Iosp.writeData(Nc4Iosp.java:2839)
>>>>>>> at ucar.nc2.jni.netcdf.Nc4Iosp.writeData(Nc4Iosp.java:2721)
>>>>>>> at ucar.nc2.NetcdfFileWriter.write(NetcdfFileWriter.java:934)
>>>>>>> at ucar.nc2.NetcdfFileWriter.write(NetcdfFileWriter.java:918)
>>>>>>> at org.mock.NetCdfCreatorLite2.main(NetCdfCreatorLite2.java:46)
>>>>>>
>>>>>>
>>>>>> At line:
>>>>>>
>>>>>> ncFile.write(otherVar, otherStructure);
>>>>>>
>>>>>>
>>>>>> If I comment the creation of structures, the application executes
>>>>>> perfectly. Even more, by only adding the addVariable of a structure 
>>>>>> line, I
>>>>>> get this exception, even though I only write the cityVar at line 50.
>>>>>>
>>>>>> What is wrong? Can you please provide me with an example of use of
>>>>>> the structures?.
>>>>>>
>>>>>> Best regards, thank you very much,
>>>>>>
>>>>>> José Évora Gómez
>>>>>> Instituto Universitario SIANI
>>>>>> Universidad de Las Palmas de GC
>>>>>> @j_evora
>>>>>>
>>>>>> _______________________________________________
>>>>>> netcdf-java mailing list
>>>>>> netcdf-java@xxxxxxxxxxxxxxxx
>>>>>> For list information or to unsubscribe, visit:
>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>> _______________________________________________
>> netcdf-java mailing list
>> netcdf-java@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
  • 2015 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: