Re: [netcdf-java] How to create a structure in netcdf4

  • To: Christian Ward-Garrison <cwardgar@xxxxxxxx>
  • Subject: Re: [netcdf-java] How to create a structure in netcdf4
  • From: José Évora <jose.evora@xxxxxxxx>
  • Date: Fri, 11 Dec 2015 10:06:43 +0000
Hi Christian!

Thank you very much. It worked perfectly and now the netcdf file is much
more clean. I just did one change which is calling the dimension
"changelog" and the structured variable "changelog" too:

structure = (Structure) ncFile.addVariable(null, "CHANGELOG",
STRUCTURE, singletonList(ncFile.addDimension(null, "CHANGELOG", 1,
true, false, false)));

As result I have next file structure:

[image: Imágenes integradas 2]

Is there anyway to have CHANGE-LOG-ENTRY instead of CHANGELOG_t in the
yellow marked entry?

Thanks in advance, you're helping me alot


José Évora Gómez
Instituto Universitario SIANI
Universidad de Las Palmas de GC
@j_evora

2015-12-08 23:57 GMT+00:00 Christian Ward-Garrison <cwardgar@xxxxxxxx>:

> Hi José,
>
> Once again, sorry for the delay. I've modified my last example to
> incorporate the changes you requested [1]. A few things to note:
>
> 1. The "Other" dimension is now shared instead of private.
> NetcdfFileWriter.addDimension creates a dimension and adds it to a Group.
> When Dimensions are contained by Groups, they should *always* be shared.
> In fact, NetcdfFileWriter.addDimension shouldn't even have the isShared
> parameter, as it is confusing and error-prone. In v5.0.0, it won't [2].
> Thanks for bringing this to my attention!
>
> 2. In the first example, the Date1 and Date2 variables use *anonymous
> dimensions*. Anonymous dimensions are both private (i.e. contained by a
> Variable, not a Group) and nameless. NetcdfFileWriter does not provide a
> direct way to create them, so you'll have to do a little more work, as I've
> shown.
>
> 3. In the first example, it looks like the "CHANGELOG=2" snippet may
> indicate a private, *named* dimension. If that's the case, create it the
> same way you did the anonymous dimensions but just – you know – give it a
> name.
>
> Cheers,
> Christian
>
> [1] https://gist.github.com/cwardgar/35427c2e245ec0157416
> [2] https://github.com/Unidata/thredds/pull/326
>
> On Tue, Nov 24, 2015 at 1:23 AM, José Évora <jose.evora@xxxxxxxx> wrote:
>
>> Hi Christian,
>>
>> Thank you very much, it worked perfectly!. However, I'm trying to
>> replicate the output as expected for a netcdf reader of a specific kind of
>> data. When I read a netcdf that is accepted by the specific reader using
>> the netcdf java library, this is what I obtain (I've have adapted the
>> content to our toy case, but it can be extrapolated to the real case):
>>
>> netcdf res/in original.nc {
>>>   dimensions:
>>
>>     City = 1;
>>
>>     Other = UNLIMITED;   // (2 currently)
>>>     ...
>>>   variables:
>>
>>     String City(City=1);
>>
>>
>>
>>     Structure {
>>>       char Date1(30);
>>>       char Date2(53);
>>>     } CHANGELOG(CHANGELOG=2);
>>>       :_ChunkSizes = 1; // int
>>> }
>>
>>
>> With the code you provided me, I obtain:
>>
>>  netcdf res/mockSimulation2.nc {
>>>   dimensions:
>>>     City = 1;
>>>     Other-Entry_Dim0 = 1;
>>>     Other = 1;
>>>     strlen = 30;
>>>   variables:
>>>     String City(City=1);
>>>     Structure {
>>>       char Date1(30);
>>>       char Date2(30);
>>>     } Other-Entry(Other-Entry_Dim0=1);
>>>
>>> }
>>
>>
>> I only did one change to what you sent me at line 26:
>>
>> Structure otherVar = (Structure) ncFile.addVariable(root, "Other-Entry",
>>> STRUCTURE, singletonList(other));
>>
>>
>> I don't know how I can:
>>
>> 1) Avoid having a dimension strlen
>> 2) Avoid having Other-Entry_Dim0
>>
>> Thanks in advance!
>>
>> José Évora Gómez
>> Instituto Universitario SIANI
>> Universidad de Las Palmas de GC
>> @j_evora
>>
>> 2015-11-23 6:59 GMT+00:00 Christian Ward-Garrison <cwardgar@xxxxxxxx>:
>>
>>> Hi José,
>>>
>>> Sorry for the late response—hopefully this is still useful to you.
>>>
>>> Writing structures to NetCDF-4 is a recent addition to our library and
>>> hasn't been tested as comprehensively as we'd like, especially since most
>>> of our users that are writing NetCDF-4 are only utilizing the classic data
>>> model (i.e. no structures). Your example uncovered multiple bugs [1][2][3],
>>> so thank you for that!
>>>
>>> That said, it is still possible to make your example work, although the
>>> admittedly limited documentation makes it tricky to get right. I've
>>> corrected your NetCdfCreatorLite2 class and posted it here [4]. Notice that
>>> the "Other" dimension is no longer unlimited and that I'm recalculating the
>>> structure's size after adding members to it.
>>>
>>> Also, for future reference, the latest versions of our library artifacts
>>> are not available on Maven Central. Instead, grab them from our
>>> organization's repository, as described here [5].
>>>
>>> Cheers,
>>> Christian
>>>
>>> [1] https://github.com/Unidata/thredds/issues/296
>>> [2] https://github.com/Unidata/thredds/issues/298
>>> [3] https://github.com/Unidata/thredds/issues/299
>>> [4] https://gist.github.com/cwardgar/7aa866886468e97c8eea
>>> [5]
>>> https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/reference/BuildDependencies.html
>>>
>>> On Thu, Nov 5, 2015 at 8:01 AM, José Évora <jose.evora@xxxxxxxx> wrote:
>>>
>>>> The dependency is being taken from Maven central. However, I have tried
>>>> this jar instead of maven:
>>>>
>>>> ftp://ftp.unidata.ucar.edu/pub/netcdf-java/v4.6/netcdfAll-4.6.jar
>>>>
>>>> And I get exactly the same problems in both java files.
>>>>
>>>> José Évora Gómez
>>>> Instituto Universitario SIANI
>>>> Universidad de Las Palmas de GC
>>>> @j_evora
>>>>
>>>> 2015-11-05 14:09 GMT+00:00 John Caron <jcaron1129@xxxxxxxxx>:
>>>>
>>>>> what repo is
>>>>>
>>>>> <dependency>
>>>>> <groupId>edu.ucar</groupId>
>>>>> <artifactId>netcdf4</artifactId>
>>>>> <version>LATEST</version>
>>>>> </dependency>
>>>>>
>>>>> coming from?
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Nov 5, 2015 at 12:13 AM, José Évora <jose.evora@xxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>> When maven resolves the code quoted, this is the library that
>>>>>> downloads:
>>>>>>
>>>>>> Maven: edu.ucar:netcdf4:4.5.5
>>>>>>
>>>>>>
>>>>>> Version seems to be 4.5.5 for the netcdf java library. For the netcdf
>>>>>> C libraries, I downloaded and installed: netCDF 4.3.3.1. Specifically, 
>>>>>> this
>>>>>> file:
>>>>>>
>>>>>>
>>>>>> http://www.unidata.ucar.edu/downloads/netcdf/ftp/netCDF4.3.3.1-NC4-DAP-64.exe
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> José Évora Gómez
>>>>>> Instituto Universitario SIANI
>>>>>> Universidad de Las Palmas de GC
>>>>>> @j_evora
>>>>>>
>>>>>> 2015-11-04 23:33 GMT+00:00 John Caron <jcaron1129@xxxxxxxxx>:
>>>>>>
>>>>>>> Hi Jose, what version of netcdf-java and netcdf C libraries are you
>>>>>>> using? John
>>>>>>>
>>>>>>> On Wed, Nov 4, 2015 at 12:54 AM, José Évora <jose.evora@xxxxxxxx>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi everyone,
>>>>>>>>
>>>>>>>> I'm trying to create a structure (also know compounded type). In
>>>>>>>> this email I attach two different ways I've tried to do it and their
>>>>>>>> respective error message. This is the dependency I'm using
>>>>>>>>
>>>>>>>> <dependency>
>>>>>>>>> <groupId>edu.ucar</groupId>
>>>>>>>>> <artifactId>netcdf4</artifactId>
>>>>>>>>> <version>LATEST</version>
>>>>>>>>> </dependency>
>>>>>>>>
>>>>>>>>
>>>>>>>> *1) NetCdfCreatorLite.java*
>>>>>>>>
>>>>>>>> The execution throws a NullPointerException at
>>>>>>>>
>>>>>>>> Structure otherVar = ncFile.addStructure(root,
>>>>>>>>> root.getParentStructure(), "Other", singletonList(other));
>>>>>>>>
>>>>>>>>
>>>>>>>> Exception in thread "main" java.lang.NullPointerException
>>>>>>>>> at
>>>>>>>>> ucar.nc2.NetcdfFileWriter.addStructure(NetcdfFileWriter.java:584)
>>>>>>>>> at org.mock.NetCdfCreatorLite.main(NetCdfCreatorLite.java:27)
>>>>>>>>
>>>>>>>>
>>>>>>>> *2) NetCdfCreatorLite2.java*
>>>>>>>>
>>>>>>>> According to the java doc, the method addstructure shouldn't be
>>>>>>>> used. Then, in this second version, I uses a addVariable method to 
>>>>>>>> create
>>>>>>>> the structure:
>>>>>>>>
>>>>>>>> Structure otherVar = (Structure) ncFile.addVariable(root, "Other",
>>>>>>>>> STRUCTURE, singletonList(other));
>>>>>>>>
>>>>>>>>
>>>>>>>> The exeception given this time is:
>>>>>>>>
>>>>>>>> Exception in thread "main" java.io.IOException: nc_put_vars: -49:
>>>>>>>>> NetCDF: Variable not found grpid=65536 objid=1
>>>>>>>>> at ucar.nc2.jni.netcdf.Nc4Iosp.writeCompoundData(Nc4Iosp.java:2895)
>>>>>>>>> at ucar.nc2.jni.netcdf.Nc4Iosp.writeData(Nc4Iosp.java:2839)
>>>>>>>>> at ucar.nc2.jni.netcdf.Nc4Iosp.writeData(Nc4Iosp.java:2721)
>>>>>>>>> at ucar.nc2.NetcdfFileWriter.write(NetcdfFileWriter.java:934)
>>>>>>>>> at ucar.nc2.NetcdfFileWriter.write(NetcdfFileWriter.java:918)
>>>>>>>>> at org.mock.NetCdfCreatorLite2.main(NetCdfCreatorLite2.java:46)
>>>>>>>>
>>>>>>>>
>>>>>>>> At line:
>>>>>>>>
>>>>>>>> ncFile.write(otherVar, otherStructure);
>>>>>>>>
>>>>>>>>
>>>>>>>> If I comment the creation of structures, the application executes
>>>>>>>> perfectly. Even more, by only adding the addVariable of a structure 
>>>>>>>> line, I
>>>>>>>> get this exception, even though I only write the cityVar at line 50.
>>>>>>>>
>>>>>>>> What is wrong? Can you please provide me with an example of use of
>>>>>>>> the structures?.
>>>>>>>>
>>>>>>>> Best regards, thank you very much,
>>>>>>>>
>>>>>>>> José Évora Gómez
>>>>>>>> Instituto Universitario SIANI
>>>>>>>> Universidad de Las Palmas de GC
>>>>>>>> @j_evora
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> netcdf-java mailing list
>>>>>>>> netcdf-java@xxxxxxxxxxxxxxxx
>>>>>>>> For list information or to unsubscribe, visit:
>>>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> netcdf-java mailing list
>>>> netcdf-java@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe, visit:
>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>
>>>
>>>
>>
>

PNG image

  • 2015 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: