Re: [netcdf-java] Write files with 64-bit long data types

  • To: Hanns Holger Rutz <rutz@xxxxxx>
  • Subject: Re: [netcdf-java] Write files with 64-bit long data types
  • From: Aaron Braeckel <braeckel@xxxxxxxx>
  • Date: Mon, 3 Jul 2017 12:41:06 -0600
It's been some time since we have worked on this, but my recollection is
that HDF 5 was moving rapidly at the time and we decided to implement HDF 5
and netCDF 4 files at a certain point in time rather than try to
incorporate everything that was happening in HDF 5.  In theory this means
that files written by Nujan should be readable by HDF 1.8.5 and later and
the C NetCDF API 4.1.1 or later.  Nujan does not currently support updating
files, only creating new ones.  One feature I know is unsupported is header
compression, which for small files can be more significant.

I can't speak to the API similarities as I haven't used both APIs.
However, you can start with NhFileWriter
<https://github.com/NCAR/nujan/blob/master/src/main/java/edu/ucar/ral/nujan/netcdf/NhFileWriter.java>
 .

I just fixed the POM dependencies so it is buildable by others now.  Fixes
and improvements welcome.

Aaron


On Fri, Jun 30, 2017 at 11:41 AM, Hanns Holger Rutz <rutz@xxxxxx> wrote:

> hello aaron,
>
> thanks for this project link, I will check it out. I guess the API is
> not identical, though? Can I use `ma2.Array` and things like that?
>
> best, .h.h.
>
>
> On 30/06/17 19:36, Aaron Braeckel wrote:
> > Hello,
> >
> > a pure-Java library exists for writing netCDF 4 and HDF
> > 5: https://github.com/NCAR/nujan.  We've not put much time into this
> > recently and the JNI/native alternative will have a lot more options and
> > features, but Nujan may be another option for you to consider.
> >
> > Aaron
> >
> > On Fri, Jun 30, 2017 at 11:29 AM, Hanns Holger Rutz <rutz@xxxxxx
> > <mailto:rutz@xxxxxx>> wrote:
> >
> >     I see -- and it's not possible to draw them in as standard Maven
> >     dependencies for the correct platforms? I think this works with other
> >     Java-C-bridging libraries such as OpenCV, for example:
> >
> >     http://search.maven.org/#search%7Cga%7C1%7Copencv
> >     <http://search.maven.org/#search%7Cga%7C1%7Copencv>
> >
> >     you have there linux-armhf.jar, linux-x86.jar,  linux-x86_64.jar,
> >     macosx-x86_64.jar, windows-x86.jar etc.
> >
> >     would it be possible to do that with NetCDF as well?
> >
> >     best, .h.h.
> >
> >
> >     On 30/06/17 19:10, Sean Arms wrote:
> >     > Greetings!
> >     >
> >     > Currently, the only way to write netCDF-4 files is through the C
> >     interface:
> >     >
> >     >
> >     https://www.unidata.ucar.edu/software/thredds/current/
> netcdf-java/reference/netcdf4Clibrary.html
> >     <https://www.unidata.ucar.edu/software/thredds/current/
> netcdf-java/reference/netcdf4Clibrary.html>
> >     >
> >     > As it stands, there isn't a pure implementation of HDF5 outside of
> the
> >     > C library, so pretty much everyone who writes netCDF needs to go
> >     > through the C library.
> >     >
> >     > Sean
> >     >
> >     >
> >     > On Fri, Jun 30, 2017 at 9:55 AM, Hanns Holger Rutz <rutz@xxxxxx
> >     <mailto:rutz@xxxxxx>> wrote:
> >     >> p.s. I use
> >     >>
> >     >>     "edu.ucar" % "netcdf4" % "4.6.10"
> >     >>
> >     >> as library. perhaps I need to change this or use a different one?
> >     >>
> >     >>
> >     >> On 30/06/17 17:51, Hanns Holger Rutz wrote:
> >     >>> hi,
> >     >>>
> >     >>> so far I have used this
> >     >>>
> >     >>>     nc2.NetcdfFileWriter.createNew(Version.netcdf3, location,
> null)
> >     >>>
> >     >>> this creates files with this type of header:
> >     >>>
> >     >>> 00000000  43 44 46 01 00 00 00 00  00 00 00 0a 00 00 00 04
> >     >>> |CDF.............|
> >     >>> 00000010  00 00 00 04 54 69 6d 65  00 00 00 b4 00 00 00 09
> >     >>> |....Time........|
> >     >>> 00000020  4c 6f 6e 67 69 74 75 64  65 00 00 00 00 00 00 0c
> >     >>> |Longitude.......|
> >     >>> 00000030  00 00 00 08 4c 61 74 69  74 75 64 65 00 00 00 24
> >     >>> |....Latitude...$|
> >     >>> 00000040
> >     >>>
> >     >>> now I have a problem where I have to include 64-bit long values,
> and
> >     >>> this seems to be unsupported: "long not support in netcdf-3"
> >     >>>
> >     >>> So I tried chaing  this to
> >     >>>
> >     >>>     nc2.NetcdfFileWriter.createNew(Version.netcdf4, location,
> null)
> >     >>>
> >     >>> now this gives me error that it wants to use C (???) libraries:
> >     >>>
> >     >>>     Nc4Iosp: NetCDF-4 C library not present
> >     (jna_path='/usr/local/lib/',
> >     >>> libname='netcdf').
> >     >>>
> >     >>> What should I do? I need a solution that is cross-platform with
> >     >>> libraries pulled from a maven repository, no matter whether user
> >     runs
> >     >>> Linux, Windows, Mac.
> >     >>>
> >     >>> Can I get a pure Java solution that supports 64-bit long? How?
> >     >>>
> >     >>> thanks, .h.h.
> >     >>>
> >     >>>
> >     >>>
> >     >>> _______________________________________________
> >     >>> NOTE: All exchanges posted to Unidata maintained email lists are
> >     >>> recorded in the Unidata inquiry tracking system and made publicly
> >     >>> available through the web.  Users who post to any of the lists we
> >     >>> maintain are reminded to remove any personal information that
> they
> >     >>> do not want to be made public.
> >     >>>
> >     >>>
> >     >>> netcdf-java mailing list
> >     >>> netcdf-java@xxxxxxxxxxxxxxxx <mailto:netcdf-java@unidata.
> ucar.edu>
> >     >>> For list information or to unsubscribe, visit:
> >     http://www.unidata.ucar.edu/mailing_lists/
> >     <http://www.unidata.ucar.edu/mailing_lists/>
> >     >>>
> >     >>
> >     >>
> >     >> _______________________________________________
> >     >> NOTE: All exchanges posted to Unidata maintained email lists are
> >     >> recorded in the Unidata inquiry tracking system and made publicly
> >     >> available through the web.  Users who post to any of the lists we
> >     >> maintain are reminded to remove any personal information that they
> >     >> do not want to be made public.
> >     >>
> >     >>
> >     >> netcdf-java mailing list
> >     >> netcdf-java@xxxxxxxxxxxxxxxx <mailto:netcdf-java@xxxxxxxxxxxxxxxx
> >
> >     >> For list information or to unsubscribe, visit:
> >     http://www.unidata.ucar.edu/mailing_lists/
> >     <http://www.unidata.ucar.edu/mailing_lists/>
> >
> >     _______________________________________________
> >     NOTE: All exchanges posted to Unidata maintained email lists are
> >     recorded in the Unidata inquiry tracking system and made publicly
> >     available through the web.  Users who post to any of the lists we
> >     maintain are reminded to remove any personal information that they
> >     do not want to be made public.
> >
> >
> >     netcdf-java mailing list
> >     netcdf-java@xxxxxxxxxxxxxxxx <mailto:netcdf-java@xxxxxxxxxxxxxxxx>
> >     For list information or to unsubscribe, visit:
> >     http://www.unidata.ucar.edu/mailing_lists/
> >     <http://www.unidata.ucar.edu/mailing_lists/>
> >
> >
>
  • 2017 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: