Hi all,
The HDF5 library has a "file image" feature that allows a memory buffer
to be opened as a "file". The netCDF-4 library could be extended to support
this feature in HDF5.
--
Quincey Koziol, The HDF Group
Full Contact Info: http://bit.ly/QuinceyKoziol
> Hi Roy: It's not too big a deal, the netCDF 3 format is nicely
> orthogonal and easy to parse. The attached perl library is only around
> 350 lines long with adequate comments, but it gives you most basic
> functionality you need for reading.
>
> Regards,
>
> Doug
>
> On 11/20/14 10:36, Roy Mendelssohn - NOAA Federal wrote:
>> Hi Doug:
>>
>> Thanks for the comments. I would be interested on seeing the perl code, just
>> to see how complicated it is to do this (though I don?t really know perl).
>> Fro my purposes. since this is something we will be distributing, we want to
>> keep what we do to code that is ?native? to the program, in this case R.
>>
>> -Roy
>>
>> On Nov 20, 2014, at 8:48 AM, Doug Hunt <dhunt@xxxxxxxx> wrote:
>>
>>> Hi Roy: I had a similar problem earlier this year. I wanted to open a
>>> netCDF 3 file and read from it using the netCDF library when the file was
>>> presented to me in memory instead of as a disk file.
>>>
>>> I spent some time examining the netCDF library source and talking to
>>> UNIDATA folks about this and determined that the library was too hard to
>>> change. The interface depended upon passing in a file name and it would be
>>> too hard to modify it to add an 'ncopen' that took a pointer to memory
>>> instead .
>>>
>>> What I ended up doing was writing a reader from scratch in perl (since this
>>> application was in perl using the PDL::NetCDF interface) that reads the raw
>>> netCDF3 binary format.
>>>
>>> This has proved to be a good lightweight solution for us, but it does
>>> restrict us to netCDF 3.
>>>
>>> I'd be happy to share my perl netCDF 3 reader library if you are interested.
>>>
>>> Incidentally, as a result of this experience, I came away with an
>>> appreciation for the elegance and good design of the netCDF 3 binary format
>>> and a suspicion of the netCDF 4 format as being too complex and having too
>>> much library overhead.
>>>
>>> Regards,
>>>
>>> Doug
>>>
>>> On 11/19/14 18:41, Roy Mendelssohn - NOAA Federal wrote:
>>>> Hi All:
>>>>
>>>> Netcdf now has the option of creating or reading a file entirely in
>>>> memory, a nice option to have. I am wondering if there is any way to do
>>>> sort of the inverse. Some web libraries allow for a pure binary download
>>>> of a file into memory, so what I have sitting there is essentially a
>>>> binary image of the netcdf file. Is there any way to open that image
>>>> without writing to disk?
>>>>
>>>> As an example, the R httr library allows this. I can for example download
>>>> to memory a netcdf file from ERDDAP, and if I then do a binary save in
>>>> httr and then read it back in using the ncdf4 package, it all works great.
>>>> But it would be even better if I could skip the extra steps, and ?open?
>>>> the image in ncdf4 directly (ncdf4 btw can do whatever the netcdf
>>>> libraries can do, so it is a question of the base libraries).
>>>>
>>>> Thanks,
>>>>
>>>> -Roy
