Kris Nuttycombe wrote:
Hi, John,
I work for Ted Habermann over here at the National Geophysical Data
Center, and I'm in the process of doing some work with your NetCDF
libraries. I'd like to write a patch for your library that permits
NetCDF files to be read over an FTP connection, in addition to the
functionality you've provided for reading NetCDF remotely over HTTP. I
have most of what I need from the source code that you distributed,
but the only thing I'm lacking are javadocs for the HttpClient library
so that I know what I need to drop in for the RandomAccessFile
subclass I'm writing to emulate the functionality you use in
HTTPRandomAccessFile exactly.
Can you point me to either the HTTPClient source code or the Javadocs
directly? Also, I'm using the jakarta commons.net.ftp library in my
implementation, so my patch (if you choose to accept it) will add that
dependency if the user want's to use these methods.
Thanks,
Kris Nuttycombe
Hi Kris, that sounds great, we will be very excited to include your
improvements in our package.
The HttpClient package lives here:
http://jakarta.apache.org/commons/httpclient/
The overall idea is that one uses a replacement for RandomAccessFile, so
that whenever the system calls for a disk access, an HTTP access is done
instead. So all we need is to be able to randomly access bytes in a file
on an HTTP server, which we can through the GetRange HTTP command. So to
do the same for FTP, we just need to use the equivilent FTP call,
whatever that is. So probably you dont needt ot study the HttpClient
code very closely, just look at RandomAccessFile and see where it makes
disk calls.
Of course, we dont really want to make a network call for everything,
so RandomAccessFile, (and HTTPRandomAccessFile) tries to intelligently
buffer. I am intending on trying to optimize that a bit more, so if you
have an ideas, try them out. I think the main thing would be to try to
read the entire header all at once, so perhaps the header parser needs
to estimate the header size, and twiddle some buffer sizes in the
RandomAccessFile.
Im quite sure we dont need all the functionality of HttpClient, but i
havent had time to figure out how to replace it or reduce it (mostly
just to reduce the footprint).
let me know what else i can help you with, and how it goes.
thanks again,
John.
