On Oct 11, 2012, at 11:31 AM, Brian Schlining <bschlining@xxxxxxxxx> wrote:
> Hi Bob,
>
> We're doing some work with ERDDAP and running into an issue using NetCDF-Java
> to access files served by ERDDAP. I think I understand the issue and know how
> to address it, so I'm passing the info onto you all so it can be addressed.
> So here goes:
>
> 1) ERDDAP allows one to download a NetCDF file by building a link appended
> with '.nc'. The link URL for the netcdf file would be something like
> http://beach.mbari.org:8180/erddap/griddap/erdRyanSST.nc. This works great
> for downloading the files. However, it does NOT work with the NetCDF-Java
> API; NetCDF-Java can normally read NetCDF files from arbitrary non-opendap
> http urls.
>
> 2) The reason it fails is because NetCDF-Java needs to know the size of the
> file being served. This requires that the HTTP response for a URL like
> http://beach.mbari.org:8180/erddap/griddap/erdRyanSST.nc to contain a
> 'Content-Length' field. ERDDAP is not sending that … here's a response header
> from ERDDAP (notice there's no 'Content-Length':
>
> HTTP/1.1 200 OK
> Server: Apache-Coyote/1.1
> Date: Thu, 11 Oct 2012 15:44:19 GMT
> Last-Modified: Thu, 11 Oct 2012 15:44:19 GMT
> xdods-server: dods/3.7
> erddap-server: 1.38
> Content-Disposition: attachment;filename=erdRyanSST_8571_f367_229e.nc
> Content-Encoding:
> Content-Type: application/x-download
> Transfer-Encoding: chunked
>
> 3) Since ERDDAP is running on Tomcat, the only way I know of to set the
> 'Content-Length' is to explicitly call 'response.setBufferSize()' in the
> servlet that returns the NetCDF file. Note that once the response size goes
> beyond the bufferSize, Tomcat will fallback to 'Transfer-Encoding: Chunked'
> (which we don't want). So make sure you're setting the buffer size to the
> correct value.
>
I don't think this is exactly correct. In the case that the content-length can
be determined, one should call setContentLength(...) on the ServletResponse
instance to set the HTTP Response Content-Length header (for Tomcat). The
transfer-encoding used to deliver the content should be abstracted by the HTTP
Client implementation one is using, you shouldn't have to worry about this.
Setting content-length is preferable but it's not always feasible in some
implementations where low-latency/high-throughput responses are desired with
large responses (i.e. avoid buffering entire response to to calculate
content-length before sending any bytes).
> Hope that helps!
>
> p.s. I cc'd this to the netcdf-java mailing list in case I got something
> wrong. Hopefully someone will correct me.
>
> Cheers
>
> -- B
> ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
> Brian Schlining
> MBARI
> Software Engineer, Research and Development
> brian@xxxxxxxxx
> (831) 775-1855
>
> http://www.mbari.org/staff/brian
>
> _______________________________________________
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
Tom Kunicki
Center for Integrated Data Analytics
U.S. Geological Survey
8505 Research Way
Middleton, WI 53562
