Ashwin,
I believe that’s correct. To be honest I am not well versed on what can/cannot
be done with the Java and C implementations of the NetCDF library once the data
have been subset and retrieved. But given that nccopy/ncdump/etc. are “client”
programs that are built on the NetCDF lib to achieve their functionality I
imagine that source code for these programs will provide examples of how to
save the response as a NetCDF file on your local disk. And I suspect that the
Java NetCDF library must have similar “client” tools that could be reviewed to
find the analogous examples in Java.
Antonio Cofino also makes the salient point (in an email that is now orphaned
from this response) that you should be thoughtful and subset/retrieve the
coordinate/map/shared_dimension variables associated with the core data so that
all of the relevant information is captured in the resulting NetCDF result.
Without this the downstream software may face additional challenges.
You might write to the operators of www.esrl.noaa.gov (esrl.psd.data@xxxxxxxx)
and ask them if they have enabled or disabled the TDS feature that will allow
you to make the subset request in such a way as to prompt the TDS to return a
NetCDF file rather than a DAP2 or ASCII encoded response.
Sincerely,
Nathan
> On May 27, 2016, at 7:30 AM, ashwinD12 . <winash12@xxxxxxxxx> wrote:
>
> Nathan,
> That is a very important point you bring up. If I want to write
> my own Java client for instance I have to convert the OpenDAP 2 response into
> a netcdf 4 file on my disk. It is very likely that the THREDDS server does
> not serve netcdf files but I can get the response which is OpenDAP and
> convert it into a netcdf 4 file.
> Correct ?
> Regards,
> Ashwin.
>
> On Fri, May 27, 2016 at 7:53 PM, Nathan Potter <ndp@xxxxxxxxxxx
> <mailto:ndp@xxxxxxxxxxx>> wrote:
> Ashwin,
>
> I’m pleased! I want to clarify what’s happen though - The data is held in a
> NetCDF file on the server. However, what is transported is NOT a NetCDF file,
> but rather the response is encoded as a DAP2 metadata/data response which is
> in turn loaded into the (client side) NetCDF library and then written to
> local disk as a NetCDF file. DAP2 is the transport protocol/encoding.
>
> Some servers (Hyrax being my favorite example ;) are able to encode the
> response as a NetCDF file at the server end. This behavior can be triggered
> (in Hyrax) by replacing the original “.ascii” component of the path URL with
> “.nc”. For example:
>
>
> These ASCII request URLs:
> http://test.opendap.org:8080/opendap/data/nc/fnoc1.nc.ascii
> <http://test.opendap.org:8080/opendap/data/nc/fnoc1.nc.ascii>
> http://test.opendap.org:8080/opendap/data/hdf5/grid_1_2d.h5.ascii
> <http://test.opendap.org:8080/opendap/data/hdf5/grid_1_2d.h5.ascii>
>
> Become NetCDF request URLs:
> http://test.opendap.org:8080/opendap/data/nc/fnoc1.nc.nc
> <http://test.opendap.org:8080/opendap/data/nc/fnoc1.nc.nc>
> http://test.opendap.org:8080/opendap/data/hdf5/grid_1_2d.h5.nc
> <http://test.opendap.org:8080/opendap/data/hdf5/grid_1_2d.h5.nc>
>
> I believe that the TDS (which is the server software in play in your example)
> also supports a similar feature, albeit through a somewhat different URL
> scheme. And I think that earlier in the thread someone mentioned that this
> feature is disabled on the server in question.
>
> Cheers,
>
> Nathan
>
>
>
> > On May 27, 2016, at 7:05 AM, ashwinD12 . <winash12@xxxxxxxxx
> > <mailto:winash12@xxxxxxxxx>> wrote:
> >
> > Nathan,
> > Beautiful ! That is exactly what I am looking for. I can come
> > up with my own version of nccopy in Java or Python if I so wish. So that
> > means DAP2 can download netCDF files using the PSD THREDDS server and I
> > can pass OpeNDAP constraint expressions.
> >
> > Regards,
> > Ashwin.
> >
> >
> > On Fri, May 27, 2016 at 7:17 PM, Nathan Potter <ndp@xxxxxxxxxxx
> > <mailto:ndp@xxxxxxxxxxx>> wrote:
> >
> >
> > Ashwin,
> >
> > The NetCDF library may be useful for your desired outcome. Because the
> > NetCDF library supports access via the DAP2 protocol it is a natural for
> > your probelm. I was able to use the NetCDF command line tools to accomplish
> > your task. Note that be dropping the “.ascii” suffix from the URL path you
> > return to the core “DAP2 URL” which is an important step in the following
> > example.
> >
> > Retrieving the data subset and saving it as a NetCDF-4 file:
> >
> > ./nccopy -4
> > "http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis2/pressure/air.2014.nc?air[480:603][2][20:34][26:40]
> >
> > <http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis2/pressure/air.2014.nc?air[480:603][2][20:34][26:40]>”
> > foo.nc <http://foo.nc/>
> >
> > Which was easily read by ncdump:
> >
> > ./ncdump -h foo.nc <http://foo.nc/>
> > netcdf foo {
> > dimensions:
> > time = UNLIMITED ; // (124 currently)
> > lat = 15 ;
> > level = 1 ;
> > lon = 15 ;
> > variables:
> > short air(time, level, lat, lon) ;
> > air:long_name = "6-hourly Air Temperature on Pressure Levels"
> > ;
> > air:valid_range = -32765s, -10260s ;
> > air:unpacked_valid_range = 137.5f, 362.5f ;
> > air:actual_range = 180.6f, 325.8f ;
> > air:units = "degK" ;
> > air:add_offset = 465.15f ;
> > air:scale_factor = 0.01f ;
> > air:missing_value = 32766s ;
> > air:_FillValue = -32767s ;
> > air:precision = 2s ;
> > air:least_significant_digit = 1s ;
> > air:GRIB_id = 11s ;
> > air:GRIB_name = "TMP" ;
> > air:var_desc = "Air temperature" ;
> > air:dataset = "NCEP/DOE AMIP-II Reanalysis (Reanalysis-2)" ;
> > air:level_desc = "Pressure Levels" ;
> > air:statistic = "Individual Obs" ;
> > air:parent_stat = "Other" ;
> > air:standard_name = "air_temperature" ;
> >
> > // global attributes:
> > :Conventions = "CF-1.0" ;
> > :title = "4x Daily NCEP/DOE Reanalysis 2" ;
> > :history = "created 2015/02 by NOAA/ESRL/PSD" ;
> > :comments = "Data is from \nNCEP/DOE AMIP-II Reanalysis
> > (Reanalysis-2)\n(4x/day). It consists of most variables interpolated
> > to\npressure surfaces from model (sigma) surfaces." ;
> > :platform = "Model" ;
> > :source = "NCEP/DOE AMIP-II Reanalysis (Reanalysis-2) Model" ;
> > :institution = "National Centers for Environmental
> > Prediction" ;
> > :References =
> > "http://www.esrl.noaa.gov/psd/data/gridded/data.ncep.reanalysis2.html
> > <http://www.esrl.noaa.gov/psd/data/gridded/data.ncep.reanalysis2.html>" ;
> > :dataset_title = "NCEP-DOE AMIP-II Reanalysis" ;
> > :DODS_EXTRA.Unlimited_Dimension = "time" ;
> > }
> >
> >
> > I am sure that there are several other ways to achieve a similar result,
> > but this one does seem simple and amenable to programatic (well, at least
> > script level) use.
> >
> > I believe that both the NetCDF C and Java APIs will also happily detect
> > that a supplied file name is a URL and then dereference and retrieve data
> > from it.
> >
> > Does this in any way address your problem?
> >
> > Regards,
> >
> > Nathan
> >
> >
> >
> >> On May 27, 2016, at 2:13 AM, ashwinD12 . <winash12@xxxxxxxxx
> >> <mailto:winash12@xxxxxxxxx>> wrote:
> >>
> >> Regarding the specifics of what you have written in relation to the
> >> programming language of choice
> >>
> >> 1) If I use java netcdf(the latest version) I can use the OPeNDAP URL that
> >> James explained to me and send it to the URL -
> >> http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis2/pressure/air.2014.nc.ascii?air[480:603][2][20:34][26:40]
> >>
> >> <http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis2/pressure/air.2014.nc.ascii?air[480:603][2][20:34][26:40]>
> >> (as an example)
> >> the data returned will be ascii but the java netcdf library will allow me
> >> to generate a netcdf file containing that data precisely that can be
> >> written to disk. Am I correct in this ?
> >>
> >> 2) The other option is to use the URL you gave me and generate the URL
> >> with the required parameters and then use Python and pycurl and write out
> >> the netcdf file to disk.
> >>
> >> Your input is appreciated,
> >> Regards,
> >> Ashwin.
> >>
> >
> > = = =
> > Nathan Potter ndp at opendap.org
> > <http://opendap.org/>
> > OPeNDAP, Inc. +1.541.231.3317
> >
> >
>
> = = =
> Nathan Potter ndp at opendap.org <http://opendap.org/>
> OPeNDAP, Inc. +1.541.231.3317
>
>
= = =
Nathan Potter ndp at opendap.org
OPeNDAP, Inc. +1.541.231.3317
