Dennis, Jeff,
In a browser I get the same message reported by Dennis, below, but this
is probably because a 64 Mb limit on opendap requests is imposed by the
server (see note at bottom of
http://www2-pcmdi.llnl.gov/esg_data_portal/dapserver). ncdump -v pr
yields (on the same URL)
data:
pr =
; // UNDEFINED
If yearly, rather than monthly data is selected (here using variable
"fd", or frost days, since there is no prw or pr in the yearly archive)
http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_1/pcmdi.ipcc4.gfdl_cm2_1.20c3m.run1.atm.yr.xml.ascii?pr
the browser does return the data -- screenfulls of it. Dataset size in
this case is only about 7 Mb. But with ncdump
$ usr/local/bin/ncdump -v fd
http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_1/pcmdi.ipcc4.gfdl_cm2_1.20c3m.run1.atm.yr.xml
Warning::USE_CACHE: not currently supported
Warning::MAX_CACHE_SIZE: not currently supported
Warning::MAX_CACHED_OBJ: not currently supported
Warning::IGNORE_EXPIRES: not currently supported
Warning::CACHE_ROOT: not currently supported
Warning::DEFAULT_EXPIRES: not currently supported
Warning::ALWAYS_VALIDATE: not currently supported
Warning::Lost attribute: cdd
Warning::Lost attribute: time
Warning::Lost attribute: lat
Warning::Lost attribute: lon
Warning::Lost attribute: etr
Warning::Lost attribute: r10
Warning::Lost attribute: bnds
Warning::Lost attribute: r5d
Warning::Lost attribute: height
Warning::Lost attribute: fd
Warning::Lost attribute: hwdi
Warning::Lost attribute: sdii
Warning::Lost attribute: r95t
Warning::Lost attribute: tn90
Warning::Lost attribute: gsl
and in the dump file,
data:
fd =
; // UNDEFINED
The "Lost attributes" listed are all the variables in the dataset, with
the exception of compound dimensions like fd.time, fd.lat, fd.lon...
----------------------
In Python, where datobj = netCDF4.Dataset(URL),
In [86]: fdobj = datobj.variables['fd']
In [87]: fdobj.shape
Out[87]: (140, 90, 144)
In [88]: fd = fdobj[:]
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
/home/amg/work/nhmm/<ipython console> in <module>()
/home/amg/usr/local/cdat/trunk/lib/python2.5/site-packages/netCDF4.so in
netCDF4.Variable.__getitem__ (netCDF4.c:14286)()
/home/amg/usr/local/cdat/trunk/lib/python2.5/site-packages/netCDF4.so in
netCDF4.Variable._get (netCDF4.c:18945)()
RuntimeError: NetCDF: Variable has no data in DAP request
---------------------
So... there seems to be a discordance between what the browser sees and
what ncdump sees, and Python seems to be seeing the latter.
Thanks for the follow-up, hope this is helpful.
Best,
Arthur
Dennis Heimbigner wrote:
Jeff- Please do the following.
1. set the environment variable OCLOGFILE to the empty string
(e.g. in bourne shell: OCLOGFILE="" ; export OCLOGFILE).
2. rerun the request
http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_0/pcmdi.ipcc4.gfdl_cm2_0.20c3m.run1.atm.mo.xml
This should give extra information.
I did that request thru a web browser
(http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_0/pcmdi.ipcc4.gfdl_cm2_0.20c3m.run1.atm.mo.xml.ascii?prw)
and got this message:
Error {
code = 2;
message = "Request for 87091200 bytes exceeds limit: 67108864";
}
And I will curious to see if you get that message also.
=Dennis Heimbigner
Jeff Whitaker wrote:
Dennis Heimbigner wrote:
I found and fixed the bug. I did not
implement the libdap algorithm quite
correctly. The fix should be available
in the netcdf snapshot tomorrow.
Thanks for pointing this out.
=Dennis Heimbigner
Dennis: Although the attributes are now appearing correctly, I'm
still having trouble accessing the IPCC data. For example,
ncdump -v prw
http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_0/pcmdi.ipcc4.gfdl_cm2_0.20c3m.run1.atm.mo.xml
gives
data:
prw =
; // UNDEFINED
}
with the Sept 16 snapshot. The coordinate variables seem to work fine
data:
lat = -89, -87, -85, -83, -81, -79, -77, -75, -73, -71, -69, -67, -65,
-63,
-61, -59, -57, -55, -53, -51, -49, -47, -45, -43, -41, -39, -37, -35,
-33, -31, -29, -27, -25, -23, -21, -19, -17, -15, -13, -11, -9, -7,
-5,
-3, -1, 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33,
35, 37, 39, 41, 43, 45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67,
69,
71, 73, 75, 77, 79, 81, 83, 85, 87, 89 ;
}
but all the data grids come back undefined. The same thing happens
when trying to access data from the IRI data library
ncdump -v hgt
http://iridl.ldeo.columbia.edu/expert/SOURCES/.NOAA/.NCEP/.EMC/.CFS/.MONTHLY_REALTIME/.PRESS/.hgt/dods
hgt =
; // UNDEFINED
}
Other dap URLs work fine though, for example
ncdump -v vvelprs
http://nomad1.ncep.noaa.gov:9090/dods/reanalyses/reanalysis-2/6hr/pgb/pgb
-Jeff
Jeff Whitaker wrote:
Dennis: If I open
http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_1/pcmdi.ipcc4.gfdl_cm2_1.20c3m.run1.atm.mo.xml
with netcdf-4.1-beta2, then get the time variable, it appears there
are no attributes associated with it. Browsing the xml file
http://esgcet.llnl.gov/aggregates/ipcc4/20c3m/gfdl_cm2_1/pcmdi.ipcc4.gfdl_cm2_1.20c3m.run1.atm.mo.xml
it does appear these attributes are defined, but somehow the client
is not getting them.
Is it a bug in the way there are serving the data, or a bug in the
opendap client in netcdf-4.1?
-Jeff
--
*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*
Arthur M. Greene, Ph.D.
The International Research Institute for Climate and Society
The Earth Institute, Columbia University, Lamont Campus
Monell Building, 61 Route 9W, Palisades, NY 10964-8000 USA
amg at iri dot columbia dot edu | http://iri.columbia.edu
*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*
begin:vcard
fn:Arthur M. Greene
n:Greene;Arthur M.
email;internet:amg@xxxxxxxxxxxxxxxx
tel;work:845-680-4436
tel;fax:845-680-4865
x-mozilla-html:FALSE
version:2.1
end:vcard
