Rob Weingruber wrote:
Alrighty....
ftp to ftp.rap.ucar.edu <ftp://ftp.rap.ucar.edu> and login as anonymous
(using your email as the password).
cd pub/irap/weingrub
and the file is called caron.jar
Unjar. (Lemme know if something aint working ;-).
The 2006070611 directory has 11:00Z gen time data, with final analysis data
for 8:00, 9:00 and 10:00. The 11:00 data in that directory is the 0
hour forecast,
the 12:00 data is the 1 hour forecast, etc.
im trying to figure out if that info is in the file itself:
in
2006070611/wrfout_d01_2006-07-06_080000.DPG_F.nc
i see global attribute
:START_DATE = "2006-07-06_07:00:00";
:SIMULATION_START_DATE = "2006-07-03_12:00:00";
and variable data
char Times(Time=1, DateStrLen=19);
data: "2006-07-06_08:00:00"
i assume this is "valid time" 2006-07-06_08:00:00,
but the "run time" of 2006-07-06_11:00:00 doesnt seem to be in there
in
wrfout_d01_2006-07-06_130000.DPG_PFCST.nc
i see
:START_DATE = "2006-07-06_10:00:00";
:SIMULATION_START_DATE = "2006-07-03_12:00:00";
char Times(Time=1, DateStrLen=19);
data:"2006-07-06_13:00:00"
So again theres no indication of the run time.
if we can figure out this, we can probably use FMRC aggregation.
The 2006070614 directory has 14:00Z gen time data, with final analysis data
for 11:00, 12:00 and 13:00. The 14:00 data in that directory is the 0
hour forecast,
the 15:00 data is the 1 hour forecast, etc.
Also, the timesExplicit.xml file is where I tried to put the time
coordinates explicitly,
with the intention of being able to retrieve times, from Java, without
opening the files -
to no avail (proven by moving the data files out of the way).
The problem is that all of the variables in the file have the time dimension, and so get aggregated, including some that we'd rather not.
When the coordinate system is constructed, we read in the ZNU field, which has
a time dimension, and so we have to access all files.
Ill have to see if the FMRC aggregation has the same problem.
-----
Another problem with this file is that there is no coordinate variable, ie
Time(Time), which the joinExisting spec requires. It seems to be tolerating it
ok, but it screws things up some.
The only thing you could do is add it in the NcML, but you have to give it
values, eg:
<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
<variable name="Time" shape="Time" type="String">
<values>2006-07-06T08:00:00Z 2006-07-06T09:00:00Z etc </values>
</variable>
<aggregation dimName="Time" type="joinExisting">
<netcdf location="file:C:/data/rap/2006070611/wrfout_d01_2006-07-06_080000.DPG_F.nc"
coordValue="2006-07-06T08:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070611/wrfout_d01_2006-07-06_090000.DPG_F.nc"
coordValue="2006-07-06T09:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070611/wrfout_d01_2006-07-06_100000.DPG_F.nc"
coordValue="2006-07-06T10:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070611/wrfout_d01_2006-07-06_110000.DPG_PFCST.nc"
coordValue="2006-07-06T11:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070611/wrfout_d01_2006-07-06_120000.DPG_PFCST.nc"
coordValue="2006-07-06T12:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070611/wrfout_d01_2006-07-06_130000.DPG_PFCST.nc"
coordValue="2006-07-06T13:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070611/wrfout_d01_2006-07-06_140000.DPG_PFCST.nc"
coordValue="2006-07-06T14:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070611/wrfout_d01_2006-07-06_150000.DPG_PFCST.nc"
coordValue="2006-07-06T15:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070614/wrfout_d01_2006-07-06_110000.DPG_F.nc"
coordValue="2006-07-06T11:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070614/wrfout_d01_2006-07-06_120000.DPG_F.nc"
coordValue="2006-07-06T12:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070614/wrfout_d01_2006-07-06_130000.DPG_F.nc"
coordValue="2006-07-06T13:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070614/wrfout_d01_2006-07-06_140000.DPG_PFCST.nc"
coordValue="2006-07-06T14:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070614/wrfout_d01_2006-07-06_150000.DPG_PFCST.nc"
coordValue="2006-07-06T15:00:00Z"/>
<netcdf location="file:C:/data/rap/2006070614/wrfout_d01_2006-07-06_160000.DPG_PFCST.nc"
coordValue="2006-07-06T16:00:00Z"/>
</aggregation>
</netcdf>
more later...
The
timesFromFiles.xml
file is working fine, though retrieves times from the files, albeit
slowly (esp if we have
tons of files).
Lastly, is there even a way thru the *API*, where I can extract times
without using
a GeoGrid (or GridDatatype)? - It seems to me that once I have a
GeoGrid, I have
already opened the file(s).
Normally the time coordinate would be available through any of the interfaces
(NetcdfFile, NetcdfDataset, Grid) without having to read all the data (assuming
you've specified it in the NcML), but the problems above are interfering with
that.
My apologies for my confusion ;-)
John Caron wrote:
probably better pput them on an http or ftp server
Rob Weingruber wrote:
Hola -
They're about 18M each - ok via email? Else Ill find somewhere to
put em for ya....
Lemme know ;-)
Thanks.
John Caron wrote:
send me a few of your files, or links to them, and ill put an NcML
file together for you.
Rob Weingruber wrote:
Hola -
How busy are you? ;-)))) I was wondering if you might be willing
to come on over here and show me a thing or two, wrt nj22 and ncml.
Im struggling with the time, aggregation and the caching issues, and I
bet you'd be able to clear lots of things up for me. The real jist
is,
Im not sure I really know how to use the nj22/ncml classes, and I dont
know if I/we can do it efficiently, especially for time list requests.
Or, maybe snippets of code could help too?
Whadayathink?
;-)
John Caron wrote:
Rob Weingruber wrote:
Hi John -
Im back at it.....
Ok, so after perusing GridDatatype andGridCoordSystem, I have a
few questions:
1. Can you explain the ensemble to me? Is that for
climatologies? (if so, i understand
the ensemble ;-).
models are sometimes run several (many) times with slight
variations of the input, these are called "ensemble" runs. The set
of results then show the fuuture in a more statistically
meaningful way. these results could be represented as an extra
"ensemble" dimension.
we havent yet tested this with any real world examples yet.
2. Is the RunTime the same as what I termed the 'generate time'?
If so, might be
better to use 'gen time' rather than run time, since a model run
may have a gen time
of 12Z, but it happens to be running at 12:18Z (my $0.02 ;-).
im not sure, i could also imagine the argument "a model run may
have a runtime of 12Z, but it happens to be generated at 12:18Z".
maybe we can have a bakeoff, i've also heard "analysis time".
Related to NCML:
3. Can I use the 'Forecast Model Run' yet? If so, whudoIdo?
we are just finishing the "Forecast Model Run Collection" (fmrc)
aggregation, you can try it out if you want, but i havent written
docs yet. (there was an older "Forecast Model Run" aggregation
which is now deprecated.)
lets pick a simple dataset you want to do, and try serving it
through the TDS:
1. have a look at the motherlode test server models
(http://motherlode.ucar.edu:9080/thredds/idd/models.html)
2. install latest 3.13 development version of thredds.war (from
http://www.unidata.ucar.edu/downloads/thredds/index.jsp)
3. pick a simple dataset, i will show you what the catalog should
be.
4. ill try to get some docs written.
now that ive said all this, maybe you just want to access the fmrc
aggregation dataset from java code?
4. How can I aggregate with the 2D time coordinate? I looked at
the joinExisting
doc
http://www.unidata.ucar.edu/software/netcdf/ncml/v2.2/Aggregation.html,
and I still dunno what to do....
its likely you need the fmrc aggregation.
;-) Thanks John!
yer welcome!
