Hi Benno,
I just have to change the name of dimension to have a continuous update of the
targeted file, we mean ?
Thank you for the correction in my interpretation of the CF convention.
Regards,
Ing. Claude DEROGNAT, PhD
ARIA TECHNOLOGIES SA
8-10, rue de la Ferme - 92100 Boulogne-Billancourt - France
tél. / phone: 33 (0)1 46 08 68 60
ligne directe / direct line: 33 (0)1 46 08 68 65
fax: 33 (0)1 41 41 93 17
e-mail: cderognat@xxxxxxx <mailto:cderognat@xxxxxxx> - http : www.aria.fr
P Avant d'imprimer ce courriel, réfléchissez à l'impact sur l'environnement, et
dans la mesure du possible éditez les documents en recto verso, merci!
De : bennoblumenthal@xxxxxxxxx [mailto:bennoblumenthal@xxxxxxxxx] De la part de
Benno Blumenthal
Envoyé : jeudi 23 février 2012 22:12
À : Claude DEROGNAT
Cc : Nathan Potter; support@xxxxxxxxxxx; Patrick West; Nathan Potter;
thredds@xxxxxxxxxxxxxxxx
Objet : Re: [thredds] OLFS and/or BES configuration to obtain a dynamic refresh
of existing files ...
While I haven't seen the original file, my point is that you are violating the
CF convention (assuming I am correct in thinking that the TDS matches the
netcdf structure): variables that have the same name as a dimension need to be
one dimensional.
Benno
On Thu, Feb 23, 2012 at 3:57 PM, Claude DEROGNAT <cderognat@xxxxxxx> wrote:
Hi Benno,
Thank you for your answer. The corresponding name between grid coordinates and
array dimension are driven by the NetCdf CF convention...
What about my real time constrain, so the require continuous update of the
provides data ? How can I reach the property from my TDS catalog ?
Regards,
Claude
Ing. Claude DEROGNAT, PhD
ARIA TECHNOLOGIES SA
8-10, rue de la Ferme - 92100 Boulogne-Billancourt - France
tél. / phone: 33 (0)1 46 08 68 60
ligne directe / direct line: 33 (0)1 46 08 68 65
fax: 33 (0)1 41 41 93 17
e-mail: cderognat@xxxxxxx <mailto:cderognat@xxxxxxx> - http : www.aria.fr
P Avant d'imprimer ce courriel, réfléchissez à l'impact sur l'environnement, et
dans la mesure du possible éditez les documents en recto verso, merci!
De : bennoblumenthal@xxxxxxxxx [mailto:bennoblumenthal@xxxxxxxxx] De la part de
Benno Blumenthal
Envoyé : jeudi 23 février 2012 19:45
À : Nathan Potter
Cc : Claude DEROGNAT; support@xxxxxxxxxxx Support; Patrick West; Nathan Potter;
thredds@xxxxxxxxxxxxxxxx
Objet : Re: [thredds] OLFS and/or BES configuration to obtain a dynamic refresh
of existing files ...
As for the two different structures -- the TDS generated a pure flat structure,
Hyrax made Grid structures, from the netcdf API point of view they are very
similar, though there is a significant difference w.r.t. height.
I wonder if the height variable is the reason TDS did not try to generate Grid
structures -- It is a netcdf convention that the height dimension should have a
height variable that is also one-dimensional. In this case, the height
variable is four-dimensional. If the height dimension were renamed slightly,
e.g. heightdim, you might get very different (and more consistent) results.
Benno
On Thu, Feb 23, 2012 at 1:09 PM, Nathan Potter <ndp@xxxxxxxxxxx> wrote:
On Feb 23, 2012, at 4:24 AM, Claude DEROGNAT wrote:
Hi,
My IT said that the file I send you is an old one. My problem is probably link
to my THREEDS catalog.
The question is so probably to threeds mailing list. I send them my question on
Monday just after your message and I still have no response back ...
So, I send you my catalog if you can have a look to it you may find why there a
mismatch between
http://web.aria.fr:443/thredds/dodsC/CHIMERE@CAMAC@reference@p02/CF_CHIMERE_20120115_d03.nc.html
and
http://web.aria.fr:443/LENVIS/CHIMERE/CAMAC/reference/p02/CF_CHIMERE_20120115_d03.nc.html
Claude,
I look at the first URL, saw that it was a instance of THREDDS Data Server (a
fairly recent version 4.0.26) and I forwarded you message to their support list.
It was only later that I realized the second URL is what appears to be a VERY
old (circa version 1.5.0), locally compiled, version of our Hyrax server.
So you are running two entirely different implementations of an OPeNDAP server.
You can see this clearly when you look at one of the collections with a browser:
TDS: http://web.aria.fr:443/thredds/dodsC/CHIMERE@CAMAC@reference@p02/
Hyrax: http://web.aria.fr:443/LENVIS/CHIMERE/CAMAC/reference/p02/
why the second is continuously updated but doesn't allow the access to
the whole set of variables stored in NetCdf file.
Why the first one are not continuously updated even if the date in the
threeds catalogs presentation always mentioned the right modified time ...
So with regards to how often datasets are "updated" - this is a function of
how the servers are implemented. I don't know what is going on in the TDS, but
in the Hyrax server each new request gets the file information regarding file
size and last modified times from the files system on which the dataset
resides. Hyrax also doesn't cache responses, so it will re-interrogate the file
every time you ask for something in or about the file.
The TDS folks are better informed to comment on what's going on in the TDS with
regards to this.
Now I think you are also saying that these two URLs:
http://web.aria.fr:443/thredds/dodsC/CHIMERE@CAMAC@reference@p02/CF_CHIMERE_20120115_d03.nc
http://web.aria.fr:443/LENVIS/CHIMERE/CAMAC/reference/p02/CF_CHIMERE_20120115_d03.nc
Reference the exact same file on the exact same file system. Is that the case?
If so, the fact that the data structure representations generated by each
server for this file are so different is somewhat surprising:
>From the TDS:
http://web.aria.fr:443/thredds/dodsC/CHIMERE@CAMAC@reference@p02/CF_CHIMERE_20120115_d03.nc.dds
Dataset {
String transverse_mercator;
Float32 x[lon = 101];
Float32 y[lat = 101];
Float32 lat[time = 48][lat = 101][lon = 101];
Float32 lon[time = 48][lat = 101][lon = 101];
String projectionCoordSys;
Float32 height[time = 48][height = 25][lat = 101][lon = 101];
Float32 time[time = 48];
Float32 Time[Time = 48];
Float32 REL[time = 48][lat = 101][lon = 101];
Float32 PHI[time = 48][height = 25][lat = 101][lon = 101];
Float32 M001S001[time = 48][height = 25][lat = 101][lon = 101];
} CHIMERE%40CAMAC%40reference%40p02/CF_CHIMERE_20120115_d03.nc;
>From (very old) Hyrax:
http://web.aria.fr:443/LENVIS/CHIMERE/CAMAC/reference/p02/CF_CHIMERE_20120115_d03.nc.dds
Dataset {
String transverse_mercator;
Grid {
Array:
Float32 x[lon = 101];
Maps:
Float32 lon[lon = 101];
} x;
Grid {
Array:
Float32 y[lat = 101];
Maps:
Float32 lat[lat = 101];
} y;
String projectionCoordSys;
Grid {
Array:
Float32 REL[time = 48][lat = 101][lon = 101];
Maps:
Float32 time[time = 48];
Float32 lat[lat = 101];
Float32 lon[lon = 101];
} REL;
Grid {
Array:
Float32 PHI[time = 48][height = 25][lat = 101][lon = 101];
Maps:
Float32 time[time = 48];
Float32 height[height = 25];
Float32 lat[lat = 101];
Float32 lon[lon = 101];
} PHI;
Grid {
Array:
Float32 M001S001[time = 48][height = 25][lat = 101][lon = 101];
Maps:
Float32 time[time = 48];
Float32 height[height = 25];
Float32 lat[lat = 101];
Float32 lon[lon = 101];
} M001S001;
Float32 Time[Time = 48];
} CF_CHIMERE_20120115_d03.nc;
This may be expected behavior, or not. I am tempted to say that you should
upgrade both the TDS Hyrax to the latest releases before we dig to deeply...
Anyone else care to comment?
Thanks,
Nathan
For instance,
why the second is continuously updated but doesn't allow the access to the
whole set of variables stored in NetCdf file.
Why the first one are not continuously updated even if the date in the threeds
catalogs presentation always mentioned the right modified time ...
Regards
Ing. Claude DEROGNAT, PhD
ARIA TECHNOLOGIES SA
8-10, rue de la Ferme - 92100 Boulogne-Billancourt - France
tél. / phone: 33 (0)1 46 08 68 60
ligne directe / direct line: 33 (0)1 46 08 68 65
fax: 33 (0)1 41 41 93 17
e-mail: cderognat@xxxxxxx - http : www.aria.fr
P Avant d'imprimer ce courriel, réfléchissez à l'impact sur l'environnement, et
dans la mesure du possible éditez les documents en recto verso, merci!
-----Message d'origine-----
De : Nathan Potter [mailto:ndp@xxxxxxxxxxx]
Envoyé : mardi 21 février 2012 17:02
À : Claude DEROGNAT
Cc : Nathan Potter; Nathan Potter; support@xxxxxxxxxxx; Patrick West
Objet : Re: OLFS and/or BES configuration to obtain a dynamic refresh of
existing files ...
Claude,
It would appear that you are using the THREDDS Data Server (TDS). I say this
because the configuration file that you attached is unique to the TDS and is
not a part of Hyrax. The TDS is a UNIDATA product and I think that they would
be better positioned to answer your questions regarding this, you xan reach
them here: thredds@xxxxxxxxxxxxxxxx
If you are pretty sure that you are trying to use Hyrax then I think we may
have entered a twilight zone where parts of the two servers are being mashed up
into one...
Can you confirm which server software you are attempting to configure and use?
Thanks,
Nathan
On Feb 21, 2012, at 5:03 AM, Claude DEROGNAT wrote:
HI,
Sorry for the delay of my answer but I had to deal with another huge
project.
My system will be dedicated to real time plume modeling.
The model runs every 30 minutes. At the beginning of each day, it
creates a result file then every 30 minutes the result file is overwritten with
a new one containing the additional time frame.
I observed à strange behavior between Opendap and thredds.
The file is continuously updated in the OpenDap server.
The threeds catalog notices in the 'Dates' field that the file are
updates but the Access/OpenDap target file are not updated. The available time
doesn't be continuously updated. May I have to perform any modification in the
thredds configuration ? A attached you my threddsConfig.xml file...
Regards
Ing. Claude DEROGNAT, PhD
ARIA TECHNOLOGIES SA
8-10, rue de la Ferme - 92100 Boulogne-Billancourt - France
tél. / phone: 33 (0)1 46 08 68 60
ligne directe / direct line: 33 (0)1 46 08 68 65
fax: 33 (0)1 41 41 93 17
e-mail: cderognat@xxxxxxx - http : www.aria.fr
P Avant d'imprimer ce courriel, réfléchissez à l'impact sur
l'environnement, et dans la mesure du possible éditez les documents en recto
verso, merci!
-----Message d'origine-----
De : Nathan Potter [mailto:npotter@xxxxxxxxxxx]
Envoyé : mardi 11 octobre 2011 19:31
À : Claude DEROGNAT; support@xxxxxxxxxxx support
Cc : Nathan Potter; Patrick West
Objet : Re: OLFS and/or BES configuration to obtain a dynamic refresh
of existing files ...
On Oct 11, 2011, at 9:22 AM, Claude DEROGNAT wrote:
Dear all,
With a correct title it should be better.
Regards,
...
Dear all,
I have a question about OpenDap functionality.
Is-it possible to configure OLFS and/or BES to obtain a dynamic
refresh of existing files.
If your model is writing to files that are within the scope of the
BES.Catalog.catalog.RootDirectory, then Hyrax will just pick up the changes in
the filesystem and refresh it's output. Little of that is cached, and if it is,
it's triggered to refresh by a change in a files last modified time.
If writing your model output into the
BES.Catalog.catalog.RootDirectory doesn't work you might be able to use
disk mounts and soft links to achieve what you want.
But I have a feeling James and I haven't answered quite the question
you were asking. The fact that James's earlier reply didn't help makes me think
that repeating his point in more detail probably didn't either.
- Are you perhaps wanting to define aggregations for a periodically
changing file set? So that the aggregation definition always reflects the
current state of the model?
- Are you talking about automatically updating thredds catalog metadata
for aggregations?
- Updating NcML definitions of aggregations to reflect the new changing
model results?
I performed daily forecast of 48h and used Opendap to provide
model
results to partners.
Thank to assimilation each 30 minutes, I want to
- add a new time frame on an existing file
Your model can just write to the file within the scope of
BES.Catalog.catalog.RootDirectory. Hyrax will see that the last- modified time
of the file has changed.
- replace (modified) some fields stored few hours ago.
Again, I think this is just your model writing to files that are within
the scope of BES.Catalog.catalog.RootDirectory
Is-is possible ?
Regards,
Ing. Claude DEROGNAT, PhD
ARIA TECHNOLOGIES SA
8-10, rue de la Ferme - 92100 Boulogne-Billancourt - France
tél. / phone: 33 (0)1 46 08 68 60
ligne directe / direct line: 33 (0)1 46 08 68 65
fax: 33 (0)1 41 41 93 17
e-mail: cderognat@xxxxxxx - http : www.aria.fr
P Avant d'imprimer ce courriel, réfléchissez à l'impact sur
l'environnement, et dans la mesure du possible éditez les
documents
en recto verso, merci!
-----Message d'origine-----
De : Nathan Potter [mailto:ndp@xxxxxxxxxxx]
Envoyé : vendredi 18 septembre 2009 18:14
À : Patrick West
Cc : Nathan Potter; Claude DEROGNAT; support@xxxxxxxxxxx
Objet : [SPAM?] Re: [Support] Bad Request 400 after a complete
re-
installation ....
Claude,
The OLFS relies on Tomcat's built in authentication mechanism.
REad
this section:
http://docs.opendap.org/index.php/Hyrax_-_OLFS_Configuration#Authentication_.26_Authorization
Of our documents for more information on that, although you
will need
to read the Tomcat documentation to get it sorted out. You can
connect
the authentication service to an LDAP server (using the
JNDIRealm) if
you wish, or you could use the example "MemoryRealm" if all you
want
is something simple but not as secure as LDAP.
Nathan
On Sep 18, 2009, at 8:26 AM, Patrick West wrote:
On Sep 18, 2009, at 5:38 AM, Claude DEROGNAT wrote:
Patrick,
Thank you very much for your help ... Thank
also Nathan and all the
opendap support !!!!
I never success without your help !!!!
Glad we could help. We are always working to make the
installs
easier, and have formed a working group around the
issue.
the configuration similar to
http://opendap.rpi.edu is perfect for
my application.
A last question ... the last I promise ;-))).
I didn't find simple way to introduce
"authentication" to restrict
access to the opendap (OLFS) server.
You look very familiar with tomcat. Do you know
simple way to only
introduce one (or more) login/password in the
tomcat server access.
I have in mind something similar to ftp access
: http://login:password@IP
:port .
I'm not sure about this last part, whether the OLFS
supports this,
can get the username and password and do anything with
it. Nathan?
Patrick
Regards,
Claude Derognat * Ingénieur R&D
ARIA TECHNOLOGIES SA
8-10, rue de la Ferme - 92100
Boulogne-Billancourt - France
tél. / phone: 33 (0)1 46 08 68 60
ligne directe / direct line: 33 (0)1 46 08 68 65
fax: 33 (0)1 41 41 93 17
e-mail: cderognat@xxxxxxx - http : www.aria.fr
P Avant d'imprimer ce courriel, réfléchissez à
l'impact sur
l'environnement, et dans la mesure du possible
éditez les documents
en recto verso, merci!
-----Message d'origine-----
De : Patrick West [mailto:pwest@xxxxxxxxxxx]
Envoyé : jeudi 17 septembre 2009 19:38
À : Nathan Potter
Cc : Claude DEROGNAT; support@xxxxxxxxxxx
Objet : Re: [Support] Bad Request 400 after a
complete re-
installation ....
For one of our projects here, actually two of
them, we have a
default
context. We set this in the server.xml file,
before the closing Host
tag:
<Context docBase="opendap" path=""
reloadable="true"
source="org.eclipse.j
st.j2ee.server:opendap"/>
</Host>
And, if you don't want to have to use :8080
when going to the
server,
and you don't already have an apache server
running, you can use
this
in your server.xml file.
<Connector port="80" address="128.113.28.182"
protocol="HTTP/1.1"
connectionTimeout="20000"
useIPVHosts="true"
redirectPort="443" />
This configuration gives us
http://opendap.rpi.edu, where we serve
some sample data.
Might need to change this in other locations as
well (port 80 for
the
port tomcat listens on, and 443 for ssl
connections.)
But if you have apache running, then this last
part won't work,
unless
you change the apache port to something other
than 80. There is a
way
to redirect in httpd.conf, but I'm not sure how
to do that.
In our case, we have one machine with three
different IP addresses,
each one with either an apache server or tomcat
server running, each
listening on port 80 for that IP address, which
is why we specify
the
address in the Connector.
Patrick
On Sep 17, 2009, at 12:58 PM, Nathan Potter
wrote:
On Sep 17, 2009, at 9:23 AM, Claude
DEROGNAT wrote:
Nathan,
Thank you for your help.
But your mail induces additional
questions ...
How do you configure Tomcat to use
Hyrax (running in
it's context of "opendap") as the
"default" servlet ?
At this point I would have to go read
through the Tomcat
documentation in order to answer your
question.
I think the answer is on this page:
http://tomcat.apache.org/tomcat-6.0-doc/default-servlet.html
But I haven't tried this and I have no
idea if it's actually
possible (or even advisable...)
I tried to couple Tomcat with Apache:
NC Services (link to BES) work properly
but I lost the most part
of
the layout: (no logo, no blue
horizontal bar, not link to
documentation...).
Did you read the section in the Hyrax
documentation on integrating
Hyrax with Apache??
http://docs.opendap.org/index.php/Hyrax_-_Apache_Integration
I know people have done this and they
have been able to make all of
it work.
What's why, I tried to deploy OLFS on
the "default" Tomcat URL (http://IP:8080
) and NAT the external IP adresse to
the LAN IP(:8080) of my
OpenDap sever. Any idea about that ?
That's always worked for me.
I just set up the firewall so that all
requests sent to http://
WAN_ADDRESS:80 are forwarded to
http://NAT_ADDRESS:8080
I hope that helps,
Nathan
...
[Message clipped]
--
Dr. M. Benno Blumenthal benno@xxxxxxxxxxxxxxxx
International Research Institute for climate and society
The Earth Institute at Columbia University
Lamont Campus, Palisades NY 10964-8000 (845) 680-4450
