Hi Pauline,
On Oct 24, 2009, at 10:30 AM, Pauline Mak wrote:
Hi all,
I think the solution depends on what statistics you're after. We
run TDS behind Apache, and then we standard apps like webalizer and
awstats to parse the apache logs to aggregate usage stats. But then
configuring these apps to get the numbers you want might be more
work than it's worth...
I may not understand the details here so my comment here may be off
base. The idea is not to look at processed logs, but to look at the
base logs, the ones generated by Apache in your case I think. In the
case of the Goddard group, they use one of the many web log analysis
programs, but I think that they have added an interface to it to allow
the data provider to retrieve the results in a fairly straightforward
way.
Peter
I have a definite need to get traffic stats on datasets. One of our
data providers (IMOS/eMII) is keen to get some numbers...
Cheers,
-Pauline.
Ted Habermann wrote:
Peter et al.,
I think John C. was correct that this could be done using the
standard web logs. I suspect that a solution build on the standard
logs, rather that extra logging built in to a server, would be most
useful. I could not tell if that was what you were suggesting...
Ted
Peter Cornillon wrote:
On Oct 24, 2009, at 8:18 AM, John Caron wrote:
Peter Cornillon wrote:
Hi Rich,
I was at a NASA meeting this week in which we were discussing
metrics for NASA related projects. As a result of the
discussion, we decided to experiment with a metrics group at
Goddard. What I want to do is to develop a general evaluation of
http logs from OPeNDAP enabled servers. The idea would be to
generate a consistent set of stats from these servers. The job
of sorting out stats can be divided into two parts, one part
that is OPeNDAP specific - how many das, dds,... accesses, how
many data accesses, what the volume of the data access was,...
The second part is provider specific, what constitutes a dataset
at that site. We will be experimenting with a Hyrax and a GDS
server, but I'm assuming that much of the OPeNDAP specific info
would be the same for a TDS. In addition to these one would
probably want to know how many WCS accesses or other accesses
were made for a TDS. Assuming that we make good progress on the
servers that we will be messing around with, it might make sense
to extend the project in the not too distant future to TDS.
Comments?
Peter
Hi Peter:
That would be very welcome.
The TDS uses standard apache log format, but we also advise to
append the amount of (wall clock) time it took to process the
request. Its up to the individual user to do this, however. Im
not sure what Hyrax/GDS uses as a log format.
The TDS uses a very regular URL naming schema that makes it
straightforward to figure out the service used. extracting the
dataset name is slightly more complicated. I can send some code
snippets if anyone is interested.
I am sure that this will be of interest when/if we get to that
point. My guess is that it will take a month or so to work out an
approach based on the Hyrax and GDS logs. Once we have done that,
assuming that Goddard is interested in pursuing this on a larger
scale, we can look at TDS logs. If Goddard is not interested in
pursuing this for the community at large, it may be worth seeking
funding to do so. I think that this would provide a very
interesting data set with which to explore use of the DAP. Thanks
for your interest John.
Peter
--
Peter Cornillon
215 South Ferry Road
Telephone: (401) 874-6283
Graduate School of Oceanography Fax:
(401) 874-6283
University of Rhode Island
Internet: pcornillon@xxxxxxxxxxx <mailto:pcornillon@xxxxxxxxxxx>
Narragansett, RI 02882 USA
------------------------------------------------------------------------
_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
Pauline Mak
Assistant Manager, ARCS Data Services
Ph: +61 3 6226 7518
Mob: +61 411 638 196
Email: pauline.mak@xxxxxxxxxxx
Jabber: pauline.mak@xxxxxxxxxxx
Calendar: http://tinyurl.com/pmak-arcs-calendar
http://www.arcs.org.au/
TPAC
Email: pauline.mak@xxxxxxxxxxx
http://www.tpac.org.au/
_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
Peter Cornillon
215 South Ferry Road Telephone:
(401) 874-6283
Graduate School of Oceanography Fax:
(401) 874-6283
University of Rhode Island
Internet: pcornillon@xxxxxxxxxxx
Narragansett, RI 02882 USA
