I haven't enabled remote monitoring yet, but the problem disappeared
since I set the low-watermark of the files-cache to 0, so after 10min
the cache is completely cleared.
I think the culprit are files being removed from another
cleanup-process, while still being in the cache. (i.e. forecast-files).
With a cache of only 10min, instead of 200 files forever, the problem is
not completely solved, but doesn't seem to appear any longer (stable for
a week yet). And we didn't recognize any performance degration.
Heiko
Richard Signell wrote:
Heiko,
Did you ever figure this out? We just had our
thredds4.0.26/tomcat6.0.18 crash with "Too man open files" also.
-Rich
On Fri, Sep 18, 2009 at 4:53 AM, Heiko Klein <Heiko.Klein@xxxxxx> wrote:
Hi,
we just had a crash of the thredds4.0.25/tomcat server and I was able to
analyse a bit the system before the operators restarted it.
Error message was: 'Too many open files' and checking the open files with
lsof or /proc/PID/fd showed 1021 open netcdf-files for the tomcat-pid. We
haven't made any changes to the thredds4 configuration concerning the
file-caches, so they should be between 200-400. We're using WCS, too, so
this might add some more files, but still not up to 1021.
We also use some aggregations of netcdf files, so one 'thredds-aggregation'
might consist of some hundred netcdf-files. Maybe this is counted as one
file in the fileCache?
I currently try to avoid further crashes by increasing the ulimit -n from
1024 to 16384, but I would rather be able to tune the file-cache properly
instead of throwing more resources at it.
Best regards,
Heiko
--
Dr. Heiko Klein Tel. + 47 22 96 32 58
Development Section / IT Department Fax. + 47 22 69 63 55
Norwegian Meteorological Institute http://www.met.no
P.O. Box 43 Blindern 0313 Oslo NORWAY
_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
Dr. Heiko Klein Tel. + 47 22 96 32 58
Development Section / IT Department Fax. + 47 22 69 63 55
Norwegian Meteorological Institute http://www.met.no
P.O. Box 43 Blindern 0313 Oslo NORWAY
