Hi Kevin:
Apologies for incorrect information. Union Aggregations do leave all the
files open at once (other aggregations do not). We need to look at your
dataset to see what the alternatives are.
John
On 10/24/2013 4:35 PM, John Caron wrote:
Hi all:
Aggregations arent supposed to leave files open. If they are, thats a bug.
The number of open files should typically be less than 1000, no matter how big
your aggregations are. It is dependent on the file cache sizes, see:
http://www.unidata.ucar.edu/software/thredds/v4.4/tds/reference/ThreddsConfigXMLFile.html#FileCache
So if you are seeing lots of open files, we have a file leak i think. You can
verify by turning your file cache to 0 or very low (10?). Then watch the pfiles
output to see whats happening.
John
On 10/24/2013 3:15 PM, Kevin Manross wrote:
This is very helpful Mike - thank you very much for the insight and your
experience.
This will be useful information for requesting the ulimit increase from
our sysadmins!
-kevin.
On 10/24/13 3:09 PM, Michael McDonald wrote:
Kevin,
We needed to add a script to "/etc/profile.d" that increases the stock
CentOS/RHEL 6.x number of open files. Change $USER == "tomcat" to
whatever you run tomcat/apache as (see below). Also assumes that the
user is running the sh/bash shell.
# cat /etc/profile.d/thredds.sh
export TOMCAT_HOME="/usr/share/tomcat6"
if [ $USER == "tomcat" ]; then ulimit -n 1000000; fi
Our THREDDS servers providing only OPENDAP requests are averaging 40k
open files. Our THREDDS servers providing only NCSS requests are
averaging 15k open files. We have a lot of files in some of our hycom
datasets.
/mike
On Thu, Oct 24, 2013 at 1:50 PM, Kevin Manross<manross@xxxxxxxx> wrote:
Hi there,
I am trying to aggregate netcdf files for some rather large datasets with
tomcat6.0.36/TDS4.19 - 20131008.1358 and am running into a "Too many open
files" error.
My catalog scan is:
<dataset name="ASR 30 km 2d surface analysis"
ID="ds631.0_anl_2d-AGG"
urlPath="ds631.0/9/best"
harvest="true">
<serviceName>all</serviceName>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
<aggregation dimName="time" type="Union">
<scan location="/glade/p/rda/data/ds631.0/asr30km.anl.2d/"
suffix=".nc" subdirs="false"/>
</aggregation>
</netcdf>
</dataset>
which contains about 4020 files and totals 470G.
The message that the server sends when trying to access the OPeNDAP service
is:
Error {
code = 404;
message =
"/glade/p/rda/data/ds631.0/asr30km.anl.2d/asr30km.anl.2d.20010619.nc (Too
many open files)";
};
When trying the CdmRemote service catalina.out repeatedly reads:
Oct 24, 2013 11:20:39 AM org.apache.jk.common.ChannelSocket
acceptConnections
WARNING: Exception executing accept
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at
java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
at java.net.ServerSocket.implAccept(ServerSocket.java:478)
at java.net.ServerSocket.accept(ServerSocket.java:446)
at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:311)
at
org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:668)
at
org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:879)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
at java.lang.Thread.run(Thread.java:679)
The cache settings in my threddsConfig.xml are the default settings.
Have I overlooked something?
-kevin.
--
Kevin Manross
NCAR/CISL/Data Support Section
Phone: (303)-497-1218
Email:manross@xxxxxxxx
Web:http://rda.ucar.edu
_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
Kevin Manross
NCAR/CISL/Data Support Section
Phone: (303)-497-1218
Email:manross@xxxxxxxx <mailto:manross@xxxxxxxx>
Web:http://rda.ucar.edu
_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
