Hi Roy and Ethan,
Just for clarity this is a more of a warning to indicated the presence of a
PermGen leak. What a PermGen leak means in this context is that the classes
associated with the TDS war can not be unloaded from PermGen memory. This is
of no consequence unless the application is undeployed or redeployed during a
given runtime of the JVM. Because the classes for the TDS war cannot be
unloaded the classes loaded for each deployment stay in memory. After a
certain number of deployment cycles PermGen memory will be exhausted and the
JVM will effectively become unresponsive (classes can't be loaded, strings
can't be interned, etc).
Longer explanation of this specific error...
The specific message states that a value stored in a ThreadLocal instance is a
class loaded from the TDS war. The implementation of ThreadLocal (under the
JRE hood) uses a static variable in a class loaded by the system class loader
(present the entire JVM runtime). Now since this value is loaded by the TDS
webapp class loader and there is a reference to this value held by a
ThreadLocal (loaded by the system class loader) and every class instance has a
reference to it's class loader (and the class loader has a reference to all the
classes it's loaded) the classes loaded for the TDS deployment can now never be
unloaded.
The way to avoid this is to *always* call remove when ThreadLocal is used.
Practically speaking many of these leaks are introduced by dependencies (like
Log4J) but many of these jars offer clean-up routines. The error below is
introduced by Log4J and one should be able to call
org.apache.log4j.LogManager.shutdown() on servlet unloading to force it to
release all it's resources.
These are very difficult leaks to track down and they are often (but not
always) introduced by dependencies. I would hope some effort would be used to
track these down and remove them as often times restarting a JVM server on
redeploy is not an ideal solution.
Tom
On Apr 18, 2011, at 6:19 PM, Ethan Davis wrote:
> Hi Roy,
>
> Yes, the TDS gets this type of messages on TDS shutdown when running
> under Tomcat 6.0.24 or later. Tomcat looks for memory (thread) leaks
> when web applications are stopped and tries to shut the threads down.
> Tomcat evidently started detecting this particular type of thread
> leakage in Tomcat 6.0.24 but proper shutdown of them isn't available
> until Tomcat 7.0.6 [1]. It does not appear that these threads are leaked
> except upon shutdown.
>
> This could be an issue if you reload (stop/start) the TDS (or any other
> web app) a number of times without restarting Tomcat. Because of this
> and other issues, we recommend stop/starting Tomcat when possible (at
> least occasionally) rather than stop/starting the TDS.
>
> We haven't yet thoroughly tested the TDS on Tomcat 7 so we don't
> recommend switching to Tomcat 7 yet.
>
> Ethan
>
> [1] http://wiki.apache.org/tomcat/MemoryLeakProtection#customThreadLocal
>
> On 4/16/2011 5:03 PM, Roy Mendelssohn wrote:
>> Hi All:
>>
>> Running TDS 4.2.6. Had to do a bunch of start/stops to get the ncISO
>> services and F-TDS running. On shutdown, I get a string of errors of the
>> form:
>>
>> SEVERE: The web application [/thredds] created a ThreadLocal with
>> key of type [org.apache.log4j.helpers.ThreadLocalMap] (value
>> [org.apache.log4j.helpers.ThreadLocalMap@612ef6]) and a value of type
>> [java.util.Hashtable] (value [{startTime=1302990404971, userid=-,
>> host=192.168.31.18, ident=-, ID=197, request="GET
>> /thredds/wms/SODA/2.0.3?VERSION=1.3.0&SERVICE=WMS&REQUEST=GetMap&CRS=CRS:84&WIDTH=400&HEIGHT=200&LAYERS=temp&BBOX=-180.0,-75.25,180.0,89.25&TRANSPARENT=TRUE&STYLES=&FORMAT=image/png
>> HTTP/1.1"}]) but failed to remove it when the web application was
>> stopped. This is very likely to create a memory leak.
>>
>> Thanks,
>>
>> -Roy
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
Tom Kunicki
Center for Integrated Data Analytics
U.S. Geological Survey
8505 Research Way
Middleton, WI 53562
