Yes, at least in the ISL script, while monitoring memory use via top, the
memory usage matches what the IDV shows as current memory usage. Once this
number meets the -Xmx limit, my ISL script grinds to a halt (i.e., goes from ~
4 sec per frame to over a minute, and then eventually freezes).
_____________________________________________
Kevin Tyle, Systems Administrator
Dept. of Atmospheric & Environmental Sciences
University at Albany
Earth Science 235, 1400 Washington Avenue
Albany, NY 12222
Email: ktyle@xxxxxxxxxx
Phone: 518-442-4578
_____________________________________________
-----Original Message-----
From: Julien Chastang [mailto:chastang@xxxxxxxx]
Sent: Monday, January 06, 2014 12:40 PM
To: Tyle, Kevin R; Don.Murray@xxxxxxxx; Yuan Ho
Cc: Bob Carp; Joleen Feltz; <idv-steering@xxxxxxxxxxxxxxxx>;
idvdevelopers@xxxxxxxxxxxxxxxx
Subject: Re: [idvdevelopers] 20131220: New test release
Happy New Year All.
Kevin,
Focusing on those diagnostic numbers can be somewhat misleading (because the
internal JVM memory management algorithms are changing and improving all the
time). For example, just because you see a high current memory number does not
mean there is a problem. It could just be that the JVM has decided not to
employ one of several memory reclamation (garbage
collection) strategies for whatever reason.
The real question is are you seeing your IDV crashing? Is there a problem?
Best,
-Julien
On 1/6/14 10:26 AM, Tyle, Kevin R wrote:
> It's a definite issue, and it becomes a real problem when running an ISL
> script that loops over all frames. I was hitting the 6GB limit by frame 33
> or so on one of my higher-RAM systems when running an ISL script that creates
> US forecast radar plots from the HRRR over all 61 frames.
>
> _____________________________________________
> Kevin Tyle, Systems Administrator
> Dept. of Atmospheric & Environmental Sciences University at Albany
> Earth Science 235, 1400 Washington Avenue Albany, NY 12222
> Email: ktyle@xxxxxxxxxx
> Phone: 518-442-4578
> _____________________________________________
>
>
> -----Original Message-----
> From: Don Murray (NOAA Affiliate) [mailto:don.murray@xxxxxxxx]
> Sent: Monday, January 06, 2014 12:24 PM
> To: Tyle, Kevin R; Yuan Ho
> Cc: <idv-steering@xxxxxxxxxxxxxxxx>; Bob Carp; Joleen Feltz;
> idvdevelopers@xxxxxxxxxxxxxxxx
> Subject: Re: [idvdevelopers] 20131220: New test release
>
> Kevin-
>
> I believe that the IDV used to do a garbage collection (i.e. clean up
> memory) after the Remove All Displays and Data action, but I know Julien and
> Sean did some work to turn off all garbage collection and let the system
> clean up by itself. Ideally, it should do that, but maybe that's not working
> as it should.
>
> I agree that it was nice to have the "how much you've used" to get an idea of
> when to clear the cache or whether you can actually load in that next display.
>
> Don
>
> On 1/6/14 8:05 AM, Tyle, Kevin R wrote:
>> I've noticed that this test release does not free memory when I select
>> "Remove All Displays and Data". Some times, but not most times, memory will
>> get freed when I proceed to load in a different data source. Usually I have
>> to completely restart the IDV to get memory use down.
>>
>> I've also noticed that the three memory use values in the lower left of the
>> screen (i.e., the ones that appear when you toggle off the realtime clock)
>> have been replaced by only two values (current use/upper limit of use) ...
>> i.e., the maximum use in the current session value no longer appears. That
>> was a nice feature ...
>>
>> --Kevin
>>
>> _____________________________________________
>> Kevin Tyle, Systems Administrator
>> Dept. of Atmospheric & Environmental Sciences University at Albany
>> Earth Science 235, 1400 Washington Avenue Albany, NY 12222
>> Email: ktyle@xxxxxxxxxx
>> Phone: 518-442-4578
>> _____________________________________________
>>
>>
>> -----Original Message-----
>> From: idvdevelopers-bounces@xxxxxxxxxxxxxxxx
>> [mailto:idvdevelopers-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Don
>> Murray
>> Sent: Friday, December 20, 2013 6:24 PM
>> To: Yuan Ho
>> Cc: <idv-steering@xxxxxxxxxxxxxxxx>; Bob Carp; Joleen Feltz;
>> idvdevelopers@xxxxxxxxxxxxxxxx
>> Subject: Re: [idvdevelopers] 20131220: New test release
>>
>> One note about this. If you made any bundles from the previous progressive
>> resolution test release, they will probably no longer work (that's why it's
>> called a test release ;-)). We had to make some significant changes to
>> simplify the underlying code and it was not feasible to keep the old test
>> bundles working. You will need to recreate them, which would be a good test
>> of whether things work from scratch and from the new bundles.
>>
>> And, as some of you have found, you cannot necessarily use any bundles
>> created under the test release with the official current release. But
>> please test any bundles made with the official release (4.1) under 5.0.
>>
>> Don
>>
>> On 12/20/13 3:52 PM, Yuan Ho wrote:
>>> Hi All,
>>> I have a new test release here for the progressive
>>> resolution and new image chooser. Please download from the following link:
>>>
>>> http://www.unidata.ucar.edu/downloads/idv/test/index.jsp
>>>
>>>
>>> The changes in the latest:
>>>
>>> 0) the release version is 5.0 for this test release
>>> 1) remove the progressive resolution checkbox from subset panel, it
>>> is now set as the user preference and can be changed in view drop
>>> down menu
>>> 2) the use display area is added as a property of the datasource in
>>> the data source properties panel
>>> 3) the show mag factor is added in the sidelegend
>>> 4) most of previous bugs should be fixed.
>>>
>>> The design should be very close to be the final. My plan is to merge
>>> this to the nightly release after the new year, and run more test
>>> and prepare for the AMS short course with this version of IDV.
>>>
>>>
>>> Yuan
>> --
>> Don Murray
>> NOAA/ESRL/PSD and CIRES
>> 303-497-3596
>> http://www.esrl.noaa.gov/psd/people/don.murray/
>>
>> _______________________________________________
>> idvdevelopers mailing list
>> idvdevelopers@xxxxxxxxxxxxxxxx
>> For list information, to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>>
> --
> Don Murray
> NOAA/ESRL/PSD and CU-CIRES
> 303-497-3596
> http://www.esrl.noaa.gov/psd/people/don.murray/
>
> _______________________________________________
> idvdevelopers mailing list
> idvdevelopers@xxxxxxxxxxxxxxxx
> For list information, to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
