I have recently started working with the IDV, and have a question about
performance and memory usage.
Although new to the IDV, I have worked for many years with OpenDX (and IBM
Data Explorer before it was open sourced). I figure I must be doing
something terribly wrong with the IDV, since it runs so slowly, and uses so
much memory, for a very simple visualization.
Thank you in advance for any help, or pointers to info, you might offer.
Here is my simple test visualization:
----------------------------------------
THE DATA
272 stations, each located at a (latitude, longitude).
Each station produces data consisting of 153 integer values, measured at
uniform time intervals, from a start time to a stop time.
The start time, stop time, and time interval are identical for all
stations.
I provide this data to IDV as an ASCII CSV file of the following format.
Now, I do realize that this format does not tell the IDV anything about the
relationship among data elements, which it could probably utilize to be
more efficient, but this was the simplest way to initially get the data in
(and the amount data is not large).
CSV FORMAT:
(index) -> (Time,Latitude,Longitude,Magnitude)
Time[fmt="yyyy-MM-dd
HH:mm:ss"],Latitude[unit="deg"],Longitude[unit="deg"],Magnitude[unit="null"]
2006-11-15 11:33:16, 32.888901, -117.105103, 85 // Start of 1st station's
data values
2006-11-15 11:33:26, 32.888901, -117.105103, 55
[another 151 entries with the same Latitude and Longitude,
at uniform time steps, with different Magnitudes]
2006-11-15 11:33:16, 48.719700, -122.707001, 802 // Start of 2nd station's
data values
2006-11-15 11:33:26, 48.719700, -122.707001, 580
[another 151 entries with the same Latitude and Longitude, at
the same uniform time steps as the first station, with different Magnitudes]
... and so on for all 272 stations.
----------------------------------------
THE VISUALIZATION
I plot these 272 stations, each as a small glyph (filled circle) on a 2D
map. I have 3D Display unchecked in preferences. The glyph color is
determined by a color map, mapping Magnitude into color. As I step through
the time steps, the set of glyphs are colored according to the Magnitude at
each station at that particular time.
----------------------------------------
THE PERFORMANCE
Running on Linux (Fedora Core 3), on a 3.2 GHz P4 with 1GB memory
IDV 2.2
java version "1.5.0_11" (the one that is packaged with IDV 2.2)
java set to start with 750 MB.
It takes over 2 minutes to just create the first display of all stations,
and uses 590 MB of memory. If I use a simple square glyph instead of a
round one, it uses 400 MB. A simple + glyph uses 370 MB.
Once the display is created, stepping through the time steps is fast.
If I change the color map, it will take many more minutes to finish the
display, and will nearly double the memory usage. It does not seem to
matter if I disable or enable caching.
Can anyone offer any suggestions on what to do?
Thanks again,
Mike
Michael Zeleznik
zeleznik@xxxxxxxxxxxxxxx
801-485-1106
