My nc_put_vara_double() call writes the entire array at once; in this
case, it sounds like the library does more or less the same thing as
my algorithm outlined in the original message.
Breaking up the write into pieces would probably be a performance hit,
and would make my code a bit more complicated, since deciding if
splitting is necessary and how many splits to make is highly system
dependent.
I guess I will have to live with being a memory hog -- if I document
this feature, then it becomes the user's problem, not mine. But it
seems to me that it is still better to let the library do the type
conversion. Will the library return a meaningful error if it is unable
to allocate the required memory? Have I overlooked any other costs/
benefits?
--Jennifer
On Nov 29, 2010, at 10:51 AM, Rob Ross wrote:
So you will want to split up the writes into multiple writes to keep
memory allocation down. Perhaps 1GB at a time, given what you
describe below. Still a lot easier than doing the conversion by hand!
Rob
On Nov 29, 2010, at 9:45 AM, Ed Hartnett wrote:
Jennifer Adams <jma@xxxxxxxxxxxxx> writes:
That's great! Does the library use a chunk of memory to do this?
If my
array of doubles is very large, say 20Gb, and I'm running on a
system with
24GB of memory, will it work?
--Jennifer
On Nov 29, 2010, at 9:32 AM, Jim Edwards wrote:
Memory will be allocated, but only enough to convert data for each
nc_put_vara call. So if you are writing 1 MB of values at a time, it
will allocated 1 MB at a time to convert, and free that 1 MB after
the
write operation is complete.
So it doesn't matter how large your variables are, only how much
you try
and write at one time.
Thanks,
Ed
--
Ed Hartnett -- ed@xxxxxxxxxxxxxxxx
_______________________________________________
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma@xxxxxxxxxxxxx
