Hello,
I've just converted some of my netCDF writing code to write/read
multiple groups rather than use an unlimited dimension. I did this so
each group can have different "base" dimensions for the data arrays.
I have one data set where the unlimited dimension is 5000. The
read/write of this data in netCDF3 format is almost instantaneous. When
I use the netCDF4 approach (reading and writing 5000 separate groups)
the reads and write can take upwards of 10minutes (I started the program
at 10:33am. It is now 10:51am and the read of the created file is still
going on).
I realise there's going to be additional overhead using the "groups"
approach (defining dimensions and variables for each group) but I
presume I'm doing something very wrong/stupid to cause the I/O to be as
slow as it is. Before I start posting code snippets, does anyone have
any experience hints as to what could be causing this supa slow I/O?
Thanks for any info.
cheers,
paulv
p.s. It's now 11:00am and the dataset reading is still going on...
