Hi Chris and Russ,
I read with interest your discussion on chunking.
I added Chris's suggestion to NCO's supported chunking options.
NCO 4.4.0 now implements six "chunking maps"
http://nco.sf.net/nco.html#cnk
including Chris's that we call the "lfp" map.
When NCO introduced chunking support in 2009 I was under the
impression that the record dimension _had to have_ a chunksize of one.
Maybe I mistook this default (1) as mandatory for the record
dimension. Anyway, now it can be anything and, with lfp, it
automatically expands until the total chunksize for the array matches
the user request (--cnk_byt) in bytes.
We'll work on optimizing chunking maps in NCO 4.4.1. Would certainly
like to include Russ's balanced 1-D/3-D access algorithm. Unfortunately
we lack resources to benchmark access time for the results. So our
goal is instead to give users a flexible menu of per-file,
per-dimension, per-variable, and per-access type chunking options that
they can benchmark. Hopefully something will be learned about optimal
defaults in the process...
Charlie
--
Charlie Zender, Earth System Sci. & Computer Sci.
University of California, Irvine 949-891-2429 )'(
