2012 Unidata NetCDF Workshop > Chunking and Deflating Data with NetCDF-4
19.0 What is Chunking?
Chunked data are written in chunks of user-defined size.
- A chunk is a multi-dimensional rectangle (or "tile"). When a variable is
chunked, each chunk is read or written as a single I/O operation,
and individually passed from stage to stage of the pipeline and
filters.
- Think of how Google Maps accesses map data, as rectangular tiles
rather than thin rows of pixels along latitude lines. To access a subset of the data,
you only read the subset of tiles ("chunks") containing the data you
want, rather than extracting data from long rows
along latitude lines.
- Contiguous layout is the default for fixed sized variables without
compression or other filters.
- Chunking is required for any variable that uses one or more
unlimited dimensions, compression, or other filter (such as check-summing).
- Chunking is set when the variable is created, and is never changed
after that. To change the chunksize you have to copy the data into a
new file.
- Chunking is transparent to the reader (unlike with Google Maps).
2012 Unidata NetCDF Workshop > Chunking and Deflating Data with NetCDF-4