many little files versus one big file

I have about 1700 files totaling about 2 Mbytes of data.  If I load
and process them individually, it takes about 7 seconds.  If I
concatenate them together on the unlimited dimension to form one big
file, it takes about 35 seconds to load and process them.  Is this
expected?  Wouldn't there be less overhead with just one file?  I got
the exact same results in both cases so I don't think did anything
terribly wrong.

Thanks,
Charles




  • 1994 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: