Now I have made some changes in my timing program, and I think I am getting better (i.e. more realistic) times.
Firstly, I now clear the cache before each read.
Secondly, I don't try and read the horizontal sections and the
timeseries in the same program run - whichever one is done first loads
the cache for the other, and gives unrealistic times. Now I time these
separately.
OK, so here are some timeseries reads times. The first row in netCDF classic data:
cs[0] cs[1] cs[2] cache(MB) deflate shuffle read_time_ser(us)
1 16 32 1.0 0 0 2434393
1 16 128 1.0 0 0 2411127
1 16 256 1.0 0 0 2358892
1 64 32 1.0 0 0 2455963
1 64 128 1.0 0 0 2510818
1 64 256 1.0 0 0 2482509
1 128 32 1.0 0 0 2480481
1 128 128 1.0 0 0 2489436
1 128 256 1.0 0 0 2504924
10 16 32 1.0 0 0 1146593
10 16 128 1.0 0 0 1156650
10 16 256 1.0 0 0 1259026
10 64 32 1.0 0 0 1150427
10 64 128 1.0 0 0 2384334
10 64 256 1.0 0 0 2438587
10 128 32 1.0 0 0 1258380
10 128 128 1.0 0 0 2521213
10 128 256 1.0 0 0 2528927
256 16 32 1.0 0 0 174062
256 16 128 1.0 0 0 358613
256 16 256 1.0 0 0 404662
256 64 32 1.0 0 0 400489
256 64 128 1.0 0 0 688528
256 64 256 1.0 0 0 1267521
256 128 32 1.0 0 0 404422
256 128 128 1.0 0 0 1374661
256 128 256 1.0 0 0 2445647
1024 16 32 1.0 0 0 78718
1024 16 128 1.0 0 0 346506
1024 16 256 1.0 0 0 378813
1024 64 32 1.0 0 0 340703
1024 64 128 1.0 0 0 665649
1024 64 256 1.0 0 0 1269936
1024 128 32 1.0 0 0 380796
1024 128 128 1.0 0 0 1269627
1024 128 256 1.0 0 0 2513330
1560 16 32 1.0 0 0 58124
1560 16 128 1.0 0 0 332641
1560 16 256 1.0 0 0 372587
1560 64 32 1.0 0 0 323445
1560 64 128 1.0 0 0 635165
1560 64 256 1.0 0 0 1263225
1560 128 32 1.0 0 0 372226
1560 128 128 1.0 0 0 1265999
1560 128 256 1.0 0 0 2712887
These numbers make more sense. It takes about 2.3 seconds to read the time series from the classic file.
Ed
