Re: [netcdfgroup] slow reads in 4.4.1.1 vs 4.1.3 for some files

I'd be interested in what you find with binary search
On Dec 13, 2016 1:24 AM, "Simon (Vsevolod) Ilyushchenko" <simonf@xxxxxxxxxx>
wrote:

> Hi,
>
> Two of several .nc file types that I have are read ~60 times slower with
> 'ncdump > /dev/null' in 4.4.1.1 as compared to my older installation of
> 4.1.3 compiled on Linux x86_64 with more or less the same config (the
> configs not identical because there are new settings). The ncdump -d
> outputs are below (the files are from HYCOM and Sentinel 3).
>
> I can do binary search over versions to narrow this down, but wanted to
> ask here first if there is a known issue.
>
> Thanks,
> Simon
>
> HYCOM:
>
> netcdf a {
> dimensions:
>         depth = 2 ;
>         lat = 2001 ;
>         lon = 4500 ;
>         time = UNLIMITED ; // (1 currently)
> variables:
>         double depth(depth) ;
>                 depth:long_name = "Depth" ;
>                 depth:standard_name = "depth" ;
>                 depth:units = "m" ;
>                 depth:positive = "down" ;
>                 depth:axis = "Z" ;
>                 depth:NAVO_code = 5 ;
>         double lat(lat) ;
>                 lat:long_name = "Latitude" ;
>                 lat:standard_name = "latitude" ;
>                 lat:units = "degrees_north" ;
>                 lat:point_spacing = "even" ;
>                 lat:axis = "Y" ;
>                 lat:NAVO_code = 1 ;
>         double lon(lon) ;
>                 lon:long_name = "Longitude" ;
>                 lon:standard_name = "longitude" ;
>                 lon:units = "degrees_east" ;
>                 lon:modulo = "360 degrees" ;
>                 lon:axis = "X" ;
>                 lon:NAVO_code = 2 ;
>         double time(time) ;
>                 time:long_name = "Valid Time" ;
>                 time:units = "hours since 2000-01-01 00:00:00" ;
>                 time:time_origin = "2000-01-01 00:00:00" ;
>                 time:calendar = "gregorian" ;
>                 time:axis = "T" ;
>                 time:NAVO_code = 13 ;
>         short water_u(time, depth, lat, lon) ;
>                 water_u:long_name = "Eastward Water Velocity" ;
>                 water_u:standard_name = "eastward_sea_water_velocity" ;
>                 water_u:units = "m/s" ;
>                 water_u:_FillValue = -30000s ;
>                 water_u:missing_value = -30000s ;
>                 water_u:scale_factor = 0.001f ;
>                 water_u:add_offset = 0.f ;
>                 water_u:NAVO_code = 17 ;
>         short water_v(time, depth, lat, lon) ;
>                 water_v:long_name = "Northward Water Velocity" ;
>                 water_v:standard_name = "northward_sea_water_velocity" ;
>                 water_v:units = "m/s" ;
>                 water_v:_FillValue = -30000s ;
>                 water_v:missing_value = -30000s ;
>                 water_v:scale_factor = 0.001f ;
>                 water_v:add_offset = 0.f ;
>                 water_v:NAVO_code = 18 ;
>
> // global attributes:
>                 :classification_level = "UNCLASSIFIED" ;
>                 :distribution_statement = "Approved for public release.
> Distribution unlimited." ;
>                 :downgrade_date = "not applicable" ;
>                 :classification_authority = "not applicable" ;
>                 :institution = "Naval Oceanographic Office" ;
>                 :source = "HYCOM archive file" ;
>                 :history = "Fri Nov 20 10:01:58 2015: ncks -4 --deflate 9
> -v water_u,water_v -d depth,1,2,1 hycom_glb_911_2015110200_t000_uv3z.nc
> hycom_glb_911_2015110200_t000_uv3z-test.nc\narchv2ncdf3z" ;
>                 :field_type = "instantaneous" ;
>                 :Conventions = "CF-1.0 NAVO_netcdf_v1.0" ;
>                 :NCO = "4.4.2" ;
> }
>
> ***
>
> Sentinel 3:
>
> dimensions:
>         columns = 4865 ;
>         rows = 3682 ;
> variables:
>         ushort Oa20_radiance(rows, columns) ;
>                 Oa20_radiance:_FillValue = -1US ;
>                 Oa20_radiance:add_offset = 0.f ;
>                 Oa20_radiance:ancillary_variables = "Oa20_radiance_err" ;
>                 Oa20_radiance:coordinates = "time_stamp altitude latitude
> longitude" ;
>                 Oa20_radiance:long_name = "TOA radiance for OLCI
> acquisition band Oa20" ;
>                 Oa20_radiance:scale_factor = 0.00326378f ;
>                 Oa20_radiance:standard_name = 
> "toa_upwelling_spectral_radiance"
> ;
>                 Oa20_radiance:units = "mW.m-2.sr-1.nm-1" ;
>                 Oa20_radiance:valid_max = -2US ;
>                 Oa20_radiance:valid_min = 0US ;
>
> // global attributes:
>                 :absolute_orbit_number = 3517U ;
>                 :ac_subsampling_factor = 64US ;
>                 :al_subsampling_factor = 1US ;
>                 :comment = " " ;
>                 :contact = "EOHElp@xxxxxxx" ;
>                 :creation_time = "2016-10-20T13:00:25Z" ;
>                 :institution = "SVL" ;
>                 :netCDF_version = "4.2 of Jul  5 2012 17:07:43 $" ;
>                 :product_name = "S3A_OL_1_EFR____20161020T102624_
> 20161020T102906_20161020T130025_0162_010_079_4319_SVL_O_NR_002.SEN3" ;
>                 :references = "S3IPF PDS 004 - i2r1 - Product Data Format
> Specification - OLCI, S3IPF PDS 002 - i1r6 - Product Data Format
> Specification - Product Structures, S3IPF DPM 002 - i2r1 - Detailed
> Processing Model - OLCI Level 1" ;
>                 :resolution = "[ 270 294 ]" ;
>                 :source = "IPF-OL-1-EO 06.04" ;
>                 :start_time = "2016-10-20T10:26:24.238891Z" ;
>                 :stop_time = "2016-10-20T10:29:06.207560Z" ;
>                 :title = "OLCI Level 1b Product, Radiance Oa20 Data Set" ;
> data:
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: