[netcdfgroup] Possible memory leak in nf90_get_var (netcdf/4.3.0)

Hi

I've run into a problem that is possibly a memory leak in the netCDF library 
(netcdf/4.3.0). When I use nf90_get_var to read from a series of 
netCDFarchives, memory leaks (andeventually uses up all RAM), even though every 
netCDF archive is closed before the next one is opened (chunk cache is not the 
cause).

I attached ademonstration code (memory_leak_demo.tgz). It can be compiled by 
running the compile.sh script, after setting the FC (Fortran compiler) and 
NETCDF (path to netCDF library) environment variables. The demonstration code 
has two executables:

memory_leak_demo_1: Creates one netCDF archivewith data, and reads the data 
from the archive once.

memory_leak_demo_2 : Creates ten netCDF archives with data, and reads the data 
from each of the ten archives once.

I attached PBS output logs for the two executables (compiled with intel/14.0.2):

MEMORY_LEAK_DEMO_1_THEIA.sh.o5600419
MEMORY_LEAK_DEMO_2_THEIA.sh.o5600420

Comparing these, memory_leak_demo_2 uses about 5 times more memory than 
memory_leak_demo_1. I suspect this is a memory leak(unless I am missing 
something crucial).

Jan

Attachment: memory_leak_demo.tgz
Description: application/compressed-tar

Start Prologue v3.0-ad-2014.12.29 Mon Dec  7 21:01:39 UTC 2015 on Mother 
Superior(head node): t1041
Node Access Status = SINGLEJOB
Job 5600419.bqs3 started for user Jan.Kazil in queue debug
======= Running Sanity Checks =======
----------------
t1041
----------------
PASS!
=======  Ended Sanity Checks  =======
End Prologue v3.0-ad-2014.12.29 Mon Dec  7 21:01:41 UTC 2015 
_______________________________________________________________

Welcome to Theia
  This system is Cray CS400 Linux Cluster

Update History:

  Version: v3.0-ad-2014.12.29
    -- Initial install on Cray systems

type:
  module avail
  for various Available Modules

_______________________________________________________
 Writing netCDF archive test.nc                         
 
 Done writing.
 
 Reading netCDF archive test.nc                         
 
 Done reading.
 
_______________________________________________________________
Start Epilogue v3.0-ad-2014.12.29 Mon Dec  7 21:01:43 UTC 2015 
=====================================
Peak memory usage summary:
  min =     572776  KB
  ave =     572776  KB
  max =     572776  KB

All nodes sorted by nodelist order: (memory usage in KB)
          % of      user        user          user         total       total
Node      limit     max         limit        current      current      phys
t1041      0.9      572776     63719424       291536      1738412     65997864 
=====================================
Job 5600419.bqs3 finished for user Jan.Kazil in queue debug with exit code 0
Node Access Status = SINGLEJOB
Statistics  cput=00:00:00,energy_used=0,mem=0kb,vmem=0kb,walltime=00:00:04
End Epilogue v3.0-ad-2014.12.29 Mon Dec  7 21:01:44 UTC 2015
_______________________________________________________________
Start Prologue v3.0-ad-2014.12.29 Mon Dec  7 21:01:39 UTC 2015 on Mother 
Superior(head node): t1054
Node Access Status = SINGLEJOB
Job 5600420.bqs3 started for user Jan.Kazil in queue debug
======= Running Sanity Checks =======
----------------
t1054
----------------
PASS!
=======  Ended Sanity Checks  =======
End Prologue v3.0-ad-2014.12.29 Mon Dec  7 21:01:41 UTC 2015 
_______________________________________________________________

Welcome to Theia
  This system is Cray CS400 Linux Cluster

Update History:

  Version: v3.0-ad-2014.12.29
    -- Initial install on Cray systems

type:
  module avail
  for various Available Modules

_______________________________________________________
 Writing netCDF archive test_0.nc                       
 Writing netCDF archive test_1.nc                       
 Writing netCDF archive test_2.nc                       
 Writing netCDF archive test_3.nc                       
 Writing netCDF archive test_4.nc                       
 Writing netCDF archive test_5.nc                       
 Writing netCDF archive test_6.nc                       
 Writing netCDF archive test_7.nc                       
 Writing netCDF archive test_8.nc                       
 Writing netCDF archive test_9.nc                       
 
 Done writing.
 
 Reading netCDF archive test_0.nc                       
 Reading netCDF archive test_1.nc                       
 Reading netCDF archive test_2.nc                       
 Reading netCDF archive test_3.nc                       
 Reading netCDF archive test_4.nc                       
 Reading netCDF archive test_5.nc                       
 Reading netCDF archive test_6.nc                       
 Reading netCDF archive test_7.nc                       
 Reading netCDF archive test_8.nc                       
 Reading netCDF archive test_9.nc                       
 
 Done reading.
 
_______________________________________________________________
Start Epilogue v3.0-ad-2014.12.29 Mon Dec  7 21:01:56 UTC 2015 
=====================================
Peak memory usage summary:
  min =    3067060  KB
  ave =    3067060  KB
  max =    3067060  KB

All nodes sorted by nodelist order: (memory usage in KB)
          % of      user        user          user         total       total
Node      limit     max         limit        current      current      phys
t1054      4.8     3067060     63719424      2791012      4224968     65997864 
=====================================
Job 5600420.bqs3 finished for user Jan.Kazil in queue debug with exit code 0
Node Access Status = SINGLEJOB
Statistics  
cput=00:00:09,energy_used=0,mem=284804kb,vmem=561384kb,walltime=00:00:17
End Epilogue v3.0-ad-2014.12.29 Mon Dec  7 21:01:57 UTC 2015
_______________________________________________________________
begin:vcard
fn:Jan Kazil, PhD
n:Kazil, PhD;Jan
org:National Oceanic and Atmospheric Administration (NOAA);Chemical Sciences 
Division
adr:325 Broadway;;Earth System Research Laboratory, R/CSD2;Boulder;CO;80305;USA
email;internet:jan.kazil@xxxxxxxx
title:CIRES Scientist III
tel;work:+1 303 497 5820
tel;fax:+1 303 497 5340
version:2.1
end:vcard

  • 2015 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: