Re: [netcdfgroup] [PossibleSpam][3.0] Re: [PossibleSpam][3.6] Re: netcdf fortran error - HDF5: infinite loop closing library

  • To: Ward Fisher <wfisher@xxxxxxxx>
  • Subject: Re: [netcdfgroup] [PossibleSpam][3.0] Re: [PossibleSpam][3.6] Re: netcdf fortran error - HDF5: infinite loop closing library
  • From: Richard Weed <rweed@xxxxxxxxxxxxxxxx>
  • Date: Wed, 09 Mar 2016 16:16:12 -0600
Ward

 Also, if you change the internal write format to

write(unit=myattr,fmt='(a6,i0)') 'myattr',i
(replace i4.4 with i0)

such that the myattr string will look like

 myattr1
 myattr2
   .
   .
 myattr10
 myattr30

etc. ie the zeros (0001,0030 etc) are removed from the i4.4 descriptor
values and the spaces removed if you use the i4 descriptor values the
error goes away. Good luck with this one cause I haven't
a clue as to what it could be. Never seen a case like this
where filling in space in a string with 0's caused something
to gag.

RW


On 03/09/2016 01:59 PM, Ward Fisher wrote:
    My initial response went to Alison only, by accident; my apologies. Iâve
    been able to observe the issue in a C program provided by Alison, so that
    rules out a Fortran-specific cause.

Iâve opened a ticket on GitHub for this issue, viewable at
https://github.com/Unidata/netcdf-c/issues/233 . Searching the web for |HDF5:
infinite loop closing library| turns up a number of other instances of this
happening. Interestingly enough, when building a version of hdf5 1.8.16 for
debugging, the issue itself disappeared. Iâm going to try to narrow down which
of the configure options (|--enable-codestack|, |--enable-debug=all|,
|--enable-using-memchecker|) resulted in the issue resolving itself.

Iâll be posting my findings at the GitHub issue linked above as well as
following up here once thereâs a resolution, but the short answer (as has
already been pointed out) is that Alison isnât doing anything wrong in the test
(although using |redef| isnât a bad idea).

-Ward

On Tue, Mar 8, 2016 at 9:57 PM, rweed <rweed@xxxxxxxxxxxxxxxx
<mailto:rweed@xxxxxxxxxxxxxxxx>> wrote:

    Alison

       If you change

    write(unit=myattr,fmt='(a6,i4.4)') 'myattr',i

       to

    write(unit=myattr,fmt='(a6,i4)') 'myattr',i

    the error disappears

    after the write with your format, myattr
    looks something like myattr0001 etc
    If you replace i4.4 with i4 myattr looks
    like myattr   1 (spaces instead of 0's

    I'm not sure why this would make a difference but it does.

    RW


    On 03/08/2016 02:19 PM, Alison Walker wrote:


        Roy, thanks for your response. I tried adding the call to nf90_redef
        after the open, as you suggest, and then re-write the attributes, but I
        still get the same error.
        Alison

        On 3/7/2016 9:36 PM, Roy Mendelssohn - NOAA Federal wrote:

            Hi Alison:

            I am not certain of this,  but nc_create and nc_open behave
            differently.  nc_create creates the file and puts you in define 
mode,
            nc_open does not put you in define mode.  I donât know what is magic
            about 30 attributes, perhaps that is what it takes to exceed the
            global attributes space because the history changes, but I wonder 
what
            happens if after the nc_open you call NF90_REDEF and then rewrite 
the
            global attributes.

            As I said, I could be wrong, but it would be easy enough to test.

            -Roy




                On Mar 7, 2016, at 7:01 PM, Alison Walker
                <postings@xxxxxxxxxxxxxxxxxxxxxx
                <mailto:postings@xxxxxxxxxxxxxxxxxxxxxx>> wrote:

                Hi netcdf folks,
                I'm getting this error when changing the values of global 
attributes
                in a netcdf4 file using the fortran interface:

                HDF5: infinite loop closing library
                
D,G,A,S,T,F,FD,P,FD,P,FD,P,E,E,SL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL


                Here is a little test program that illustrates the problem. It 
first
                creates a netcdf file and writes some global attributes, then 
tries
                to write them again after closing and re-opening the file.
                Interestingly, if the number of attributes is less than 30, 
there is
                no error. If the number is 30, the error occurs. I'm using 
netcdf
                4.3.0, netcdf fortran 4.2 and hdf5 1.8.16 on Linux x86_64.

                program tester

                   use netcdf
                   implicit none

                   integer, parameter :: n_attrs = 30

                   integer :: iok, ncid, varid, i
                   character*10 :: myattr

                   ! Create the file and write global attrs

                   iok = nf90_create('tester.nc 
<http://tester.nc>',NF90_HDF5,ncid)
                   print *,'create iok=',iok
                   do i=1,n_attrs
                      write(unit=myattr,fmt='(a6,i4.4)') 'myattr',i
                      iok = nf90_put_att(ncid,NF90_GLOBAL,myattr,i)
                      print *,i,' put iok=',iok
                   enddo
                   iok = nf90_close(ncid)
                   print *,'close iok=',iok

                   ! Open the file again and re-write global attrs

                   iok = nf90_open('tester.nc 
<http://tester.nc>',NF90_WRITE,ncid)
                   print *,'open iok=',iok
                   do i=1,n_attrs
                      write(unit=myattr,fmt='(a6,i4.4)') 'myattr',i
                      iok = nf90_put_att(ncid,NF90_GLOBAL,myattr,i)
                      print *,i,' put iok=',iok
                   enddo
                   iok = nf90_close(ncid)
                   print *,'close iok=',iok

                end program tester

                When I run this test program with 30 attributes I get this error
                result:

                create iok=           0
                            1  put iok=           0
                            2  put iok=           0
                            3  put iok=           0
                            4  put iok=           0
                            5  put iok=           0
                            6  put iok=           0
                            7  put iok=           0
                            8  put iok=           0
                            9  put iok=           0
                           10  put iok=           0
                           11  put iok=           0
                           12  put iok=           0
                           13  put iok=           0
                           14  put iok=           0
                           15  put iok=           0
                           16  put iok=           0
                           17  put iok=           0
                           18  put iok=           0
                           19  put iok=           0
                           20  put iok=           0
                           21  put iok=           0
                           22  put iok=           0
                           23  put iok=           0
                           24  put iok=           0
                           25  put iok=           0
                           26  put iok=           0
                           27  put iok=           0
                           28  put iok=           0
                           29  put iok=           0
                           30  put iok=           0
                close iok=           0
                open iok=           0
                            1  put iok=           0
                            2  put iok=           0
                            3  put iok=           0
                            4  put iok=           0
                            5  put iok=           0
                            6  put iok=           0
                            7  put iok=           0
                            8  put iok=           0
                            9  put iok=           0
                           10  put iok=           0
                           11  put iok=           0
                           12  put iok=           0
                           13  put iok=           0
                           14  put iok=           0
                           15  put iok=           0
                           16  put iok=           0
                           17  put iok=           0
                           18  put iok=           0
                           19  put iok=           0
                           20  put iok=           0
                           21  put iok=           0
                           22  put iok=           0
                           23  put iok=           0
                           24  put iok=           0
                           25  put iok=           0
                           26  put iok=           0
                           27  put iok=           0
                           28  put iok=           0
                           29  put iok=           0
                           30  put iok=           0
                close iok=           0
                HDF5: infinite loop closing library
                
D,G,A,S,T,F,FD,P,FD,P,FD,P,E,E,SL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL


                Can anyone help me to understand why this error is occurring 
when
                re-writing the attributes?is it a bug with netcdf/hdf or am I 
doing
                something wrong?
                Thanks,
                Alison

                _______________________________________________
                netcdfgroup mailing list
                netcdfgroup@xxxxxxxxxxxxxxxx 
<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
                For list information or to unsubscribe,  visit:
                http://www.unidata.ucar.edu/mailing_lists/

            **********************
            "The contents of this message do not reflect any position of the 
U.S.
            Government or NOAA."
            **********************
            Roy Mendelssohn
            Supervisory Operations Research Analyst
            NOAA/NMFS
            Environmental Research Division
            Southwest Fisheries Science Center
            ***Note new address and phone***
            110 Shaffer Road
            Santa Cruz, CA 95060
            Phone: (831)-420-3666 <tel:%28831%29-420-3666>
            Fax: (831) 420-3980 <tel:%28831%29%20420-3980>
            e-mail: Roy.Mendelssohn@xxxxxxxx <mailto:Roy.Mendelssohn@xxxxxxxx>
            www: http://www.pfeg.noaa.gov/

            "Old age and treachery will overcome youth and skill."
            "From those who have been given much, much will be expected"
            "the arc of the moral universe is long, but it bends toward justice"
            -MLK Jr.



        _______________________________________________
        netcdfgroup mailing list
        netcdfgroup@xxxxxxxxxxxxxxxx <mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
        For list information or to unsubscribe,  visit:
        http://www.unidata.ucar.edu/mailing_lists/


    --
    --
    ---

    Richard Weed, Ph.D.
    Associate Research Professor
    Center for Advanced Vehicular Systems (CAVS)
    Mississippi State University

    Email: rweed@xxxxxxxxxxxxxxxx <mailto:rweed@xxxxxxxxxxxxxxxx>
    Ph: (662) 325-5450 <tel:%28662%29%20325-5450>


    _______________________________________________
    netcdfgroup mailing list
    netcdfgroup@xxxxxxxxxxxxxxxx <mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
    For list information or to unsubscribe,  visit:
    http://www.unidata.ucar.edu/mailing_lists/

â

--
---
-----

Richard Weed, Ph.D.
Associate Research Professor
Center for Advanced Vehicular Systems (CAVS)
Mississippi State University

email: rweed@xxxxxxxxxxxxxxxx
Phone: (662) 325-5450