Hi John Sheldon, hi netcdfgroup,
thanks for the comments and corrections. I actually encountered the Problem 2
(see appended mail at the end of this mail) right
after I submitted the patch. The tests I did before looked alright ...
I am sorry that I didn't announce the problem in the mailing list but
since there was no reaction on my posting I assumed it to be
unimportant for others.
I am sorry about that simple mistake. Naturally it was luck that it
worked with the first tests. A character byte containing a space
character is not a correctly terminated string and so the assignment
to the string pointer had to result in "wrong" nonsense strings.
Your correction is simpler than my first version so I add it at the
end of the mail as a patch for the netCDF group members.
Problem 1 is -- I assume too -- a problem for Cray.
To the importance of the patch:
I did a lot of testing especially on Cray file systems with concated
partitions and found that twiggling the switches can yield remarkable
speed gains (for a test programm writing an netCDF-array with one
statement i.e. most likely sequential). (-p and -q options of
assign). The tests aren't finished yet but if you like I can send you
the results.
Thanks for the correction and good luck
Olaf R. Heudecker
Attached:
- original mail from John Sheldon
- new corrected patch (not extensivly tested!)
----------
On Tue, 1998-4-14 11:56:09 -0400, <jps@xxxxxxxx>
John Sheldon wrote:
> Hi-
>
> We've been trying to apply Olaf Heudecker's "cray-patch" to netCDF 3.4
> on our IEEE T90 (uname -a = "sn7203 t90 9.1.0.1 gfd.11 CRAY TS"), but
> have encountered two problems during "make test":
>
> 1. With NETCDF_FFIOSPEC set to "cachea:112:7", the following cmp fails:
> cmp testfile.nc testfile_nc.sav
> testfile.nc testfile_nc.sav differ: char 391, line 3
> Make: "cmp testfile.nc testfile_nc.sav": Error code 1
> If NETCDF_FFIOSPEC is set to "cache:112:7", this test passes. Octal
> dumps of the two files show 2 bytes to be different in what appears
> to be the header part of the file; ncdump-ing the files show no
> difference. This may not be a problem in practice, but it did not
> occur with the basic 3.4 distribution.
>
> 2. In a later test, with NETCDF_XFFIOSPEC *not* set to anything, we get:
> *** Testing nc_get_var1_text ...
> lib-1503 ./nc_test: UNRECOVERABLE library error
> Attributes in the assign environment file have an unreadable format.
> Make: "./nc_test" terminated due to signal 6 (core dumped)
> If we set NETCDF_XFFIOSPEC to nothing ("setenv NETCDF_XFFIOSPEC"),
> the test passes. We think that the problem is that "xtra_assign"
> really points to a single character, not a null-terminated string as
> it should in this case. Although the resulting string "assignstr"
> seems fine (as does the ".assign" file), the fact that "xtra_assign"
> is not null-terminated somehow clobbers another area of memory when
> it is used, resulting in an eventual core dump. The fix is to change
> those two lines to:
>
> char emptystr='\0';
>
> Now "xtra_assign" will point to a null-terminated string when getenv
> returns NULL, although it's only one character long. (Thanks to Hans
> Vahlenkamp for figuring this out.)
>
> Problem #2 seems solved, but problem #1 is probably something for Cray
> to look at. In any case, we thought that you would want to know...
>
> John P. Sheldon
> (jps@xxxxxxxx)
> Geophysical Fluid Dynamics Laboratory/NOAA
> Princeton University/Forrestal Campus/Rte. 1
> P.O. Box 308
> Princeton, NJ, USA 08542
> http://www.gfdl.gov
>
> (609) 987-5053
> --
> "We are Microsoft. Resistance Is Futile. You Will Be Assimilated."
> --
---------- patch for ffio.c to include NETCDF_XFFIOSPEC option
-- for more explanation netCDF-group Date: Tue, 17 Mar 1998
-- or email oheudeck@xxxxxxxxxxxxxxxxxx
---- patch follows: apply with patch ffio.c < patchfile
------------ start of patchfile -----------
*** ffio.c.save Wed Apr 15 11:49:54 1998
--- ffio.c Wed Apr 15 11:49:54 1998
***************
*** 3,8 ****
--- 3,10 ----
* See netcdf/COPYRIGHT file for copying and redistribution conditions.
*/
/* $Id: ffio.c,v 1.48 1998/03/07 06:46:06 davis Exp $ */
+ /* addition by O. Heudecker, AWI-Bremerhaven, 12.3.1998 */
+ /* added correction by John Sheldon and Hans Vahlenkamp 15.4.1998*/
#include "ncconfig.h"
#include <assert.h>
***************
*** 16,21 ****
--- 18,25 ----
#include <ffio.h>
#include <unistd.h>
#include <string.h>
+ /* Insertion by O. R. Heudecker, AWI-Bremerhaven 12.3.98 (1 line)*/
+ #include <fortran.h>
#include "ncio.h"
#include "fbits.h"
***************
*** 477,487 ****
{
ControlString="bufa:336:2";
}
!
if(fIsSet(ioflags, NC_NOCLOBBER))
fSet(oflags, O_EXCL);
! fd = ffopens(path, oflags, 0666, 0, &stat, ControlString);
if(fd < 0)
{
status = errno;
--- 481,515 ----
{
ControlString="bufa:336:2";
}
! /* Insertion by Olaf Heudecker, AWI-Bremerhaven, 12.8.1998
! to allow more versatile FFIO-assigns */
! /* begin new */
! {
! char assignstr[128];
! char emptystr='\0';
! char *xtra_assign;
! xtra_assign = getenv("NETCDF_XFFIOSPEC");
! if(xtra_assign == NULL)
! {
! xtra_assign=&emptystr;
! }
! if (strlen(ControlString)+strlen(xtra_assign)+21 > 127) {
! /* Error: AssignCommand too long */
! status=E2BIG;
! goto unwind_new;
! }
! sprintf(assignstr,"assign -F %s %s f:%s",ControlString,
! xtra_assign,path);
! /* printf("1 assignstr=>%s<\n",assignstr); */
! /* Aufruf der Fortran-Routine assign von C aus */
! ASSIGN(_cptofcd(assignstr,strlen(assignstr)));
! }
! /* end new */
if(fIsSet(ioflags, NC_NOCLOBBER))
fSet(oflags, O_EXCL);
! /* Orig: fd = ffopens(path, oflags, 0666, 0, &stat, ControlString); */
! fd = ffopen(path, oflags, 0666, 0, &stat);
if(fd < 0)
{
status = errno;
***************
*** 559,566 ****
{
ControlString="bufa:336:2";
}
! fd = ffopens(path, oflags, 0, 0, &stat, ControlString);
if(fd < 0)
{
status = errno;
--- 587,620 ----
{
ControlString="bufa:336:2";
}
+ /* Insertion by Olaf Heudecker, AWI-Bremerhaven, 12.8.1998
+ to allow more versatile FFIO-assigns */
+ /* begin new */
+ {
+ char assignstr[128];
+ char emptystr='\0';
+ char *xtra_assign;
+ xtra_assign = getenv("NETCDF_XFFIOSPEC");
+ if(xtra_assign == NULL)
+ {
+ xtra_assign=&emptystr;
+ }
+ if (strlen(ControlString)+strlen(xtra_assign)+21 > 127) {
+ /* Error: AssignCommand too long */
+ status=E2BIG;
+ goto unwind_new;
+ }
+ sprintf(assignstr,"assign -F %s %s f:%s",ControlString,
+ xtra_assign,path);
+ /* printf("2 assignstr=>%s<\n",assignstr); */
+ /* Aufruf der Fortran-Routine assign von C aus */
+ ASSIGN(_cptofcd(assignstr,strlen(assignstr)));
+ }
+ /* end new */
! /* Orig: fd = ffopens(path, oflags, 0, 0, &stat, ControlString); */
! fd = ffopen(path, oflags, 0, 0, &stat);
!
if(fd < 0)
{
status = errno;
------------------ snip -------- end of patch file
