Sorry, forgot to cc the netcdf group. Who will get this tomorrow, since I'm not a list member :( Dana ---------- Forwarded message ---------- From: Dana Robinson <derobins@xxxxxxxxxxxx> Date: Mon, Jun 10, 2013 at 3:11 AM Subject: Re: [Hdf-forum] [netcdfgroup] Make the Cmake Windows build staticplease ! To: HDF Users Discussion List <hdf-forum@xxxxxxxxxxxxxxxxxx> Hi Pedro, Answers interspersed below... On Sun, Jun 9, 2013 at 11:26 PM, Pedro Vicente <pvicente@xxxxxxx> wrote: > ** > Hi Dana > > My original thread title > > "Make the CMake Windows build static" > > is a bit misleading, sorry. > > HDF5, in the NCO and netCDF cases, is only one of several input libraries . > > You can think of both HDF5 and netCDF in 2 ways > > 1) A library that will be used as *input* for a program (EXE file) that > any HDF/netCDF user wants to build, his/her own program. > In this case he will need your LIB (in case he wants to compile) or the > DLL (if he just wants to run the EXE) > > 2) A final program destination already done (EXE) , like h5dump or ncdump, > that can be either statically built (no DLL) or dynamically built (need DLL) > > For NCO, only case 2) exists. NCO is just the final program, not an input > library to anyone. > We only distribute EXE files, not LIB files, like you do. > > If I am taking the HDF5 binaries you provide, and combine with other > several other library binaries, and even other libraries source code , > in a combination of several LIB, DLL, and EXE files, dynamic is the way > to go, because of the reasons you mention. > > Note: just the binaries. Not the whatever way I have to obtain them, in > this case, a Cmake compiler flag. > > But, on the other hand, if I take all the source code only from these > “libraries”, and combine then into an EXE file statically build, > then I can consider all those “libraries” not libraries any more, just > pieces of source files that will produce an EXE. > > One note, though, and here I would like you to correct me, if I’m wrong . > > I can separate all these individual “libraries”, like HDF5, into an > intermediate input file, a LIB file, > that I use *only* for my compilation into an EXE, *not* to distribute to > anyone. > > Take the simplest case of 2 source files > > a.c that I compile into a.LIB > > main.c that I compile into main.EXE with a link input of a.LIB > > If I compile both of these Visual Studio projects with the same static > flag, then, > there is no DLL dependency, and I think the CRT issues you mention do not > happen. > > > Do you agree with this? > > No. a.lib and main.exe will have separate CRT states. You will not be able to malloc in a.c and free in main.exe without risking heap corruption in your exe. I have attached a helpful set of slides that explains this. Note that what you described will "work" in the sense that the OS won't complain. It's not very well documented, but it appears that the Win32 underpinnings are perfectly happy (this really isn't that surprising since everything belongs to the same process and the Win32 layer doesn't care about what's going on above it). It's just the bookkeeping information in the CRTs that gets messed up and this may or may not manifest itself as an error in your application. At any rate, it's not correct behavior. This is not what happens in Linux and other Unix-like operating systems. The standard C library is a central part of the Linux OS but Windows uses the Win32 API exclusively. The bizarre linking issues on Windows seem to be due to this non-central nature of the library. This is also why you have to install the CRT on Windows, whereas on Linux it's just there. The executive summary is that you should dynamically link ALL libraries that expose CRT objects via the library API. This obviously includes the Microsoft CRTs and, not so obviously, the HDF5 library. Also, just to be clear, it is totally ok to write to memory allocated by a different CRT. That doesn't affect internal CRT heap tracking. It's just unsafe to resize or free the memory unless that CRT provided it. > > When I started building Windows software in the 90’s with the Visual > Studio from that time I was using the DLL option. > > Why? It seemed like a good idea. Dozens of programs done, thousands of > compiler builds, why not just make them share the system DLL? > > Many of this software I kept both the source and the final EXE, stored in > a backup drive. But I forgot to store also the DLL, just because I never > cared about them, > I was not distributing software to anyone, just doing my college things. > > Fast forward 20 years, I want to run the EXE from that program. what > happens? DLL whatever year 1993 is missing, oh my , I forgot to backup it . > And now where can I find it?, nowhere :-) > > The funny thing is that the EXE is still *binary* compatible regarding the > Windows year 1993 version and Windows 7 today. > I just cannot run it because I don’t have the DLL. If I *had* used the > static option, now, today, I could run that EXE. > > > So, that’s all what I was trying to say with my comments, “people, don’t > use DLLs, they are bad” > > Totally different case as compared to distributing HDF5 as a library. > > CMake is only an automatic generator of Visual Studio projects. > > Like it is now, I can go to your generated projects and change the > compiler flags. > > But I thought it would be a valuable feature for the community to have > this choice already built directly into the Cmake step, like Ward did for > netCDF, > and John also did, I think, in the original setup, but you took off the > final distribution. > > Is that right, John? > Because I think the option –enable-static/shared, is just a given thing > with the system, just a matter of adding a one liner to the script ? > > So, regarding your comment > > “Personally, I'm not keen on making it easy for people to statically link > to the CRT since it's a documented bad idea” > > I don’t think you should be limiting our choices. > > I am just asking to let us do your own choices in the way we compile our > software. > > As for supporting static CRT linking in the HDF5 library, the problem we face is support. If we supply it, people will use it, especially when it's the "easy" thing, and then complain when it doesn't work for them. We don't put it in CMake for the same reason you don't store knives with the pointy ends up or allow smoking at the gas station. Given what I've demonstrated in the attached slides, you can probably see why I feel this way. I'm not 100% opposed, I'm just strongly opposed, and I'd want such an option flagged so harshly that people would feel personally insulted when they selected it. It's not enough to just document it, because almost nobody reads documentation. I understand that this sucks. Using standard C on Microsoft platforms has been that way for a very long time. There's no C99 support, no standard place to put libraries (the system directory does not count), you have to rely on the PATH environment variable to find dlls and tools, POSIX support barely exists, etc. AND then you have the CRT linking issue on top of it. It's a complete nightmare, really. Also, if I'm saying anything incorrect here, I would really appreciate being corrected. I'd be especially keen on seeing any authoritative sources that contradict what I've said. I've spent a lot of time digging around for information on this topic and MS doesn't make it easy. Cheers, Dana
Attachment:
CRT State Stuff.pptx
Description: application/vnd.openxmlformats-officedocument.presentationml.presentation
netcdfgroup archives: