Building libDispatch on Linux crashes

Hey all-

Okay, I'm at Witt's End and am making a request for some help. I am trying to build Swift 5.2 on Fedora Rawhide that has Clang 10 and CMake 3.17.0-rc2 as of this writing. It fails to build libDispatch because swiftc is passed with -output-file-map src/swift/CMakeFiles/swiftDispatch.dir/output-file-map.json which contains as the first entry:

"" :
         {
                 "swift-dependencies" : "src/swift/CMakeFiles/swiftDispatch.dir/Release/swiftDispatch.swiftdeps"
         },

and what I've discovered is that the "Release" directory, along with swiftDispatch.swiftdeps does not exist, and the swiftc then crashes because it's trying to open the swiftDispatch.swiftdeps file in a non-existent directory.

I compared this against a version of Fedora with Clang 10 but CMake 3.16 and "Release" is not added to the directory and as a consequence swiftDispatch.swiftdeps is created properly, and everything continues on fine.

I have been trying to find where output-file-map.json in libDispatch is created and I just can't seem to pin it down; lots of projects make use of similarly-named files and while it's clearly generated, there doesn't seem to be any code I can grep that points to it being created at all, let alone trying to determine why it's erroneously putting the non-existent "Release" directory entry in the file.

Any info on getting to the bottom of this would be appreciated; I found and reported a CMake bug that was causing Swift to fail to build before and perhaps there's a new one that needs to be fixed before everyone moves to 1.17.

Thanks,

Ron

I think that may be a recent regression due to the multi-configuration support that is being worked on in CMake. I would recommend that you test with 3.16 and see if that is newly introduced. That should be an easy thing to fix in CMake if it is a new regression.

I can confirm it does work under 3.16. One thing to note is that I'm using the installed CMake packages, not the one that is included when building from master, so I'm guessing there's a difference in implementation;

Oh, wait, I found this CMake commit which suggests that the code to handle this is actually in the CMake executable, not within the swift source, which I suppose is what you were referring to.

I tracked it down to this line; "config" is set to "Release" and if removed, Swift builds correctly all the way through.

Thats the correct area, though I believe that the change should be to the path construction for the output-file-map about 4 lines above (it should consider config). This definitely sounds related to the multi-configuration ninja build support that Kyle has been working on.

the path to the actual output-file-map.json file? If so, it does not go into the "Release" directory (and no Release directory is created, thus causing swiftc to crash).

Right, but it should go into the release directory because the file list can be dependent on the configuration.

When I saw "Release" I immediately saw the benefit; it does seem like a good idea, just seems to be an implementation issue. Is there anything I can do to help?

Feel free to put up a MR to adjust for that along with a test. I don't think I will be able to get to fixing that until tomorrow.

Hmm, the multi-configuration requires a completely separate generate (Ninja Multi-Config). How are you configuring dispatch?

It's in the swift/util/build-settings-impl file I believe; it's just part of the full Swift build.

That makes it far too difficult to reproduce :). That doesn’t run on Windows and my Linux setup currently has a broken libc.

AFAICT, the build is fine - I just built libdispatch with ToT cmake on Windows. Perhaps it’s an issue with build-script?

Oof, tell me about it; on my server it takes a minimum of an hour just to get to the spot where I can see the results of my changes. :slight_smile:

I'm not sure about the issue with the build-script; if I check out the 3.16 version, recompile it and rerun the build, Swift builds fine, all the way through. If I remove config from the file path when writing the output.json file, it works fine too, with 3.17, straight from master. I also put in some additional debug output and the same issue would have happened with Foundation and FoundationXML, in that they too would have had 'Release' in the filename and subsequently would have failed.

I'm currently trying to track down where config is getting set in CMake to Release; it's not set all the time, just when the json file appears to being built.

BTW, the change was introduced in commit 6e65b869c38c97eed1dafb6269b0242d6b571615, but the commit message is somewhat vague,

Ninja strcat usage