Current state of affairs with Foundation on Windows

**compnerd** · March 31, 2019, 6:01pm

Since it seems that there are a bunch of folks interested in Foundation on Windows, I figure that it makes some sense to disseminate some information around the current state of Foundation on Windows.

Foundation is possible to currently build on Windows in Debug and RelWithDebInfo configurations. Release mode exposes a bug in the swift compiler (SR-10192 [1]) which needs to be investigated. A temporary workaround is possible adding -Xlinker -force:multiple to the link invocation.

In order to support Foundation builds on the nightlies, unfortunately, a set of changes are needed to improve the Foundation build (which I had not expected). Most of the work for that has preliminary work completed for it and has patches on GitHub [2], [3], [4], [5]. This should pretty much enable the nightlies to build Foundation properly [6].

I had started improving the implementation of Foundation on Windows such that it could pass the associated test suite, but found that there is a lot of work to be done still. There were a number of issues, including TimeZone support being completely broken [7], bundle loading being broken [8], amongst other smaller issues. There are still many issues left to investigate. One of the problems is that the test suite is a monolithic blob, and many of these test failures will cause the test harness itself to crash! XCTest has been absolved of its involvement in this - there is one known issue left there [9] which is unrelated to the crashes.

There are issues left within libdispatch (needing support for file/socket sources) and Foundation's curl integration to support HTTPServer [10] for the test suite. As an example: the socket handle is currently truncated at both levels. Additionally, the MultiHandle is not correctly being constructed IIRC.

Removing some portion of the tests (~25 or so IIRC), it is possible to get through a full run of the test suite, although there are still failures to look into. However, even then, about 80% or so of the test suite passes on Windows, so its not like the library is not usable as it currently stands (hey, it is used to run XCTest which is needed to run the test suite!). However, there is still a pretty large surface area to cover, so, others are welcome to join the bug hunting :-).

CC: @millenomi @pvieito @abesmon

[1] https://bugs.swift.org/browse/SR-10192
[2] https://github.com/apple/swift-corelibs-foundation/pull/2004
[3] https://github.com/apple/swift-corelibs-xctest/pull/261
[4] https://github.com/apple/swift-llbuild/pull/460
[5] https://github.com/apple/swift-package-manager/pull/2054
[6] https://dev.azure.com/compnerd/windows-swift/_build/results?buildId=838
[7] https://github.com/apple/swift-corelibs-foundation/pull/2017
[8] https://github.com/apple/swift-corelibs-foundation/pull/2020
[9] https://bugs.swift.org/browse/SR-10144
[10] https://github.com/apple/swift-corelibs-foundation/pull/2013

**abesmon** · March 31, 2019, 7:03pm

Am I understand right, that literal $ must be written as $$ relates to [2]? And can you explain some details, why is it fails on azure, and not failing while I build it on win O.o? I'm just curious, because I'm just starting to digging into all these build systems and not much into them for now

**compnerd** · March 31, 2019, 8:05pm

Not sure what you are referring to wrt $ vs $$.

As to why it fails on Azure is because of the path - it finds ld instead of link and tries to use that as the linker which is not going to work. The only way to fix that is to hoist the configuration to the top level which is what the changes are trying to do (which incidentally is what I wanted to do for some time to ease the pain of development).

**abesmon** · March 31, 2019, 8:25pm

thanks for explaining :) I think I got it.

About literal $ must be written as $$ it's that, what was written in azure's log On foundation build step

**compnerd** · March 31, 2019, 8:41pm

Oh, I see. That seems like a generation failure, not sure why that is happening. Probably something is wrong with the CMake invocation.

**abesmon** · March 31, 2019, 9:03pm

Relates to: Windows Nightlies
Strange things here :/

As I can see in code <fts.h> only includes when not DEPLOYMENT_TARGET_WINDOWS, but as error says, it's trying to include. I'm not sure, where from this variable come from. Must I provided it while building Foundation, or is it somehow comes from system. And if so, why it's trying to include on windows?

**pvieito** · March 31, 2019, 9:08pm

Yes, that's the same behavior I am seeing when trying to use Foundation building a simple Swift script. I think we should pass some flag to swiftc to ensure the header is correctly interpreted, but I don't know which flag is needed.

**abesmon** · March 31, 2019, 9:46pm

I was able to workaround the issue (hacky hack). I modified ForSwiftFoundationOnly.h on my machine, to explicitly set the variable, so it's like:

...
#include <CoreFoundation/ForFoundationOnly.h>
#define DEPLOYMENT_TARGET_WINDOWS 1
#if DEPLOYMENT_TARGET_WINDOWS
#define NOMINMAX
#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#else
...

But there is another issue then:

C:\SwiftDev\Dev>swiftc demo.swift -o demo.exe -v
Swift version 5.0-dev (LLVM 553359b5a4, Swift 7cf4c6b1a7)
Target: x86_64-unknown-windows-msvc
"C:\\Library\\Developer\\Toolchains\\unknown-Asserts-development.xctoolchain\\usr\\bin\\swiftc.exe" -frontend -c -primary-file demo.swift -target x86_64-unknown-windows-msvc -disable-objc-interop -color-diagnostics -module-name demo -o "C:\\Users\\abesmon\\AppData\\Local\\Temp\\demo-d38d93.o"
"C:\\Library\\Developer\\Toolchains\\unknown-Asserts-development.xctoolchain\\usr\\bin\\clang++.exe" -target x86_64-unknown-windows-msvc -L "C:\\Library\\Developer\\Toolchains\\unknown-Asserts-development.xctoolchain\\usr\\lib\\swift\\windows/x86_64" "C:\\Library\\Developer\\Toolchains\\unknown-Asserts-development.xctoolchain\\usr\\lib\\swift\\windows\\x86_64\\swiftrt.obj" "C:\\Users\\abesmon\\AppData\\Local\\Temp\\demo-d38d93.o" -v -o demo.exe
compnerd.org clang version 8.0.0 (https://github.com/apple/swift-clang f9c1074e13aab7a469f3e1379b00b86619d8f520) (https://github.com/apple/swift-llvm 553359b5a468a7bb5d4852f70dba364f594211f9) (based on LLVM 8.0.0)
Target: x86_64-unknown-windows-msvc
Thread model: posix
InstalledDir: C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin
 "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX64\\x64\\link.exe" -out:demo.exe -defaultlib:libcmt "-libpath:C:\\Library\\Developer\\Toolchains\\unknown-Asserts-development.xctoolchain\\usr\\lib\\swift\\windows/x86_64" -nologo "C:\\Library\\Developer\\Toolchains\\unknown-Asserts-development.xctoolchain\\usr\\lib\\swift\\windows\\x86_64\\swiftrt.obj" "C:\\Users\\abesmon\\AppData\\Local\\Temp\\demo-d38d93.o"
LINK : fatal error LNK1104: эх єфрхЄё  юЄъЁ√Є№ Їрщы "Foundation.lib"
clang++: error: linker command failed with exit code 1104 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1104 (use -v to see invocation)

It says, that Foundation.lib can't be opened (I don't know what's up with my coding ) And this is where I don't know where to go and what I can't try to workaround this one

Update
I found it! I found the flag :D
swiftc demo.swift -o demo.exe -Xcc -DDEPLOYMENT_TARGET_WINDOWS -v
Even though, linker error still here :)

**pvieito** · March 31, 2019, 10:33pm

Oh, that solved my problem:

Z:\Downloads\PathTool>PathTool.exe
[ ] Bundle Path: Z:\Downloads\PathTool\PathTool.resources
[ ] Bundle Executable: Z:\Downloads\PathTool\PathTool.exe
[ ] Bundle Identifier: com.pvieito.PathTool

Foundation on Windows!

I think you need to copy Foundation.lib from the build directory to the toolchain at usr/lib/swift/windows/x86_64.

**compnerd** · April 1, 2019, 1:16am

Yes, you are supposed to put -Xcc -DDEPLOYMENT_TARGET_WINDOWS into the compilation (see the patches that I have put up for porting swift-package-manager for Windows).

Foundation.lib needs to be placed into the library search path for the build (again, see the patches for swift-package-manager).

**abesmon** · April 1, 2019, 7:00am

strange that Foundation.lib was not generated after cmake ... --target install. There was .dll, and no .lib . I was confused because in usr/lib/swift/windows/x86_64 there was Foundation.dll, and I can't understand why compiler says Foundation.lib can't be opened. I missed the extension So I got to copy .lib from \b\foundation and here I go. Foundation runs on my Windows

**compnerd** · April 1, 2019, 2:47pm

Oh, interesting. That sounds like a bug in the install setup in CMake. I wasn't aware of it, thanks! That should be easy for me to fix. The cmake ... --target install should install the import library on Windows.

**abesmon** · April 1, 2019, 3:21pm

Also, i was curious: are .exe that was build with swift could be executed on any other Windows machine or output file require swift runtime to be on same machine?

**compnerd** · April 1, 2019, 3:45pm

You do need the swift runtime to be present. However, there is a runtime-only download available as well: https://dev.azure.com/compnerd/_apis/resources/Containers/1730264?itemPath=windows-runtime-amd64.msi%2Fwindows-runtime-amd64.msi . Since it is just the runtime, its relatively small (a few megabytes) compared to the entire toolchain.

That will install just the runtime, which means that you can distribute the runtime MSI with your application and have it run on other targets.