Swift as a cross-platform language and Windows support

Hey @compnerd, hats off to you for driving the Windows support! I'm very interested in getting the entire Swift toolchain running on Windows, and I do see that there is a Jira Epic for Windows Support. Wondering what's the current state of the Swift Package Manager, and are there starter tasks external contributors like me could help with? I've been mostly toying around the various community Cygwin ports for quite some time.

1 Like

I might be able to help answer that.

If cross-compiling's an option, Swift Package Manager works for most basic use cases (including fairly complex package structures) when cross-compiling to Windows; you can set a Windows target in a destination JSON file and it'll happily produce Windows executables provided you have a Windows toolchain in the lib/swift/windows directory of the Swift compiler.

In terms of the immediate work: @compnerd has a few PRs up at the moment that are critical to being able to build working non-trivial binaries for Windows. The most important is this PR which fills in the metadata for non-trivial types.

Once those PRs are merged in, I'd say the most important thing is to make it easy for people to build the toolchain on Windows, comprising of the stdlib for Windows and possibly Dispatch. A very simple Python or Bash script to just run the CMake commands might be a good starting point, at least until more of the main build-script is ported over to Python.

Alternatively, I think @compnerd's done a lot of work to get both CoreFoundation and Dispatch building on Windows, so filling out some of the missing functionality in Foundation would also be useful, especially since Foundation is a dependency for building SwiftPM natively on Windows. I've made a (messy) start on that here.

All of this might be a little out of date, but I'm planning to take a look at the state of all of this over the weekend, so will post updates if anything here is wrong (or someone can correct me).

EDIT: I've just built a cross-compile Windows toolchain from Ubuntu for Windows. The build script for cross-compiling works properly when invoked as per the docs. Unfortunately, the VWT patch as it stands breaks the runtime in a few ways on all platforms; I ended up with corrupt binaries when I built with it for Windows. Once that's fixed and merged in, however, it should be on to the test suite, Dispatch, LLDB, and Foundation.

4 Likes

Okay, so most of the changes that are needed are available in some form. @Slava_Pestov wants to refactor some stuff for apple/swift#20233. With that merged, I think that non-trivial programs should be possible to write. There are some minor issues with the libdispatch build when it comes to cross-compiling, but nothing insurmountable. I have gotten a libdispatch "hello world" program to run successfully on Windows with the work that is on GitHub. At this point, outside of enumerating and applying the patch set, I would say that @Torust is correct that the barrier to entry may come down to automation of the build.

@Torust, I would be interested in knowing what breaks with the VWT patch, as that seems to be working for me.

1 Like

I built for Windows x64 with the patch merged in as it stood a week ago. I could run the produced executable, and in debug mode it runs correctly, but a build compiled with optimisations crashes partway through execution, seemingly due to heap memory corruption. I tried debugging with print statements inserted within the stdlib; doing so broke the debug build as well, with the print statements showing up to a certain point and then the program crashing.

More specifically, it seemed to fail during a call to Builtin.allocWithTailElems_1 from within _ContiguousArrayBuffer(_uninitializedCount:minimumCapacity). Recompiling the stdlib with slight changes alters where the crash appears - sometimes both -Onone and -O builds work, while sometimes neither work. I'm wondering if the metadata for the Element generic parameter is corrupted somehow.

I started debugging further but then noticed the test suite was failing with the patch even on Linux, so I left it for until that issue was fixed.

Would you be able to provide some demo code to illustrate the issue? I think that if there are issues like that, we need to get those sorted out sooner rather than later. Please use the current master for the builds, as there is a bunch of work that Im still pushing through.

I've been trying to keep a log of the things that are not working. I think that at this point, the possibilities for starter tasks are starting to open up. The current state of the tree is enough where you should be able to build for Windows and get something that works.

The next big thing is going to be getting the test suite passing. I've started porting the support libraries to Windows. But there is still a slew of tests which could be made to work without that work. There were ~612 test failures on my last run. I would say that attacking the failing tests would be a great way to get an introduction to the code base as well as find the areas of immediate impact. The failing tests are all over the place, so which ever strikes your interest would be good. I would mention that @gmittertreiner has been looking at the driver tests, so I would recommend that you do not go after those to avoid overlapping work.

I've been working, well, everywhere. We should be able to use the epic JIRA issue as a coordination point to avoid overlapping work.

Welcome to the effort, I look forward to your contributions!

2 Likes

I've uploaded the source to https://gist.github.com/troughton/9fdecf4dc09b0406c444c28f40f80d43. It works correctly in debug mode but crashes in release mode; it's hitting an internalInvariant on line 31 of ArrayBody.swift. (As a side note, this is a Swift 2-era program that's been lightly modified, so the code style is very out of date).

From the looks of things, it does seem like the generic parameter to Builtin.allocWithTailElems_1 is invalid somehow, so that allocation is getting corrupted. I haven't investigated too thoroughly, though.

I'll note that I ran into an issue in the build process - SwiftPrivatePthreadExtras can't be built because the definition of _beginthreadex isn't found (perhaps because it's only building against the single-threaded CRT?). I disabled that target to get past the error.

1 Like

official msvc build guide

however i still like to stick with cmake + native msbuild stye. I am start working at compile Swift without clang. Everything should be ok if the dependencies are available in windows

There are a number of deficiencies with MSVC. At least as of the 14.16.2703 toolset, the compiler has bugs with various template instantiations. It also doesn't support C11 (which we are using in a few places). Finally, MSVC doesn't support the swift calling convention. As I have put in the document, you can get the compiler to build with MSVC (although, that does require a few changes), but you cannot build the standard library without clang.

2 Likes

I have been following the latest developments in the Swift and Swift Foundation repos, and it seems Windows support is becoming a reality (thanks @compnerd, awesome job!).

Is there any plan at Apple / the Swift for TensorFlow team / other organization to hook up a Windows machine to the Swift CI infrastructure? I would love to be able to try the toolchain and follow its evolution on Windows but I don't have time/resources to build it myself (probably the same applies to many others).

10 Likes

A year has passed. What's the current status of Swift on Windows?

2 Likes

I think it would also be good to know the status of Android and Linux as well. So much work has been done so far, where are we? And what is the next focus? @compnerd @Buttaface

Linux distros such as redhat Debian should be first to officially supported along with Ubuntu latest lts release 20.4.
Windows and Android maybe the second wave to add support.

You can compile Swift code on Windows. All the core libraries can be linked against (Foundation, FoundationNetworking, FoundationXML, Dispatch, XCTest), and also the WinSDK. But the result is still fragile at runtime. Of the tests I have started running on Windows, just under half of them crash with segmentation faults. It seems to be mostly centred around the compiler botching complicated protocol hierarchies or something, although I haven’t been able to narrow it down yet. The places I would have expected the most difficultly, because of the greatest platform differences—like FileManager and Process—are actually working flawlessly. SwiftPM is not yet functional, so you have to use CMake.

Linux has been officially supported and on par with macOS for some time now. You will just find fewer third‐party development utilities for Linux. (Not sure about variants besides Ubuntu though, if that is what you were referring to.)

Android works and some companies are using it in production. Of the core libraries, only FoundationNetworking has been giving me problems. Both SwiftPM and CMake can cross‐compile from Linux. Apparently SwiftPM can also be used to compile source natively on Android. The compiler occasionally trips for Android where it wouldn’t for macOS or Linux, resulting in the need for either #if !os(Android) or code to be refactored. (I think this too is mostly concentrated around complex protocol hierarchies.)

The main hurdles are that:

  • These platforms are still experimental and not officially supported.
  • The necessary toolchains aren’t available at the official Swift site, so to install you have to do more work collecting the pieces (and figuring out where to find them in the first place). Or else you have to build the whole toolchain from source.
  • You will need to supply a lot more build flags manually, because the compiler, CMake and SwiftPM haven’t yet been taught where to find everything either.

You can see an example of a package being tested on all platforms here. And you can search it’s source for os(Windows) or os(Android) to get a sense of some of the platform differences and what does and doesn’t work yet. The approximate number of lines in the various set‐up scripts can serve as a rough comparison of how easy it is to run tests on each platform:

Platform Build Test Notes
macOS 1 1 “swift test”
Linux 3 3 download Swift, unpack, “swift test”
Windows 40 40 configure Visual Studio; install platform module maps, Swift, and ICU; minor repairs; additions to `PATH`; run CMake; test
(This figure does not include writing the `CMakeLists.txt` files.)
Android 60 70 install official Swift, cross‐compilation toolchain, Android SDK and ICU; minor repairs; write out a destination file, “swift build”; copy libraries; install on emulator/device; execute test executable
(This figure assumes you already have the Android NDK installed.)

It won’t help you do a proper install on your own device, but if you just want to experiment, the same GitHub workflow linked above can be set up for any package with this tool. It includes the automatic generation of all the CMake pieces. (You can copy and paste instead, but you’ll then have to manually adjust pieces of it according to your particular package manifest.)

6 Likes

I can't speak much for linux, or even Android cross-compilation, as my work has been to get a Swift toolchain natively running on an Android device. The goal is to get it to the point where you can just type pkg install swift and start writing Swift packages on your Android tablet, but for now you will have to build the Swift toolchain from source, with a few simple steps. The advantage of getting everything running natively is that I can run all the tests easily and I reported last month that I was down to only 16 test failures across all the tests from the Swift compiler validation suite up through the SwiftPM and sourcekit-lsp tests.

Since then, the last test fix for Foundation has been merged, so the only failures are 11 in Swift validation, 1 in XCTest, and 3 in SwiftPM, largely inconsequential issues like some validation tests not having RUNPATH set like Android requires or some LLVM sanitizers not working on Android. Of course, I'm guessing the tests don't cover everything and there is some functionality that hasn't been implemented for Android, such as Bundle support (ironically, @pvieito, who started this thread a couple years ago, tried to fix it and then abandoned it), but I haven't looked into how much of that there is.

I tried building and testing some Swift packages on an Android device, such as SwiftyJSON, the most-starred Swift package on github that advertises linux support. While other packages worked great, with swift build; swift test just working and passing all their tests, SwiftyJSON has been broken on linux itself with Swift 5.1. It's a simple fix though, and with a couple other tweaks for the lack of Bundle support, I was able to get the same test results on both linux x86_64 and Android aarch64, around 5 test failures, will probably submit a pull with those tweaks. Another package that had problems was swift-nio, which supposedly worked at some point for cross-compiling to Android, but will require some patching to build natively.

I have not tried building an Android app like the flowkey guys or writing much of anything myself with this new Swift toolchain on Android, so I can only report test results. Jeremy has another perspective just above.

I think it might be possible to set up a native Android CI now using the new Anbox Cloud service I mentioned in my post last month. I'll reach out to them and see how it works.

6 Likes

Arch Linux is a popular choice in Linux world, however getting Swift built and run is somewhat painful. I also want to use Swift on FreeBSD (or even on NetBSD) but *BSDs are quite unsupported platforms.

Arch Linux is a popular choice in Linux world, however getting Swift built and run is somewhat painful.

Have you built it lately? I've been building Swift master on Arch Linux x86_64 occasionally since last year, and while there used to be some tweaks needed back then, I haven't had to patch it at all lately. If you try master or the 5.2 branch, it should just work.

I also want to use Swift on FreeBSD (or even on NetBSD) but *BSDs are quite unsupported platforms.

There is some support and since Darwin/macOS and Android's Bionic libc use a lot of BSD code, I doubt it would take much to get the BSDs working.

1 Like

I wanted to confirm it but ran into a weird issue compiling compiler-rt with the latest 5.2 DEVELOPMENT branch (sanitizer_platform_limits_posix.cc:1131:1: error: 'assertion_failed__1131' declared as an array with a negative size). I will file a bug.
According to this post Swift on FreeBSD FreeBSD needs a helping hand, specifically porting Foundation stuff.

1 Like

The "array declared with a negative size" is an assertion failure. Something is incorrect about the configuration.

Can't tell you what went wrong. I've been using the same build enviroment for building Swift snapshots. Ticket for this issue is opened https://bugs.swift.org/browse/SR-12224

Terms of Service

Privacy Policy

Cookie Policy