CI for Packages on Windows & Android?

Using Swift
, ,
1

These questions are mostly directed at @compnerd, but I’ve created it as a thread in case anyone else happens to have useful advice. My primary questions are in bold.

It has been a while, but now it has finally reached the top of my list to try and make several Swift packages Windows and Android proof. I do not have either OS around to try it myself, but there are users who want to use the packages on those platforms. If the code becomes littered with #if os(Windows), then users will think it targets Windows, but if the code does not actually undergo any consistent testing, then it will be far too easy to introduce breakage by accident. So my aim is to get some form of CI set up for those platforms.

It is import to note that I don’t really care (yet) how much code needs to disabled behind #if statements. Even if 99% of a library turns out to be currently non‐functional, I want to be able to guard it with #if so that the remaining 1% can be both made available and subjected to tests to ensure that the percentage only ever goes up over time.

I am aware of the Azure builds of Swift, though I’m not sure I’m clear on how to use them. The associated GitHub repository contains Docker instructions. Does that mean they could be used in a GitHub Action? The main Actions documentation seems to say Docker cannot be used on Windows, but the Windows runner lists Docker as installed, so I am a little confused.

I’ve seen the work that’s been done over the last few months on SwiftPM. But I don’t know how far there is still to go. Do the Azure builds contain SwiftPM? How operable is it? If it doesn’t work yet, I’ve been thinking about how to most easily convert a package into CMake, which seems to be the current recommendation for those platforms if I recall correctly. I’ve found these examples of Windows + CMake + Swift, and it looks like it shouldn’t be too hard to derive that sort of CMake instructions from the package manifest on macOS, to then be used instead of SwiftPM when the repository is checked out on Windows. But I don’t want to start work on that if there is already a better way.

How stable is the Docker container associated with the Azure builds? Am I correct that it is always more‐or‐less at the cutting edge of Swift’s development state? Is there a way to reference a fixed version of it, so that CI reliably uses the exact same state until manually updated?

6 Likes
2

Hi,

I'm going to respond to various bits, if I miss something, please point it out, I probably just missed answering it.

I would love to get help here. I would like to improve the documentation on GitHub to make this more useful so people can get started more easily. What it amounts to roughly is downloading a few zip files, extracting them, and running with a set of flags. This is a regression as the MSI builds broke and Ive not gotten around to fixing them. That is still on my list, but of course help is welcome!

Something along the lines of:

python -m pip install --user azure-devops tabulate
python swift-build.py --build-id VS2019 --latest-artifacts --filter windows-x64 --download

should download the latest zip files for you. swift-build.py is in the utilities directory in swift-build. Extract the contents to C:\. You will need to add the following flags when invoking swiftc.exe: -resource-dir C:/Library/Developer/Platforms/Windows.platform/Developer/SDKs/Windows.sdk/usr/lib/swift. You must invoke this in the x64 Native Developer Command Prompt (or run VsDevCmd -arch=amd64 -host_arch=amd64 prior to using Swift).

Yes, the docker work is something that I've been looking at to ease the introduction to Swift for new comers.

You could try, I've not tried that. Note that they are not sufficient for building for Windows currently (beyond a few missing files that I need to add), they need the user to provide the Windows SDK as I cannot redistribute that. You can find that as part of the VS Build Tools installer on Windows. That is, you need to run the docker container on Windows or somehow provide the necessary content.

Sadly, I've not made as much progress as I would have liked to. The CI work ended up taking a turn for the worse and required a lot more effort to get to a state where it could be useful and understandable. Im hoping that will wrap up soon and I can return to working on more interesting things. I have a back log of patches that is sufficient to get tools-support-core building and almost working enough to get swift-package-manager building and running on Windows. I haven't fully gotten it to bootstrap yet though. There are issues with paths, and some more issues regarding paths in Foundation that need to be addressed.

Correct, that is exactly what I have done for porting the packages that I have added CMake builds for. They were largely just enumerating the targets and sources.

They are just daily updates, I do not really have much in the way of testing. If you have the time or inclination to setup testing, Id be happy to see that work get merged.

It is for now. If I manage to get some time, Id like to setup 5.2 builds. For now, its just tracking master.

Id like to delete the development snapshots at some point since they seem pretty big and it seems silly to just grow the collection unbounded. But, Id probably retain stable snapshots of major releases for how ever long they let you keep the images.

1 Like
3

The way I'd approach this would be to cross-compile the Windows build from an Ubuntu host (macOS also works but is a little trickier) using Swift Package Manager. If you download one of @compnerd's Windows-SDK builds and a matching Ubuntu toolchain from swift.org, you can copy the directory contents into the Swift toolchain to get a toolchain that can build for Windows.

If you want to also test the built executables, you could use a Windows image with Ubuntu installed under WSL, building on WSL and testing natively.

You'll still need access to the Windows Kits and MSVC include and lib folder from a Visual Studio installer, and may have to make some modifications to the headers, but once those are available cross-compiling is reasonably straightforward. I can provide more details and help you get it set up if that's an approach you want to take.

2 Likes
4

That was the other thing I meant to ask. How do you tell SwiftPM to cross‐compile? Is that what customCompileDestination and its CLI option are about? I’m not sure I understand what JSON file it needs to point at.

(Regardless of what I do with Windows, I’ll need to do cross‐compilation for Android anyway, right?)

5

You can pass a JSON file with the --destination argument to swift build, and the JSON file should follow the template from https://github.com/apple/swift-package-manager/blob/master/Sources/Workspace/Destination.swift#L148.

As an example, here's one I've used in the past: Substrate/WindowsDest.template at Vulkan · troughton/Substrate · GitHub. I wouldn't copy that exactly though, since a lot of the flags or defines are incorrect or hacky workarounds.

I haven't built for Android myself, but yes, I'd imagine you'll need to cross-compile for that.

6

Thank to both of you. Your comments have been extremely helpful. I’ll see what I can figure out from here.

7

For the sake of having all the information in one place and easy to find until I get this figure out:

8

This is turning out to be harder than I expected, but I haven’t given up yet.

It took several days to refactor my package to sanitize file names according to Window’s limitations just so that I could successfully check the repository out on Windows.

Then the last two days were spent trying to figure out how to find and apply the environment from vcvarsall.bat/VsDevCmd from the command line remotely. In the end it means launching a batch file which runs the relevant command and then launches a bash script, which writes the environment to a file, which the outermost shell can then read in so that it can apply the file contents to the actual environment.

Now to actually fetch the toolchain and SDK onto GitHub’s host...

9

You may want to have a look at the .ci directory in my swift-build repository at GitHub - compnerd/swift-build: Alternate Swift Builds which contains a bunch of automation for building Swift content on Azure which should be largely similar to the GitHub builders (minus syntax for the GitHub Actions rather than Azure Pipelines).

2 Likes
10

Two more questions (while I continue to experiment):

  1. Do the artifacts have to be unpacked to C:\? On macOS and Linux you can put a toolchain virtually anywhere you want, as long as you add it to the path. I’m asking because it would be convenient to cache them, and that requires them to be self‐contained and live inside the repository directory (albeit .gitignored).

  2. Since I’m not testing Swift itself, but rather testing packages that use of Swift, it would be nice to have stable and persistent access to a particular build. If I periodically choose a recent artifact to use as a sort of “stable release”, do I have permission to upload them to a GitHub release or something similar, in order to both persist the artifact and maintain a stable URL to fetch it from? Or do you have a better idea to accomplish that?

11

Nope, they do not need to be unpacked to C:\. That is the path I document because of the fact that it is closest to what I use (S:). The reason that I do that is because if you are on the same drive, you can then copy-paste the command across OSes (i.e. /Library/Developer/Toolchains/unknown-Asserts-development.xctoolchain/usr/bin/swiftc is the universal path to the Swift compiler - Linux, macOS, or Windows).

I'm trying to work out a stable URL for downloading for long-term persistent storage. If you merely care about a URL to fetch from and not necessarily the version: Utilities/swift-build.py in the swift-build repository is your friend. It has a --latest-artifacts --build-id VS2019 --download --filter windows-x64 option to download the toolchain. You can play around with filters to download what you want. Of course you are free to store the releases, I'd just like to generally encourage a single source of truth longer term.

1 Like
12

Yup. And I’ve already got it working like a charm. I’m just hesitant that with it potentially pulling a new image from one CI run to the next, it will be harder to tell the difference between a break due to a regression in my code and a break due to a regression in Swift itself.

And I would much rather point to your single source of truth (which is where the builds will ultimately come from either way). There is only some hesitation about the unpredictability mentioned above, and the enticement of a possible reduction in the 10 min download time.

I’m still experimenting, so it remains to be seen what strategy I actually end up deciding on.

13

Completely understandable. So, the reason for that suggestion was two fold:

  1. The current retention period is 30 days. Now, I do intend to actually create a 5.2 release which can be preserved long term on GitHub releases once it is marked as final. In the mean time, there is no stable point to start from is the issue. If you intend to rev within 30 days, then sticking to an exact build number makes sense.
  2. The python script does not currently support explicitly specifying a build ID. That is easily resolved (especially if you send a patch to add --build-number :smiley: )

So, if you are okay with the retention period on the exact ID, I think that using an exact ID is feasible even with the current setup.

14 
$ [...]/swift.exe --version
compnerd.org Swift version 5.2-dev (Swift 00dd7204b7)
Target: x86_64-unknown-windows-msvc

:tada:

Now to actually do something with it...

15

That is definitely what I will use once it is available.

Barring the unexpected, I don’t really plan on bumping what I’m pointing at any more often than happens on macOS and Linux with official releases. The point is to be able to say, “My package is tested and known to work with Swift [some version or specific development build]. Use that version or something compatible with it.” I would be pointing at Swift 5.1.3 if such a thing existed and were viable.

Would you be willing to create a single 5.2-dev-2020‐01‐16 “pre‐release” just as something consistent to point at until 5.2 is finalized? (That is basically what I will end up doing myself if the system of querying Azure turns out to be flaky.)

16

No luck so far launching Linux within GitHub’s Windows host for cross‐compiling.

Docker

$ docker run compnerd/swift bash 'echo "Hello, world!"'
C:\Program Files\Docker\docker.exe: image operating system "linux" cannot be used on this platform.

$ docker run --platform linux compnerd/swift bash 'echo "Hello, world!"'
"--platform" is only supported on a Docker daemon with experimental features enabled

$ docker run --experimental --platform linux compnerd/swift bash 'echo "Hello, world!"'
unknown flag: --experimental

$ '/c/Program Files/Docker/Docker/DockerCli.exe' -SwitchLinuxEngine
d:\a\_temp\89cbc20f-f7ba-45fe-a5d9-fed553cb9de4.sh: line 8: /c/Program Files/Docker/Docker/DockerCli.exe: No such file or directory

WSL

https://github.com/actions/virtual-environments/issues/50

It looks like it will have to be the CMake route until SwiftPM works natively on Windows.

17

I think that is the piece I’m missing. But I don’t know how to feed that through from the CMake command line invocation (or an environment variable). In any case, I’m stuck right now with the following error from CMake:

:0: error: unable to load standard library for target 'x86_64-unknown-windows-msvc'

18

-DCMAKE_Swift_FLAGS=...

I do this in a number of places in the CI configuration BTW :slight_smile:

19

:man_facepalming: Sorry. I have so many separate links I’m looking back at as I do this that I keep finding myself missing the needle at the top of the haystack. Thanks for your help and patience.

20

You’re welcome. It’s quite alright, I’ve had my fair share of trouble getting things correct, especially when trying to create a unified command that works across all the targets. I totally understand the frustration it can cause :slight_smile:.