CI for Packages on Windows & Android?

sspringer · March 11, 2020, 5:03am

OK, I understand. But I would now try to add those arguments until it somehow compiles – I think an example project like this one that is referenced by the "Getting Started (Windows)" documentation should compile without hurdles and that should be done by someone who knows how to do it right. Maybe an example project is needed for each platform then. Do not understand me wrong, I am not trying to be lazy here or not willing to contribute.

compnerd · March 11, 2020, 5:17am

-D CMAKE_Swift_FLAGS="-sdk <Path to SDK> -I <Path to SDK>/usr/lib/swift -L <Path to SDK>/usr/lib/swift/<os>"

The patch that I had to remove the need for the -I and -L parameter bit-rot before I could get it merged. The value is quite simply repetition, ideally, this would just be a single flag (-sdk <Path to SDK>).

There is no right way here - there are constant improvements being made, so its very easy to drift between master and the current release.

sspringer · March 11, 2020, 7:28am

OK, clear enough, thank you. I tried to update the documentation and sent you pull requests for GitHub - compnerd/swift-build: Alternate Swift Builds and GitHub - compnerd/swift-cmake-examples: Swift example projects. Will try to keep it up-to-date for the "newest" version (= the version you would download when following the instructions on "Getting Started (Windows)") when things change.

compnerd · March 25, 2020, 3:12am

Hey @SDGGiesbrecht,

Now that 5.2 has been finalized, I've archived a build from my Azure setup on GitHub. At the moment, it only has the MSIs and the exe installer. I am wondering if that is sufficient for you were working on or if you the tarball is necessary. The difference is is stark - the tarball is ~2G but the installer is ~300M. The individual MSIs are saved because the runtime and ICU are needed for re-distribution.

SDGGiesbrecht · March 25, 2020, 3:20am

Thanks for the heads up.

That also means a whole bunch of other tasks just budged in to the front of the line in my to‐do list. I will let you now when I get to this.

SDGGiesbrecht · April 2, 2020, 10:51pm

Okay, I’ve come back to this now. Forgive my Windows illiteracy, but how is the installer.exe supposed to be used? I downloaded and launched it in a shell remotely, and now I’m at 15 mins and counting with no indication of what it is doing. Is it waiting for me to react to something a GUI?

compnerd · April 2, 2020, 11:38pm

Yes, the installer.exe is a UI application. I believe that there are command line options that you can pass to it to run that in a scripted fashion.

SDGGiesbrecht · April 3, 2020, 1:09am

Is this the sort of executable installer.exe is?

I had switched to trying the .msi installers before you answered. With them, it’s having trouble finding the C headers for CoreFoundation and dispatch. The diff from what worked before is here. Am I doing something wrong?

SDGGiesbrecht · April 3, 2020, 1:55am

Adding -I C:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\include to the CMake invocation successfully worked around it.

compnerd · April 3, 2020, 2:32am

The installer is a WiX installer, the docs for that are on its website at https://wixtoolset.org

swift-build/GettingStartedWindows.md at master · compnerd/swift-build · GitHub has instructions which include the flags you should be including.

SDGGiesbrecht · April 3, 2020, 4:21am

Thanks. Updating the CMake flags to those ones worked.

SDGGiesbrecht · April 4, 2020, 3:14am

The Windows archives are working like a charm.

Would you be willing to archive the Android SDKs too? A .zip would be better than an .msi, since it also gets used from Linux. Am I correct in assuming the SDK would work with the standard 5.2 Linux toolchain, or would you need to provide an archive of the toolchain too?

compnerd · April 4, 2020, 8:04pm

Yeah, the Android SDK is a zip on the azure side. Once the 5.2 installer is repaired, Im happy to upload a zip of the Android SDK as well for the 5.2.1 release.

Yes, the Android SDK should be compatible with the 5.2.1 Linux toolchain as well, I'm building at the same points.

SDGGiesbrecht · April 6, 2020, 11:26pm

I’m not really sure what you are referring to, but let me know if I can be of assistance.

compnerd · April 7, 2020, 1:58am

I recently added the LLDB python scripts to the installer to ensure that formatting works in the 5.3 release to get the REPL on Windows in a better state. Unfortunately, that had a small unintended consequence of breaking the 5.2 release builds. I'm just trying to get the installer for that fixed up so I can upload binaries for the 5.2.1 release.

Actually, it seems that something broke in a recent update - Foundation is failing with invalid arguments on the 5.2 branch (-pthread is unrecognized). If you could help investigate what is the cause of that so that we can get that fixed, it would be greatly helpful for future updates.

Additionally, the Windows SDK seems to have failed to build due to curl.h not being found. Hopefully that is a transient error.

SDGGiesbrecht · April 7, 2020, 4:18am

I just saw your downloads for 5.2.1 are available. I’ll let you know when I’ve had a chance to take them for a spin.

SDGGiesbrecht · April 7, 2020, 8:42pm

The update for Android to Swift 5.2 went without a hitch. I’m now cleaning up some of the flags and patches that are no longer necessary. Thanks for providing the release, @compnerd!

compnerd · April 8, 2020, 3:05am

Great! Good to hear that this is useful to others as well

Ideally this will become simpler with fewer flags and fewer changes. Could you by any chance summarize what patches and configuration beyond the documented flags are needed?

SDGGiesbrecht · April 8, 2020, 4:14am

This obsoleted most of the patches and workarounds. (All of them on Windows.)

For Android, a user still needs to:

Acquire ICU separately. (Only necessary at runtime.)
Adjust the glibc modulemap to point at the NDK on the current device.
When using SwiftPM, shuffle the documented flags accordingly, because SwiftPM’s CLI is slightly different than the compiler’s. (master has already reduced these even further.)

compnerd · April 9, 2020, 12:12am

Amazing, that is great to hear!

The android side is obviously still a bit rougher as Windows has been serving as my guinea pig. However, I've tried to keep all the work as generic as possible so that all the platforms can benefit from it. To figure out how to handle the last of the Android side of things:

ICU should remain a separate component. I think that figuring out the cross-compilation and getting the ICU data generation for android working would make this reasonably nice: nightly builds of ICU that can be made part of the artifacts.
The modulemap issue I think needs to be solved. At the very least, I think that there is a secondary modulemap that we generate and should be installed instead so that the build tree is not emitted into the modulemap. The resulting modulemap could just be injected into the android NDK.
SwiftPM definitely is not as easy to cross-compile with. I dont really know what can be done here.

It seems that at least the first two items are tractable and something that anyone willing to get involved could easily tackle. I think that if you (or someone else) is willing to help with these issues, it might be possible to get the changes into the 5.3 branch, making the experience significantly better.