Initiative: Swift Platform Work Group

To be clear, this is no more than an initiative, instead of an announcement — SPWG is not a thing yet, but I would like to see people pushing it and finally bringing it to life.

Despite not being mentioned in the post, making Swift available on more Linux distributions is also a promising goal. Thank all of you who’re working on it!

2 Likes

Super happy to see this initiative. I've been trying to dip my toes in cross compiling swift a few times in the previous years without success, so i'm afraid i won't be able to help much at least in the beginning, beside writing some documentation and testing some scripts on my configuration.

Where can people join this workgroup ?

Just to be clear - Windows ARM support is in many senses already there. Although not released, there are ARM64 toolchain builds, and an ARM64 SDK as well. There is a cache file available to build the ARM64 SDK

The ARMv7 support has a few known issues with code generation: e.g. HFAs are not lowered properly ( WIP/IRGen: PCS for ARM VFP CC does not correctly handle HFAs by compnerd · Pull Request #14304 · apple/swift (github.com))

The ARM64 runtime does build, though there is a failure that needs to be investigated still in Foundation that currently causes issues with the swift-driver. The C++ driver does function properly though.

In fact, the X86 target also has been repaired for the most part. The X86 support has one known limitation: calling conventions on C functions. In particular here, SIL has a limitation that it does not properly preserve the calling convention and if the calling convention does not match the default calling convention, it will lower the calls to C or closures improperly.

5 Likes

The Slack was renamed from the Swift Server Slack to Swift Open Source to encompass more than just server work. The Website Workgroup also coordinate their efforts their so the more the merrier! Feel free to create public/private channels as needed

3 Likes

A stated goal of the group IMO should be to get the current ports merged into mainline and importantly CI set up too. That makes adoption on those platforms significantly easier and better long term. Will need support from the Core Team and Language Team however

3 Likes

I think this is a fantastic idea, and an important area for Swift to improve. Thank you for writing up this proposal!

Others have noted how this intersects with the proposed Swift Tooling Workgroup. I recommend narrowing this bullet point to reduce the overlap and provide more focus for this effort. What about something like:

  • Simplify cross-compilation from any host: From any host toolchain, it should be easy to cross-compile to another target given the appropriate SDK, without having to build a complete, separate toolchain.

To the best of my knowledge, Swift's language feature set is consistent across the various platforms. The difference with Apple's platforms is that they are the only ones shipping ABI-stable Swift, because the Swift standard library and runtime is part of the OS. Platform API availability, library evolution, and ABI stability are all part of one conceptual feature, and if some other platform were to choose to ship an ABI-stable Swift, that would involve adding it as a platform with version mappings between platform version and Swift version, along with enabling library evolution for the Swift standard library. That might be a reasonable thing to do, but I don't consider that to be "Empowering Swift compiler for existing platforms."

I recommend either removing this bullet or narrowing it down to something focused on bringing ABI stability to additional platforms. I'm mixed on whether other platforms actually need ABI stability at this point.

I totally understand where you're coming from here, but with my Apple hat on... I'd rather this bullet not be part of the proposal. The changes we all make to Swift on GitHub make it into the toolchains Apple ships in Xcode and uses in its OS platforms.

I think low-resource and bare-metal platforms will benefit greatly from the initiative you are proposing, but I agree that it's a separate thing. It's good to stay focused.

I agree with this :100: . I would like the barrier for a port to be part of the main GitHub repository to be fairly low, so long as it's not breaking CI or causing problems elsewhere, and have ports mature in-tree. The overhead of maintaining separate toolchains for a port is fairly high (e.g., half of SwiftWASM's diff with the main Swift repository is in build/CI configuration), and it makes it unclear where one should start if they just want to try out Swift.

Doug

16 Likes

I think we need to be very explicit here about whether the focus of the platform workgroup is solely about the core language or also encompasses things like Foundation, SwiftPM, XCTest etc that aren't part of the language but still considered to be part of 'Swift' because these things certainly don't have good enough consistency across the different supported platforms

11 Likes

Even if I agree with basically everything you wrote, I'd just like to say that library evolution and availability annotations for third party libraries can be very useful as a subset of that 'one conceptual feature' for people shipping 'enterprise style' applications, we'd definitely want to use that and have opened a couple of cases for it:

We played with enabling .xcframework support on Linux with a fork of SwiftPM (which was trivial) and it seems to work quite nicely, although the current lack of availability annotations makes it impractical at the moment.

The ability to ship libraries with evolution mode turned on (on other platforms than macOS, Linux specifically in our case - and hopefully with availability annotations for third party libraries in the future) even if requiring the same toolchain version, would be very useful in practice...

We'd happily build our software with N compilers for distribution if needed for our customers, so don't care so much about the ABI-stability-per-toolchain-version restriction really.

1 Like

Indeed. It would be a shame if at least those three you have mentioned would not be part of this proposal. Although they're technically not part of Swift, they very much belong to it for almost everyone writing code that should work on both the APPLE and the !APPLE world.

1 Like

Yes, this is a good point; the proposal should be explicit about where the boundaries are. The paragraph I was responding to was focused on the compiler and mentions "language features" specifically. I think it's reasonable for this initiative to stay focused on compiler/standard library/runtime, because there is plenty of work to do there and I feel that a more focused workgroup will be more effective. It could expand in scope later to pull in Foundation/XCTest/etc.

I agree that availability annotations for third party libraries are a useful addition, but I'd consider those to be the domain of the existing Language Workgroup.

Yeah. Library evolution should be enable-able anywhere that we're dynamically linking the Swift runtime.

Doug

2 Likes

On the other hand, if one of the main target for the group is the Android platform, it seems like the very first use case would be to help developers port their existing ios / swift codebase to Android (simply from a market point of view). Which most likely includes heavy Foundation usage (files, network, and GCD at the minimum) as well as XCTest, etc...

IMHO scoping the group to include Foundation doesn't mean it has to work on that topic right away if there are more fundamental things to work on. It just helps to set the endgoal.

1 Like

No, it’s not, you cannot practically use async/await on Windows (AMD64) because of this.

1 Like

I had forgotten about this bug, thank you for the reference! This calling convention needs to be fully implemented for windows.

Doug

4 Likes

I’m still hesitated to narrow it down to covering cross-compilation only. At least, compiling natively (to the host platform) is also largely affected by platform issues (eg. mis-packaged Windows toolchain). Tooling could work around such problems, but eventually we want the toolchain layout to be correct. I would say this is an area where the two workgroups should collaborate on heavily.

When I wrote this bullet, I mainly thought of the poor state platform libraries (eg. Glibc) are in. Unlike Darwin which is nicely modularized and structured, other platform libraries are not well designed (due to drawbacks of overlay, Swift not supporting submodules, ...). Some features are, thus, unusable. I also came into the Windows case where API availability can only be detected at runtime, and I would like to see if #available to support such use case.

In short, I think this bullet should have better wording, since it involves more than the compiler. Thanks for your suggestions!

I don’t think ABI stability is something we should pursue in short term, especially when platform API stability is not in a good state yet. We could possibly set it as some end goal.

I fully understand your concern. This should not be a part of the official proposal or announcement (maybe just ignore Apple platforms?). I’m mostly happy with the state of how Swift toolchain is on macOS, so I would like the workgroup to focus on other platforms instead.

Yes, this is one of the key tasks for the new workgroup. Once CI is set up, we can have patches verified and merged faster, and in-tree development means the port has far lower risk of diverging from the mainline. Like what we’re doing for RPMs, I believe providing experimental toolchains on Swift.org will attract more developers to try out.

From my perspective, Foundation and Dispatch is, of course, part of the platform efforts (especially regarding to how they’re widely used as an abstraction of native platform APIs). Development of SwiftPM and other developer tools, on the other hand, seems more of the tooling side. Sometimes platform-related changes will affect tooling, and I think that’s where cross-workgroup collaboration and discussion is important.

Yes, library evolution is important and it’s a pursuable goal even under restricted ABI stability. You’ve got a nice try👍 IIRC library evolution itself is not so platform dependent, and neither is the artifact bundle support, so these two seem to be domains of language and tooling workgroup respectively.

We’re missing some key functionalities even on supported platforms, and they should be tracked centrally. Such work had better start from Swift compiler and standard libraries first, because the list is already long: unreliable concurrency (Windows x64), broken REPL (Windows), missing backtrace (non-Darwin)...

I don’t know if we will have volunteers to fix all of these, but at least we should let Swift developers know these restrictions and possible workarounds, in case they meet them when handling cross-platform use cases.

1 Like

I would be interested in more details on this. The toolchain backtraces are not symbolized due to missing symbol data and no debug information. For locally built packages, I would expect Windows backtraces to be symbolized, at when debug information is present. swift-server/swift-backtrace: :boom: Backtraces for Swift on Linux and Windows (github.com) also supports Windows and does work. The general issue for the runtime is distribution size of the debug information. The ideal thing to do here would be to enhance the CodeView emission and actually figure out how to get hosting sorted out for a source and symbol server so that the debug information can easily be distributed.

If possible, I would like to see runtime backtrace feature being implemented in Swift (EDIT: then it looks like domain of language workgroup?) — this makes a lot of sense for quick debugging and bug fixing, especially when we distribute our programs and rely on bug reports from users. SwiftPM and other developer tools are exactly the case, and silent crashes can’t help users know what’s happening and what they should do.

Backtrace partially solved the problem, but adopting it is hard for scripts and other non-SwiftPM scenarios. Missing backtrace support may also affect the compiler as we’re moving to Swift implementation.

I believe CodeView support is another promising point, but not what I initially meant. Earlier discussion seemed to indicate the drawback of Backtrace approach, and that’s where CodeView or DWARF support can largely help.

It’s also worth nothing that we should do better about async backtraces and follow task pointers etc. this is done on Darwin but not on Linux and other platforms. There’s no inherent reason why we couldn’t implement it for all platforms afaik.

6 Likes

I'm not sure I understand what exactly you are looking for here. This is already implemented in the runtime and does work across platforms (this is how the stack trace on abort works). It does require that you do not strip your binary on Linux and on Windows requires that you have debug information available. Is what you are asking here that the runtime function be made available to users?

I'm having trouble understanding the point here. CodeView and DWARF are merely encoding for debug information. The debug information itself is needed to fully symbolize stack traces, be that with the backtrace package or the runtime system.

As to the "drawback", I assume you are referring to the scripts and non-SPM usage? For non-SPM use cases, I feel like expecting the user to port to the desired build system is reasonable. For scripting scenarios, it seems that this may be a bigger issue - there are many helpful packages, e.g. swift-argument-parser. Perhaps something like a shared package cache might be an approach that could work for scripting, but this seems like an enhancement to how scripting support is implemented rather than a platform support issue.

2 Likes

I'm glad to see more platforms getting noticed by Apple for Swift support. I've submitted an initial RISCV-64 Linux patch and I've been maintained scripts and patches to cross compile Swift for 32-bit ARM Linux for Debian, Buildroot and Yocto distributions. I've been able to cross compile Swift 5.7 with less patches because most of them have been merged it. The main blocker right now is the Concurrency support being broken and having some kind of official CI would help a lot.

I would like to be a part of this workgroup as the company I currently work for has invested a lot in running Swift on Armv7, we have it deployed in production since 2017. Other people at the company have contributed to the Armv7 Linux port as well (I've taken over since 5.4), so we have a business case for maintaining it, aside from my personal interest.

Also the Linux/glibc and Android/bionic Armv7 ports both use cross compilation and interact with Linux kernel, so @Finagolfin and I have collaborated in the past and a lot of the fixes on one platform applies for the other.

5 Likes

I've released scripts to cross compile Swift for 32-bit ARM on Debian 11, along with Docker images to make it easy.

4 Likes