Increasing visibility of server side swift platform gaps?

One thing I’ve thought a bit about is whether it’d make any sense trying to accumulate references to known Swift on Server Linux issues for making the platform gap more visible?

Has there been any discussions in the SSWG in that direction?

There are a number of loose ends one encounters (e.g. lldb crashes, TSAN false positives, installation experience (know that’s on the radar), packages that aren’t cross platform (e.g. Sourcery)) all creating just a little bit of friction each.

The idea would be to give a better understanding what the gap is, but also to give
anyone who wants to pitch in some starting points where we can help out - and
as the gap shrinks, perhaps better showing that swift truly is cross-platform - and
given that it was one of the first goals for Swift 6 in Teds post ‘on the road to swift 6’
maybe help allocate engineering time to work on that goal.

Just wanted to open up a discussion if it makes sense and discuss how it possibly
could be done in practice.

Thoughts?

6 Likes

In my opinion, this is a worthwhile thing, although you need to clarify how much emphasis you give to the "server" in "server side swift".

As I have mentioned before, I'm very unhappy with the mass of feature disparity and the difference of semantics. The sad thing is, the gap is growing instead of shrinking ­– despite the warm words written on, e.g., https://github.com/apple/swift-corelibs-foundation/blob/main/README.md:

Our primary goal is to achieve implementation parity with Foundation on Apple platforms. This will help to enable the overall Swift goal of portability .

This is just not true, especially if you hop over to the issue tracker and take a look at how the community contributions get "considered".

I think the main problem is that many people in the "Server-side Swift" community seem to take the "server" very seriously, thus completely ignoring developers who want to write cross-platform software that does not necessarily do "server" things.

Yes, you can continue to tell me "Don't use Foundation on Linux", but this is bad joke, if I want the core libraries of my APPLE apps to also run on !APPLE devices.

So… yes, making the gap more visible is surely a good thing. Will it help? Probably not. Apple has their own Agenda here and seem to ignore features that they do not need.

Which really is a shame, since Swift has so much potential, in particular on Linux and embedded systems.

1 Like

Apple has their own Agenda here and seem to ignore features that they do not need.

How do you imagine it should work: Apple commits their developer resources to features they don't need? If so, do you plan to reciprocate by volunteering your time to work for free on features Apple needs?

For Swift to become a real open source project, people need to stop expecting Apple to do stuff and pick it up for themselves. I understand you are just one man, but other OSS language communities use sponsorships and feature bounties to fund such work, no reason you and I can't do the same without Apple being involved.

3 Likes

Not when it comes to core libraries, or are you expecting the community to maintain a forked version of Swift with another Foundation? For certain things, we need Apple to cooperate with a more broad vision of how people are using Swift on !Apple-systems. Which ­– at least watching some of the issue trackers ­– isn't what they're doing.

are you expecting the community to maintain a forked version of Swift with another Foundation?

If Swift ever becomes successful outside Apple platforms, I expect there to be multiple divergent forks, just as there are multiple linux kernels.

It depends on if your issue is with reviewing contributions or working on needed features, I was talking about the latter above. If you believe instead that outside code contributions are not being reviewed and ignored, I suggest you take that up with the core team. If that still doesn't work, I see nothing wrong with forking.

In my opinion, this is a worthwhile thing, although you need to clarify how much emphasis you give to the "server" in "server side swift".

I'm afraid I probably qualify into the bucket of people who take 'server' seriously, but that doesn't mean I think it is bad to also capture Foundation gaps etc (although I will confess to opt-out from Foundation from what we are doing, hoping that we'll over time evolve an ecosystem with smaller packages that supplants Foundation as things move forward - it is quite painful now though with things like calendars and timezones etc...) - but those are just our preferences, I can definitely see the value in general support for !Apple platform support for many other applications and think we could capture that too.

It's really why I brought this up, I think there are a fair amount of non-Apple people who would like to see first-class support for Swift on Linux (and also Windows, but that is outside my platform radar at this point).

I agree with @Finagolfin - need to stop expecting Apple to do everything (although it's much appreciated when they do pitch in - for some pieces like e.g. libdispatch where source drops come at a random timeline, it's a bit more difficult for third parties to pitch in). We definitely need responsive PR reviews though from the teams owning repos, otherwise it's difficult to get traction - I see forking as a last resort when things doesn't move forward and would want to avoid that in general...

So what would be a good way to capture things then, a GitHub repo with links to relevant issues / gaps in other repos? Then people can provide PR:s and it is possible to maintain on a community basis.

If there are enough people interested I can set such a repo up.

We have our hands tied.

People would love to contribute to and advance Foundation, but the project is limited to having API parity (and bug parity!) with Apple’s proprietary implementation.

I think only two approaches makes sense:

  1. Apple goes hands-on and contributes to corelibs Foundation, even for features they don't immediately need
  2. They go hands off, and let others build out the ecosystem.

Right now we're in a nonsensical middle-ground, where Apple owns the direction and API design, but has no incentive to implement anything that isn't for their platforms.

4 Likes

Compiling is too slow in the Linux (3-5 min for a hello world vapor server project compared to other language only in 10 seconds) and set Swift to incremental compiling in Linux is difficult (I didn't figure it out right now).

Are you talking about incremental compilation in general (which should just work) or incremental compilation in ephemeral Docker containers? The latter was -- until recently -- a little bit harder because of an implementation detail in Docker which gives folders that get mapped into containers fresh inode numbers which invalidates the build cache.

If you're talking about Docker, the key to making it work is

RUN --mount=type=cache,target=/code/.build swift build -c release

The --mount=type=cache,target=/code/.build will put SwiftPM's .build folder into a cache volume which retains inode numbers, mtimes and everything else. With that, incremental compilation will just work

7 Likes

Thank you for considering the case with Docker.
I just tried it with Amazon Linux (without Docker), and unfortunately I could not compile the hello world vapor project in a short time. It took 3-5 min each time each try. And I cannot figure out how to incremental complication in this case. 3-5 min complication time is hard to bear even incremental complication could reduce the complication time in the second try.

this is a game-changer for me! thanks!!

1 Like

one question we really need to answer as a community is “what does cross-platform mean”?

  1. packages that support linux only, instead of Apple platforms only, or
  2. packages that support linux and Apple platforms.

as a “server” library developer, #1 is easy for me, since my personal use case is almost always linux first.

on the other hand, i and others have had great difficulty ([0], [1], [2]) achieving the latter. which is not ideal.

2 Likes

I’d definitely be in the 2. Camp - we want to deploy ‘server’ (non-ui) code on both macOS and Linux. (But I don’t care about the iOS style platforms personally)

So I guess I’m 3:

Portable between macOS and Linux as a minimum to be considered ‘cross platform’. (Acknowledging that other people can have other preferences, but those are mine)

what annoys me is that a lot of times packages that work on macOS, should “just work” on iOS too, but because of minor obstacles, don’t.

there’s no good reason why swift-hash shouldn’t work on iOS. it doesn’t interact with the OS or anything. and some multiplatform libraries, like swift-png have features like iPhone-optimized format support that are useful on iOS.

it shouldn’t require this much effort to support iOS, watchOS, etc, if a package already supports macOS.

1 Like

That is super odd. Would you mind sharing (probably via a DM) the exact steps you're running? I'd like to reproduce this as I've never seen this before.