Large Proposal: Non-Standard Libraries

It seems you misunderstood (or maybe I did ;-) the motivation for this thread:
Afaics, it's not about adding anything to the stdlib at all, but having a place to put types and maybe algorithms that aren't essential for the language itself, but still are quite fundamental for coding.

Most coding is trivial - it could all be done with Assembler, without Swift or other fancy languages.
But even trivial things can be expensive.
If there's anything I expect from a good programming environment, it's protection from petty tasks like writing boilerplate or code like

func translate(input: Lib0.Point) -> Lib1.Point {
   return Lib1.Point(x: input.x, y: input.y)
}
5 Likes

Should we even make this discussion on Swift Evolution or is Using Swift more appropriate since Libraries are essentially a usage of Swift and not directly a language evolution?

1 Like

Another interesting debate is if Foundation evolution should start to be driven by the Swift community instead of only by Apple. At least it would be great to know if the community could start to extend Foundation in the Swift side without affecting the legacy Obj-C closed-source code.

This could be done by implementing new common code in swift/stdlib/public/SDK/Foundation (the Swift wrapper around the Obj-C Foundation used by Xcode) and swift-corelibs-foundation (for the rest of platforms).

1 Like

i don’t think mixing free and proprietary code under the same project would be good for either Apple or the free software community

1 Like

I think this entire discussion is borne out of frustration because it isn't easy enough to use 3rd-party packages in Swift projects yet, so people jump to the idea of bundling everything as some kind of quasi-standard library.

I would suggest people refocus their energy towards improving the package manager. There are actually a couple of cool proposals floating around right now which could be a starting point for plugging those holes, e.g.:

3 Likes

I don't think we should mix free and proprietary code. But if Apple/Swift wants to clearly define Foundation as the basic cross-platform Swift library, the best move is to evolve it as a fully open source project (even when the Objective-C code could still be closed source).

I mean, for now if we want to add any API to Foundation the only option is to fill a radar and wait to see if in the next release Apple has included it, and finally reimplementing it in Swift Foundation.

For me, this is absolutely not the case. My frustration comes from having seen how convenient Go's standard library is and what a mess outsourcing everything to 3rd parties is (npm).

Maybe outsourcing can work, I just haven't seen it happening personally (but I haven't used too many languages to be sure).

2 Likes

For me, that's not the case either, and I wouldn't want those supplied libraries to be large.
I just think it's ugly to re-invent the wheel whenever a basic type is needed, and even if SPM would be perfect, it wouldn't reduce the number of implementations that don't add any value, but just incompatibility (quaternions are my favorite example here).

2 Likes

It’s not exactly an either-or situation: we will need great support for 3rd-party libraries anyway. We should give that an actual chance before we absorb all kinds of secondary/tertiary functionality in to the standard library. It might seem like the standard library actually moves faster than the package manager, but there’s also a large maintenance burden to it.

NPM has some problems that we should definitely learn from. Go’s standard library is more like Swift + Foundation. The Foundation APIs could also be improved.

Standardisation is another topic. Maybe it makes sense for an officially-endorsed library of basic maths types and algorithms, but is there really such a huge difference between bundling it and adding a line to your Package.swift? If it is really so onerous, we should address that!

Maybe you think quarternions are important enough for every Swift installation everywhere to have a library knowing about them and how to use them. Where do we draw the line?

The recent talk about removing IndexDistance was also interesting in terms of seeing the standard library more a collection of common (but ultimately limited) patterns. Any endorsed libraries will also have limitations. Maybe you will still need a custom quaternion-type, even if the standard library provided one (maybe for data packing or marshalling reasons), in the same way you might need a custom Collection protocol if you wanted a count bigger than Int.max

it’s less about how many lines you add to Package.swift as it is about which lines you add

and yes, Quaternions are that important

1 Like

It would be enough if we had a single implementation that is widely accepted - automatically including it in every Swift-installation wouldn't be a requirement (although it would definitely help).
But considering that defining a quaternion would only take a couple of bytes, imho it's reasonable to just bundle it:
Quaternions haven't changed much in the last twenty years, so maintenance shouldn't be an issue.

Of course, that's just a single example, and it shouldn't receive special treatment - but we could do the same for basic types in fields like imaging, networks protocols and graphs without a significant impact on the size of Swift.

1 Like

Right, but Foundation is a black box as far as Swift is conerned.

Bringing in the code is the easy part. 3rd party code is a liability, the author may one day delete it, or they may lose interest and abandon it, there are no guarantees at all. Having useful code bundled in is all about quality guarantees.

This is very easy, the Swift community will draw the line! Perhaps through swift evolution or something similar.

2 Likes

I guess that depends on the tasks you are doing ;-) - but they are a great example, because even Apple itself suffers from lack of standardization:
When you want to visualize the orientation of an iPhone, you have to deal with two different implementations that aren't compatible... that's really ugly, and it's just because this basic concept neither belongs to SceneKit, nor to CoreMotion - but there's no place below those two where you could put it.

I would really like to see this.

I agree with the sentiment expressed by Karl – that this is borne out of frustration because it isn’t easy enough to use 3rd-party packages – as it's a frustration I have myself.

However, even once this is solved, I really like the idea of having an additional working group directly affiliated with Swift.org that I can go to and find a recommended library for a particular task. I think having a working group that enforces a level of quality, expectation on maintenance and documentation, guidance on development, etc. as a matter of policy on any library endorsed by it could be a really valuable thing for the community.

It would also be a great staging area for ideas that may or may not end up in the core lib. For example, today I noticed a debate on twitter about the inclusion in the core lib of some basic functional utils – id, const, curry and the like.

A working group like this would be a great way of supporting initiatives like these with a core team sanctioned library but without risking long-term flexibility for the language.

3 Likes

Hello,

Since it's easy to gest lost in meta discussions about the ideal non-standard library, I was wondering if it would help going down to earth with, say, a checklist that any library should pass before considering inclusion in the NStdLib.

This checklist could be tested with a few existing libs that are wildly considered of high quality.

A few existing libs would emerge as really good candidates for inclusion. Not all would check all items in the checklist, because the checklist would be very ambitious.

The next step would be to involve the maintainers of those libraires so that 1. those libs eventually comply with all items of the checklist, and 2. a first team of core maintainers of the NSfdLib would organically emerge. 3. the need for a practical NStdLib that community can use becomes a high priority topic. 4. SPM has even more traction.

3 Likes

Could the licensing terms include letting the Swift project host a copy in a repo it controls? Just as a backup, in case the original author bails.

1 Like

I agree that this is probably a good idea, and perhaps part of the "checklist" mentioned here:

should include continuous integration testing and a step to merge the repo with a semi-official "Swift 3rd Party" group's fork on successful builds or when new tags/versions are released. Then developers could always include the Swift 3rd Party's fork URL in their Package.swift and hopefully never deal with a developer deleting a repo.

maybe off-topic but there could be serious licensing issues going on here as the Swift Project is not very GPL friendly at the moment due to Apple’s concerns

i say this because i submitted my image codec library to the source compatibility suite and it was rejected because it was GPL

That doesn't solve the issue of the library not being maintained. And I doubt Apple will agree to host copies of unmaintained libraries.

If the goal is to set up a set of libraries that are going to have assurances of being maintained, that's simply not feasible without funding. I don't see how any discussion here about who hosts the code is going to change that. At best, it provides an illusion of future support to users, which is misleading at best and a liability issue at worst.

1 Like