Large Proposal: Non-Standard Libraries

I do not agree with Ted that only a few projects should be ranked,
everyone, as it is in npm should be available. Only be graded according to
recommendations.

I’m a bit confused. I’m not sure what comments of mine I’m referring to.

Clearly I’m double confused. That meant to read “I’m not sure what
comments of mine *you* are referring to”.

Now we are triple confused.

I just read your comment again and i misinterpreted one part. Sorry.

···

Em qua, 8 de nov de 2017 às 17:58, Ted Kremenek <kremenek@apple.com> escreveu:

On Nov 8, 2017, at 11:40 AM, Ted Kremenek via swift-evolution < > swift-evolution@swift.org> wrote:
On Nov 8, 2017, at 4:30 AM, Wallacy via swift-evolution < > swift-evolution@swift.org> wrote:

I fully support having a broad spectrum of libraries that the community
builds and uses. Any library that we decide to make part of “core Swift” —
IMHO at a mature point in a library’s evolution — would need to have high
value to the majority of the community and would need to feel solid enough
that we can lock it in for both source and binary compatibility, high
quality of implementation with sustained maintenance, etc.

These are some really interesting analogies. How would you imagine the community “governance” of these “plugins” (which I assume would be libraries or packages) to be managed? What does it mean for the “full community” to manage them, and provide the rough guarantees you suggest?

I like the general premise here: support a library ecosystem for Swift, with possibly layers of expectations and guarantees on the quality of those libraries and the features they support.

A related question, what can we do to how the communication discussion via Swift.org <http://swift.org/&gt; can be used to facilitate building such a library ecosystem? Kelvin Ma made a great point about shouting out on the mailing list for a project and getting no real response. I’m getting optimistic that we can move to a forum soon; could we use that forum in a way to help facilitate something like what you are proposing?

···

On Nov 8, 2017, at 4:30 AM, Wallacy via swift-evolution <swift-evolution@swift.org> wrote:

@Félix Fischer

Yes, I wanted to say central index. I chose the wrong term.

Actually I could have summed it up like this:
- Central Index
- SwiftPM to download/search using this index like npm/nuget
- GStreamer organization Style.

I do not agree with Ted that only a few projects should be ranked, everyone, as it is in npm should be available. Only be graded according to recommendations.

For exemple this is what GStreamer do:

gst-plugins-base
a small and fixed set of plug-ins, covering a wide range of possible types of elements; these are continuously kept up-to-date with any core changes during the development series.

We believe distributors can safely ship these plug-ins
People writing elements should base their code on these elements
These elements come with examples, documentation, and regression tests
gst-plugins-good
a set of plug-ins that we consider to have good quality code, correct functionality, our preferred license (LGPL for the plug-in code, LGPL or LGPL-compatible for the supporting library).

We believe distributors can safely ship these plug-ins
People writing elements should base their code on these elements
gst-plugins-ugly
a set of plug-ins that have good quality and correct functionality, but distributing them might pose problems. The license on either the plug-ins or the supporting libraries might not be how we'd like. The code might be widely known to present patent problems.

Distributors should check if they want/can ship these plug-ins
People writing elements should base their code on these elements
gst-plugins-bad
a set of plug-ins that aren't up to par compared to the rest. They might be close to being good quality, but they're missing something - be it a good code review, some documentation, a set of tests, a real live maintainer, or some actual wide use. If the blanks are filled in they might be upgraded to become part of either gst-plugins-good or gst-plugins-ugly, depending on the other factors.

If the plug-ins break, you can't complain - instead, you can fix the problem and send us a patch, or bribe someone into fixing them for you
New contributors can start here for things to work on

For example, let's say that the proposal on type Result <T> is not approved.

The community cam build a Result<T> "project" ate "swift-extension-base" and everyone at "base" level must agree to use then (or improve then) to avoid one of the concerns at Result<T> proposal (several projects build the own)...

So in Swift side maybe the indexer (and the community) can classify projects in this way:

swift-extension-base // (Swift Core Team maybe can manager this one)

a small and fixed set of features, covering a small range of possible types of elements; these are continuously kept up-to-date with any core changes during the development series.

We believe distributors can safely ship these features
People writing elements should base their code on these elements
These elements come with examples, documentation, and regression tests
swift-extension-good // (Community can manage this, and Swift Core Team can do some advices)
a set of features that we consider to have good quality code, correct functionality, our preferred license (Swift Licence), and covering a wide range features.

We believe distributors can safely ship these plug-ins
People writing elements should base their code on these elements
swift-extension-ugly // (Full Community)
a set of features that have good quality and correct functionality, but distributing them might pose problems. The license on either the or the supporting libraries might not be how we'd like. The code might be widely known to present patent problems.

Distributors should check if they want/can ship these plug-ins
People writing elements should base their code on these elements
swift-extension-bad // (Full Community)
a set of features that aren't up to par compared to the rest. They might be close to being good quality, but they're missing something - be it a good code review, some documentation, a set of tests, a real live maintainer, or some actual wide use.

If the features break, you can't complain - instead, you can fix the problem and send us a patch, or bribe someone into fixing them for you
New contributors can start here for things to work on

I just change plugins to feature to make more sense here... And at "base" level we can just drop very small projects, and make few guarantees that will always work. And at good level we can drop bigger projects like Alamofire, Perfect etc, because of the nature of those projects, we know that will get a good support but we cant guarantee that documentation and tests and language updates will always be pair with the rest.

And at "bad" level, will be like NPM is today... just a easy way to obtain some 3rd party code but no guarantees at all!

We heve IBM Swift Package Catalog (https://packagecatalog.com <https://packagecatalog.com/&gt;\) but i think we can do better.

Also, if we (and Apple) do nothing, the community someway will do! It's like Brew/Macports at some point people will build the tools to make the ecosystem easier. Is better make something oficial (like npm/nuget) of course.

A "Non-Standard Libraries" its a non start for me, but good indexer, a good integration with SwiftPM, a oficial help/support of the Swift Core Team, and a set of rules/patterns to rule this (and other) projects is a good feature to have.

Em ter, 7 de nov de 2017 às 21:16, Alejandro Martinez via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> escreveu:
I’m with Ted on this one.
I would also love to see the Swift ecosystem grow but I think that it has to happen with SPM. With improvements on SPM (as discussed in other threads) and having a proper index (imho Cocoapods webpage is the best one out there, with stats about docs, unit testing, downloads...) that makes finding libraries easier.
But also a discussion forum where great libs can be shared and highlighted to the rest of the community and big community efforts coordinated. If you look at Rust is a great example, their forums attract the whole community (sadly not something the mailing list does) and big and important projects have come up from the community that have made a huge impact. With that in place and some libs out there maturing I’m sure that in the future the conversation to include battle tested code into an oficial distribution would be easier.

Sent from my iPad

On 7 Nov 2017, at 21:58, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi Dave,

Thanks for bringing up this topic. This has been kicked around a little, and we’re still exploring different models on how to extend Swift.

The server APIs work group is one operational model for the community to build out a new set of core libraries. That work group was formed out of the observation that there were different implementations of the same thing being created by different Swift on server efforts, and that those different efforts should pool those resources together and standardize on some fundamentals.

That analogy could work here. However, it also has possible major downsides. It can be heavyweight, and also artificially raise the importance of certain people’s library efforts over others by creating a formal work group whose efforts are expected to eventually be incorporated into the core Swift distribution. It would also force a discussion up front of what even makes sense to incorporate into the core Swift distribution, which might be artificially constraining the exploration of the set of libraries to implement.

I need to think about your idea more, but my first reaction is that my preferred route is that the community builds these libraries, ideally using Swift Packages, and through trial and use we evaluate those libraries and then decide if they should be incorporated as part of the core distribution. There’s many factors involved in deciding if they can be incorporated into the core distribution, but all of those could be informed by actually building libraries and see them getting used.

We could also figure out a way to possibly highlight these efforts to the Swift community, maybe on swift-evolution or other means — but all with the expectation that these libraries are not *necessarily* going to be standardized as part of the core swift distribution. I don’t think that’s all that bad either; not every library will make sense to incorporate into the core Swift distribution (e.g., they are highly domain specific) but still supporting their development would be beneficial to the community.

Note that any change going into the Swift core distribution inevitably involves swift-evolution and the core team. Changes going into the core Swift distribution must meet a very high bar as far as implementation and design, the confidence we have into committing to specific APIs (especially since we’ll have ABI stability in Swift 5), and other factors. Thus this will not really alleviate much burden on the Core team or on the community — one of the core goals of your proposal. Further, incorporating all those concerns up front when building libraries might be counterproductive.

FWIW, Ben Cohen and I have been talking about possibly using Swift packages as a way to seed out experimental ideas for extensions to the Standard Library. This would allow ideas to be trialed by real usage (a complaint I’ve seen about some changes we’ve made to Swift in the past). Users could build things on top of those libraries, knowing they are available as packages, and if an API “graduates” to being part of the Standard Library the user can then depend upon it being available there. If it never graduates, however, the package remains around.

One thing that resonates me in what you propose is about having a “well-organized effort” whose aim is to make these libraries feel cohesive and implemented well. However, given that many of these naturally fall somewhere in the spectrum of responsibilities within the Standard Library and Foundation, I think it is self-misleading to think that the Core Team or others would not be involved in these efforts if the intention is to possibly incorporate these one day into the core Swift distribution. There also may be other ways to achieve that level of cohesion in API design, such as through community discussion (possibly via the Swift.org <http://swift.org/&gt; mailing lists/forums). This discussion would not necessarily be the same as the path to “ratification” of a library into core Swift, but a step that could benefit many library authors (including considering ideas one day incorporated into the core swift). With such a mechanism many library authors could benefit even if they do not intend to have the library as part of the core distribution.

My apologies that these are all hand-wavy ideas, but I’m trying to paint a possible alternative way to achieve your goal of more library evolution for Swift without having such a formal work group that may be too heavy weight or too narrowly focused.

Ted

On Nov 7, 2017, at 9:54 AM, Dave DeLong via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi Swift-Evolution,

The Standard Library's goal is to be small and targeted. However, many aspects of Apple-provided frameworks need or offer opportunities for improvement or wholesale replacement. These enhancements lie beyond the scope of the Standard Library.

To address this, we'd like to propose the idea of a "Non-Standard Library"; this would be a library that ships with a regular installation of Swift, but is not imported into .swift files by the compiler, unless explicitly requested by the developer.

We are proposing a well-organized effort to parallel the Standard Library without putting additional implementation responsibilities onto the core team. This effort would mitigate what we see as platform-independent requirements that provide native Swift implementations that aren't burdened by Apple history.

Mission Statement

We propose to create an open-sourced "Non-Standard Library" effort that coexists with, coordinates with, and is blessed by the open source Swift development community. The "Non-Standard Library" will offer a well-curated, high-value collection of routines, types, and documentation to support Swift development on all platforms.

Goals

The main goals of this effort would be:
Alleviate pressure on the Core Team while providing the developer community with functionality that normally falls under Apple's internal development umbrella.
Deliver authoritative, reliable, and regularly updated libraries avoiding issues faced by third-party dependencies.
Provide oversight, organization, and full community involvement to ensure its components are worthy, maintained, and passing a bar of need and excellence.
Suggested Libraries

There are several areas we think that are ripe for improvement and implementation as a non-standard library, such as (but not limited to):
A BigNum library
A full-featured Random library
A simplified date/time library
A library for manipulating paths (that is not based on URL or String)
An expanded Swift-native Geometry library and so forth.
The scope and extent of the sublibraries would be proposed and debated on a parallel Non-Standard Library Evolution development list.

Coordination

This effort would be fully coordinated with the Swift development team, respecting the team's existing commitments and responsibilities. We would request an Apple body to act as an official coordinator, enabling both oversight of this effort and to expose Apple-sourced suggestions to the Non-Standard community for action.

Next Steps

To proceed, we need a general community consensus that this effort is worth the significant overhead it would involve.

We must then:
Select a project lead, who appoints technical leaders from the community.
Recruit a core team, a small group responsible for strategic direction, pulled from experienced members well versed in contributing to Swift-dev.
Establish a Non-Standard Swift Evolution process, based on the one that is currently followed by Swift Evolution. Following SE practices will guarantee a consistent and excellent language experience for those developers including the Non-Standard library into their projects.
Build a Non-Standard Swift Evolution repository home.
Adopt a code license, based on Swift's current licensing.
Define the community working group rules and code of conduct.
Establish a mailing list and/or forum.
Begin the selection of target sublibraries and populating them by recruiting committers and contributors.
Alternative Names

Suggested names for this effort include the following.
Extended-Standard Library
Community-Standard Library
Non-Standard Library
We are not married to any of these names and welcome alternative suggestions.

Measures of Success

A successful Non-Standard Library will provide a stable and incrementally grown ecology of Swift frameworks that developers can depend upon with a minimum of turbulence and regret cycles. For that reason, the most successful content will be core functionality, with significant field testing prior to adoption. We recommend that NSSE follow a model of provisionary adoption and refinement before deployment to ensure any adopting code base will not be affected by later module changes.

Thanks,

Dave DeLong and Erica Sadun
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I agree that the tooling support around SwiftPM is not sufficiently advanced yet to support this for everybody. Further, I don’t think there would be a need to preclude other ways to share libraries for this purpose, even if the SwiftPM tooling support was more mature.

The primary point I wanted to make was more about the model itself. I’d prefer the community grow up a set of libraries that trialed and used before focusing on prematurely baking them into the core Swift distribution.

···

On Nov 8, 2017, at 4:54 AM, Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On Nov 7, 2017, at 1:58 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

FWIW, Ben Cohen and I have been talking about possibly using Swift packages as a way to seed out experimental ideas for extensions to the Standard Library. This would allow ideas to be trialed by real usage (a complaint I’ve seen about some changes we’ve made to Swift in the past). Users could build things on top of those libraries, knowing they are available as packages, and if an API “graduates” to being part of the Standard Library the user can then depend upon it being available there. If it never graduates, however, the package remains around.

Yeah this is exactly the problem that the package manager is there to solve, right? It’s supposed to make it ridiculously easy to integrate libraries and manage your dependencies.

The problem is that most people writing Swift code every day are doing it to make graphical applications on iOS/macOS. SwiftPM doesn’t support those, so if I want to test a library, it’s just a one-off thing that I play with in a Playground.

I think that the best thing we could do to encourage people to write, use and contribute to public libraries would be to improve the package manager. SwiftPM is still basically a toy (or an interesting curiosity), until it can actually be used in the projects most Swift devs get paid to work on every day. Talking about it supporting a community is way premature; it’s not even close to ready to taking on that responsibility, IMO.

Huge +1

Dealing with path manipulations via Foundation is a real pain on Linux. I
end up using a lot of C code from Glibc. And random support is another pain
point.

There are tons of uses for a non-std library, however, i would like to see
python style imports (if possible) to avoid importing a potentially large
library when i may only need one or two functions from it. I hate importing
all of foundation just so i csn work with Dates.

Same thing would be handy in a non-std library. I wont do much with
geometry or many other things, but i need a secure random number generator?
`import NonStd.SecureRandom`
Now I’ve got what I need for generating cryptographically secure random
numbers, but none of the additional bloat.

This is probably something that would require an additional proposal, and
I’m not even sure if it would be possible with the way swift compiles
frameworks, but it would be nice

···

On Wed, Nov 8, 2017 at 5:54 AM Karl Wagner via swift-evolution < swift-evolution@swift.org> wrote:

On Nov 7, 2017, at 1:58 PM, Ted Kremenek via swift-evolution < > swift-evolution@swift.org> wrote:

FWIW, Ben Cohen and I have been talking about possibly using Swift
packages as a way to seed out experimental ideas for extensions to the
Standard Library. This would allow ideas to be trialed by real usage (a
complaint I’ve seen about some changes we’ve made to Swift in the past).
Users could build things on top of those libraries, knowing they are
available as packages, and if an API “graduates” to being part of the
Standard Library the user can then depend upon it being available there.
If it never graduates, however, the package remains around.

Yeah this is exactly the problem that the package manager is there to
solve, right? It’s supposed to make it ridiculously easy to integrate
libraries and manage your dependencies.

The problem is that most people writing Swift code every day are doing it
to make graphical applications on iOS/macOS. SwiftPM doesn’t support those,
so if I want to test a library, it’s just a one-off thing that I play with
in a Playground.

I think that the best thing we could do to encourage people to write, use
and contribute to public libraries would be to improve the package manager.
SwiftPM is still basically a toy (or an interesting curiosity), until it
can actually be used in the projects most Swift devs get paid to work on
every day. Talking about it supporting a community is way premature; it’s
not even close to ready to taking on that responsibility, IMO.

- Karl
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

These are great points.

FWIW, I’m getting optimistic about moving to a forum soon. Would you expect that a forum could provide a better vehicle than a mailing list to arrange communication and interest within the community around building libraries? Not just doing shout outs for projects, but also doing possible API design review, etc.?

As an analogy, within Apple we have various mailing lists to review APIs, which is one mechanism used for different teams to co-review newly proposed APIs and consider how they compose together with other APIs. It’s not always perfect, but it does help facilitate a culture of API review so that various APIs can be considered together and part of the same (or compatible) design philosophies.

One of the things that resonated to me from Dave DeLong’s proposal was a sense about having a set of libraries that are well-considered and their efforts coordinated. While the coordination pitched in Dave’s proposal was about a focused effort on a particular set of libraries/features, coordination can also take the form of having a community that cares about building good APIs and can constructively discuss them. This can be done while also completely factoring out whether or not those APIs are part of “core Swift”. Further, shared API review wouldn’t necessarily be about making actual decisions — which is the case of swift-evolution when evaluating language and standard library changes — but offering advice. Fundamentally the library author still stays in control of their library and APIs, but the community could help in shaping up the gestalt of what are considered well-crafted Swift APIs in general.

Of course the big difference here with this idea compared to Apple’s internal API review process is that for Apple the APIs it vends are intended to be shipped together, and thus they must work together. In open source, however, efforts on various libraries are often (usually?) independent. Projects are usually created independently by different authors, and while it may be desirable for APIs from various libraries to feel natural to work with together, it’s not a requirement on their construction in general.

···

On Nov 8, 2017, at 12:08 PM, Kelvin Ma <kelvin13ma@gmail.com> wrote:

On Wed, Nov 8, 2017 at 1:58 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Nov 8, 2017, at 11:40 AM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Nov 8, 2017, at 4:30 AM, Wallacy via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I do not agree with Ted that only a few projects should be ranked, everyone, as it is in npm should be available. Only be graded according to recommendations.

I’m a bit confused. I’m not sure what comments of mine I’m referring to.

Clearly I’m double confused. That meant to read “I’m not sure what comments of mine *you* are referring to”.

I fully support having a broad spectrum of libraries that the community builds and uses. Any library that we decide to make part of “core Swift” — IMHO at a mature point in a library’s evolution — would need to have high value to the majority of the community and would need to feel solid enough that we can lock it in for both source and binary compatibility, high quality of implementation with sustained maintenance, etc.

i mean I don’t think these approaches are incompatible. The “swift core” could just make the process of independent libraries getting started easier. Like right now there’s really no place to say “hey I just started a library project for X, and anyone who wants to be involved should contribute at Y github repo where it lives right now”. I’ve tried sending that on this list before and it didn’t really work because mailing lists aren’t really a good medium for that and no one wants the swift-evolution list getting clogged with project-specific messages most people don’t care about.

I think there are two ideas discussed in this thread at the same time. One
is for a more extended standard library, that is developed and shipped as
part of the language. This is similar to what Python, Go and other
languages are doing. The second is for a more wide collection of packages,
developed by 3rd parties and not included with the language. This is
similar to npm, PyPI, etc.

If I understood it correctly, the original proposal is about the first idea
and I don't think an "open market" approach works well for this. As for
what should be included, I find Go to be a nice example, that could be used
as a starting point:

https://golang.org/pkg/

To summarize, it includes:
- archival and compression formats (tar, zip, gzip, ...)
- a few data structures
- cryptographic functions (including hashes and random numbers)
- sqlite and basic database drivers support
- various data encodings (CSV, XML, JSON, ...)
- some stuff used internally by the go tools (AST trees, debugging symbols)
- basic graphics and image support
- basic text and HTML templates
- math and bignums
- networking and HTTP support (client and server)
- OS support (including path handling, command line flags etc)
- Miscellanous stuff like dates, unicode, IO, string parsing, testing, etc.

Go considers all these packages part of the language and offers source
compatibility guarantees for all of them. I find the availability of these
packages and the fact that they are developed and maintained by the Go core
team to be very powerful.

We can debate what should be included and if these should be separate
modules or not, but I think the key insight here is that they are a curated
list of useful libraries that are developed and maintained together with
the language.

···

On Wed, Nov 8, 2017 at 9:37 PM, Ted Kremenek via swift-evolution < swift-evolution@swift.org> wrote:

On Nov 8, 2017, at 4:54 AM, Karl Wagner <razielim@gmail.com> wrote:

On Nov 7, 2017, at 1:58 PM, Ted Kremenek via swift-evolution < > swift-evolution@swift.org> wrote:

FWIW, Ben Cohen and I have been talking about possibly using Swift
packages as a way to seed out experimental ideas for extensions to the
Standard Library. This would allow ideas to be trialed by real usage (a
complaint I’ve seen about some changes we’ve made to Swift in the past).
Users could build things on top of those libraries, knowing they are
available as packages, and if an API “graduates” to being part of the
Standard Library the user can then depend upon it being available there.
If it never graduates, however, the package remains around.

Yeah this is exactly the problem that the package manager is there to
solve, right? It’s supposed to make it ridiculously easy to integrate
libraries and manage your dependencies.

The problem is that most people writing Swift code every day are doing it
to make graphical applications on iOS/macOS. SwiftPM doesn’t support those,
so if I want to test a library, it’s just a one-off thing that I play with
in a Playground.

I think that the best thing we could do to encourage people to write, use
and contribute to public libraries would be to improve the package manager.
SwiftPM is still basically a toy (or an interesting curiosity), until it
can actually be used in the projects most Swift devs get paid to work on
every day. Talking about it supporting a community is way premature; it’s
not even close to ready to taking on that responsibility, IMO.

I agree that the tooling support around SwiftPM is not sufficiently
advanced yet to support this for everybody. Further, I don’t think there
would be a need to preclude other ways to share libraries for this purpose,
even if the SwiftPM tooling support was more mature.

The primary point I wanted to make was more about the model itself. I’d
prefer the community grow up a set of libraries that trialed and used
before focusing on prematurely baking them into the core Swift distribution.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

1 Like

These are some really interesting analogies. How would you imagine the
community “governance” of these “plugins” (which I assume would be
libraries or packages) to be managed?

Pretty much like a TV/Games/etc community forums. The owner (In this case
Apple/Core Team) invite some people to help (Moderators etc) to maintain
the main project, look if everyone is follow the rules, etc.

Also is similar as other opensource projects.

What does it mean for the “full community” to manage them, and provide the
rough guarantees you suggest?

NPM has thousands of projects, is impossible to check all of then. And
besides be a good reference, here we have the opportunity do to something
better... Because...Because...Because I believe we can ;)

So what i expect?

Lets make some assumptions here, just to play a little:

Let assume we build a Indexer like IBM Swift Package Catalog (
https://packagecatalog.com/ ) over Swift.org.

And like Swift Source Compatibility Suite to be "indexed" the author of the
project needs to send the request to be accepted.

Lets assume the same rules as Compatibility Suite:
Acceptance Criteria

To be accepted into the Swift source compatibility test suite, a project
must:

   1. Target Linux, macOS, or iOS/tvOS/watchOS device
   2. Be an *Xcode* or *Swift Package Manager* project (Carthage and
   CocoaPods are currently unsupported but are being explored to be supported
   in the future)
   3. Support building on either Linux or macOS
   4. Be contained in a publicly accessible git repository
   5. Maintain a project branch that builds against Swift 3.0 compatibility
   mode and passes any unit tests
   6. Have maintainers who will commit to resolve issues in a timely manner
   7. Be compatible with the latest GM/Beta versions of *Xcode* and
   *swiftpm*
   8. Add value not already included in the suite
   9. Be licensed with one of the following permissive licenses:
      - BSD
      - MIT
      - Apache License, version 2.0
      - Eclipse Public License
      - Mozilla Public License (MPL) 1.1
      - MPL 2.0
      - CDDL

And lets assume like GStreamer we have 4 categories of projects.

Base; Good; Bad; Unsupported. (just as exemple)

*swift-extension-base *
a small and fixed set of features, covering a small range of possible types
of elements; these are continuously kept up-to-date with any core changes
during the development series.

   - We believe distributors can safely ship these features
   - People writing elements should base their code on these elements
   - These elements come with examples, documentation, and regression tests

*swift-extension-good *a set of features that we consider to have good
quality code, correct functionality, our preferred license (Swift Licence),
and covering a wide range features.

   - We believe distributors can safely ship these plug-ins
   - People writing elements should base their code on these elements

*swift-extension-bad *
a set of features that have good quality and correct functionality, but
distributing them might pose problems. The license on either the or the
supporting libraries might not be how we'd like. The code might be widely
known to present patent problems.

   - Distributors should check if they want/can ship these plug-ins
   - People writing elements should base their code on these elements

*swift-extension-unsupported *
a set of features that aren't up to par compared to the rest. They might be
close to being good quality, but they're missing something - be it a good
code review, some documentation, a set of tests, a real live maintainer, or
some actual wide use.

   - If the features break, you can't complain - instead, you can fix the
   problem and send us a patch, or bribe someone into fixing them for you
   - New contributors can start here for things to work on

After request to be accepted is made, let's assume some area using forums
or similar to make that request happen, the project will be classified in
one of those categories, and will have people to look if projects accepted
on "base"/"good"/"bad" have the minimum requirements to keep on those
categories. Unsupported can be used for projects are not maintained any
more, or does not follow all rules, but at least one of then.

How?

At base/good level wee need only a few projects? 10-100? I don't know, but
a sane number to according to the number of people who are available to
monitor them, a project on this level may even have no commit for a while.
At "bad" level we can have much more projects and only rely to automatic
checks like, number of downloads, starts, comments, commits etc to see if
this project is maintened (packagecatalog give us a few metrics too). If
the project owner believes that your project is relevant enough to be
considered "good" needs to make a request.

In this cenário, you can have your project automatically downgraded to
unsupported group, but to level-up you need to do a request and we will see
if meet the requirement of that level.

SwiftPM should default only to base/good level:

$: SwiftPM install BigInt

for bad:

$: SwiftPM --enable-bad-projects install SomeCrazyLib

In this cenário we will not actually have a extremely big indexer, this is
limited by definition, but the the community will "battle" to make relevant
projects to be accept in this "oficial" indexer. And actually, we already
have few of then now.

And... well, a BigInt, a Future Lib, of entire HTTP server can be indexed
here, and because in this case "quality matter", this can be assumed to be
a "non-standard" library in some way. Several projects, solving several
problems, without the craziness of "npm/nuget"

···

Em qui, 9 de nov de 2017 às 03:42, Ted Kremenek <kremenek@apple.com> escreveu:

I like the general premise here: support a library ecosystem for Swift,
with possibly layers of expectations and guarantees on the quality of those
libraries and the features they support.

A related question, what can we do to how the communication discussion via
Swift.org can be used to facilitate building such a library ecosystem?
Kelvin Ma made a great point about shouting out on the mailing list for a
project and getting no real response. I’m getting optimistic that we can
move to a forum soon; could we use that forum in a way to help facilitate
something like what you are proposing?

On Nov 8, 2017, at 4:30 AM, Wallacy via swift-evolution < > swift-evolution@swift.org> wrote:

@Félix Fischer

Yes, I wanted to say central index. I chose the wrong term.

Actually I could have summed it up like this:
- Central Index
- SwiftPM to download/search using this index like npm/nuget
- GStreamer organization Style.

I do not agree with Ted that only a few projects should be ranked,
everyone, as it is in npm should be available. Only be graded according to
recommendations.

For exemple this is what GStreamer do:

*gst-plugins-base*
a small and fixed set of plug-ins, covering a wide range of possible types
of elements; these are continuously kept up-to-date with any core changes
during the development series.

   - We believe distributors can safely ship these plug-ins
   - People writing elements should base their code on these elements
   - These elements come with examples, documentation, and regression
   tests

*gst-plugins-good*
a set of plug-ins that we consider to have good quality code, correct
functionality, our preferred license (LGPL for the plug-in code, LGPL or
LGPL-compatible for the supporting library).

   - We believe distributors can safely ship these plug-ins
   - People writing elements should base their code on these elements

*gst-plugins-ugly*
a set of plug-ins that have good quality and correct functionality, but
distributing them might pose problems. The license on either the plug-ins
or the supporting libraries might not be how we'd like. The code might be
widely known to present patent problems.

   - Distributors should check if they want/can ship these plug-ins
   - People writing elements should base their code on these elements

*gst-plugins-bad*
a set of plug-ins that aren't up to par compared to the rest. They might
be close to being good quality, but they're missing something - be it a
good code review, some documentation, a set of tests, a real live
maintainer, or some actual wide use. If the blanks are filled in they might
be upgraded to become part of either gst-plugins-good or gst-plugins-ugly,
depending on the other factors.

   - If the plug-ins break, you can't complain - instead, you can fix the
   problem and send us a patch, or bribe someone into fixing them for you
   - New contributors can start here for things to work on

For example, let's say that the proposal on type Result <T> is not
approved.

The community cam build a Result<T> "project" ate "swift-extension-base"
and everyone at "base" level must agree to use then (or improve then) to
avoid one of the concerns at Result<T> proposal (several projects build the
own)...

So in Swift side maybe the indexer (and the community) can classify
projects in this way:

*swift-extension-base *// (Swift Core Team maybe can manager this one)

*a small and fixed set of features, covering a small range of possible
types of elements*; these are continuously kept up-to-date with any core
changes during the development series.

   - We believe distributors can safely ship these features
   - People writing elements should base their code on these elements
   - These elements come with examples, documentation, and regression
   tests

*swift-extension-good *// (Community can manage this, and Swift Core Team
can do some advices)
a set of *features *that we consider to have good quality code, correct
functionality, our preferred license* (Swift Licence)*, and *covering a
wide range features.*

   - We believe distributors can safely ship these plug-ins
   - People writing elements should base their code on these elements

*swift-extension-ugly *// (Full Community)
a set of *features *that have good quality and correct functionality,
but distributing them might pose problems. The license on either the or the
supporting libraries might not be how we'd like. The code might be widely
known to present patent problems.

   - Distributors should check if they want/can ship these plug-ins
   - People writing elements should base their code on these elements

*swift-extension-bad *// (Full Community)
a set of *features *that aren't up to par compared to the rest. They
might be close to being good quality, but they're missing something - be it
a good code review, some documentation, a set of tests, a real live
maintainer, or some actual wide use.

   - If the *features *break, you can't complain - instead, you can fix
   the problem and send us a patch, or bribe someone into fixing them for you
   - New contributors can start here for things to work on

I just change plugins to feature to make more sense here... And at "base"
level we can just drop very small projects, and make few guarantees that
will always work. And at good level we can drop bigger projects like
Alamofire, Perfect etc, because of the nature of those projects, we know
that will get a good support but we cant guarantee that documentation and
tests and language updates will always be pair with the rest.

And at "bad" level, will be like NPM is today... just a easy way to obtain
some 3rd party code but no guarantees at all!

We heve IBM Swift Package Catalog (https://packagecatalog.com) but i
think we can do better.

Also, if we (and Apple) do nothing, the community someway will do! It's
like Brew/Macports at some point people will build the tools to make the
ecosystem easier. Is better make something oficial (like npm/nuget) of
course.

A "Non-Standard Libraries" its a non start for me, but good indexer, a
good integration with SwiftPM, a oficial help/support of the Swift Core
Team, and a set of rules/patterns to rule this (and other) projects is a
good feature to have.

Em ter, 7 de nov de 2017 às 21:16, Alejandro Martinez via swift-evolution < > swift-evolution@swift.org> escreveu:

I’m with Ted on this one.
I would also love to see the Swift ecosystem grow but I think that it has
to happen with SPM. With improvements on SPM (as discussed in other
threads) and having a proper index (imho Cocoapods webpage is the best one
out there, with stats about docs, unit testing, downloads...) that makes
finding libraries easier.
But also a discussion forum where great libs can be shared and
highlighted to the rest of the community and big community efforts
coordinated. If you look at Rust is a great example, their forums attract
the whole community (sadly not something the mailing list does) and big and
important projects have come up from the community that have made a huge
impact. With that in place and some libs out there maturing I’m sure that
in the future the conversation to include battle tested code into an
oficial distribution would be easier.

Sent from my iPad

On 7 Nov 2017, at 21:58, Ted Kremenek via swift-evolution < >> swift-evolution@swift.org> wrote:

Hi Dave,

Thanks for bringing up this topic. This has been kicked around a little,
and we’re still exploring different models on how to extend Swift.

The server APIs work group is one operational model for the community to
build out a new set of core libraries. That work group was formed out of
the observation that there were different implementations of the same thing
being created by different Swift on server efforts, and that those
different efforts should pool those resources together and standardize on
some fundamentals.

That analogy could work here. However, it also has possible major
downsides. It can be heavyweight, and also artificially raise the
importance of certain people’s library efforts over others by creating a
formal work group whose efforts are expected to eventually be incorporated
into the core Swift distribution. It would also force a discussion up
front of what even makes sense to incorporate into the core Swift
distribution, which might be artificially constraining the exploration of
the set of libraries to implement.

I need to think about your idea more, but my first reaction is that my
preferred route is that the community builds these libraries, ideally using
Swift Packages, and through trial and use we evaluate those libraries and
then decide if they should be incorporated as part of the core
distribution. There’s many factors involved in deciding if they can be
incorporated into the core distribution, but all of those could be informed
by actually building libraries and see them getting used.

We could also figure out a way to possibly highlight these efforts to the
Swift community, maybe on swift-evolution or other means — but all with the
expectation that these libraries are not *necessarily* going to be
standardized as part of the core swift distribution. I don’t think that’s
all that bad either; not every library will make sense to incorporate into
the core Swift distribution (e.g., they are highly domain specific) but
still supporting their development would be beneficial to the community.

Note that any change going into the Swift core distribution inevitably
involves swift-evolution and the core team. Changes going into the core
Swift distribution must meet a very high bar as far as implementation and
design, the confidence we have into committing to specific APIs (especially
since we’ll have ABI stability in Swift 5), and other factors. Thus this
will not really alleviate much burden on the Core team or on the community
— one of the core goals of your proposal. Further, incorporating all those
concerns up front when building libraries might be counterproductive.

FWIW, Ben Cohen and I have been talking about possibly using Swift
packages as a way to seed out experimental ideas for extensions to the
Standard Library. This would allow ideas to be trialed by real usage (a
complaint I’ve seen about some changes we’ve made to Swift in the past).
Users could build things on top of those libraries, knowing they are
available as packages, and if an API “graduates” to being part of the
Standard Library the user can then depend upon it being available there.
If it never graduates, however, the package remains around.

One thing that resonates me in what you propose is about having a
“well-organized effort” whose aim is to make these libraries feel cohesive
and implemented well. However, given that many of these naturally fall
somewhere in the spectrum of responsibilities within the Standard Library
and Foundation, I think it is self-misleading to think that the Core Team
or others would not be involved in these efforts if the intention is to
possibly incorporate these one day into the core Swift distribution. There
also may be other ways to achieve that level of cohesion in API design,
such as through community discussion (possibly via the Swift.org
<http://swift.org/&gt; mailing lists/forums). This discussion would not
necessarily be the same as the path to “ratification” of a library into
core Swift, but a step that could benefit many library authors (including
considering ideas one day incorporated into the core swift). With such a
mechanism many library authors could benefit even if they do not intend to
have the library as part of the core distribution.

My apologies that these are all hand-wavy ideas, but I’m trying to paint
a possible alternative way to achieve your goal of more library evolution
for Swift without having such a formal work group that may be too heavy
weight or too narrowly focused.

Ted

On Nov 7, 2017, at 9:54 AM, Dave DeLong via swift-evolution < >> swift-evolution@swift.org> wrote:

Hi Swift-Evolution,

The Standard Library's goal is to be small and targeted. However, many
aspects of Apple-provided frameworks need or offer opportunities for
improvement or wholesale replacement. These enhancements lie beyond the
scope of the Standard Library.

To address this, we'd like to propose the idea of a "Non-Standard
Library"; this would be a library that ships with a regular installation of
Swift, but is not imported into .swift files by the compiler, unless
explicitly requested by the developer.

We are proposing a well-organized effort to parallel the Standard Library
without putting additional implementation responsibilities onto the core
team. This effort would mitigate what we see as platform-independent
requirements that provide native Swift implementations that aren't
burdened by Apple history.

*Mission Statement*

We propose to create an open-sourced "Non-Standard Library" effort that
coexists with, coordinates with, and is blessed by the open source Swift
development community. The "Non-Standard Library" will offer a
well-curated, high-value collection of routines, types, and documentation
to support Swift development on all platforms.

*Goals*

The main goals of this effort would be:

   - Alleviate pressure on the Core Team while providing the developer
   community with functionality that normally falls under Apple's internal
   development umbrella.
   - Deliver authoritative, reliable, and regularly updated libraries
   avoiding issues faced by third-party dependencies.
   - Provide oversight, organization, and full community involvement to
   ensure its components are worthy, maintained, and passing a bar of need and
   excellence.

*Suggested Libraries*

There are several areas we think that are ripe for improvement and
implementation as a non-standard library, such as (but not limited to):

   - A BigNum library
   - A full-featured Random library
   - A simplified date/time library
   - A library for manipulating paths (that is not based
   on URL or String)
   - An expanded Swift-native Geometry library and so forth.

The scope and extent of the sublibraries would be proposed and debated on
a parallel Non-Standard Library Evolution development list.

*Coordination*

This effort would be fully coordinated with the Swift development team,
respecting the team's existing commitments and responsibilities. We would
request an Apple body to act as an official coordinator, enabling both
oversight of this effort and to expose Apple-sourced suggestions to
the Non-Standard community for action.

*Next Steps*

To proceed, we need a general community consensus that this effort is
worth the significant overhead it would involve.

We must then:

   - Select a project lead, who appoints technical leaders from the
   community.
   - Recruit a core team, a small group responsible for strategic
   direction, pulled from experienced members well versed in contributing to
   Swift-dev.
   - Establish a Non-Standard Swift Evolution process, based on the one
   that is currently followed by Swift Evolution. Following SE practices will
   guarantee a consistent and excellent language experience for those
   developers including the Non-Standard library into their projects.
   - Build a Non-Standard Swift Evolution repository home.
   - Adopt a code license, based on Swift's current licensing.
   - Define the community working group rules and code of conduct.
   - Establish a mailing list and/or forum.
   - Begin the selection of target sublibraries and populating them by
   recruiting committers and contributors.

*Alternative Names*

Suggested names for this effort include the following.

   - Extended-Standard Library
   - Community-Standard Library
   - Non-Standard Library

We are not married to any of these names and welcome alternative
suggestions.

*Measures of Success*

A successful Non-Standard Library will provide a stable and incrementally
grown ecology of Swift frameworks that developers can depend upon with a
minimum of turbulence and regret cycles. For that reason, the most
successful content will be core functionality, with significant field
testing prior to adoption. We recommend that NSSE follow a model of
provisionary adoption and refinement before deployment to ensure any
adopting code base will not be affected by later module changes.

Thanks,

Dave DeLong and Erica Sadun
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I do not agree with Ted that only a few projects should be ranked,
everyone, as it is in npm should be available. Only be graded according to
recommendations.

I’m a bit confused. I’m not sure what comments of mine I’m referring to.

Clearly I’m double confused. That meant to read “I’m not sure what
comments of mine *you* are referring to”.

I fully support having a broad spectrum of libraries that the community
builds and uses. Any library that we decide to make part of “core Swift” —
IMHO at a mature point in a library’s evolution — would need to have high
value to the majority of the community and would need to feel solid enough
that we can lock it in for both source and binary compatibility, high
quality of implementation with sustained maintenance, etc.

i mean I don’t think these approaches are incompatible. The “swift core”
could just make the process of independent libraries getting started
easier. Like right now there’s really no place to say “hey I just started a
library project for X, and anyone who wants to be involved should
contribute at Y github repo where it lives right now”. I’ve tried sending
that on this list before and it didn’t really work because mailing lists
aren’t really a good medium for that and no one wants the swift-evolution
list getting clogged with project-specific messages most people don’t care
about.

These are great points.

FWIW, I’m getting optimistic about moving to a forum soon. Would you
expect that a forum could provide a better vehicle than a mailing list to
arrange communication and interest within the community around building
libraries? Not just doing shout outs for projects, but also doing possible
API design review, etc.?

this forum thing has been talked about for almost a year now and i am for
real convinced it will never happen tbh. if it did, it would probably be
more conducive but ultimately it comes down to the number of people
interested because right now it’s like there’s only a handful of people on
this list who want to write open source libraries (or really anything
without a graphical interface since it’s basically 90% ios devs here). will
a forum encourage more people to get involved in the language? maybe, for
me it took a really long time to actually sign up for this mailing list and
talk in it because mailing lists are just not easy to get involved in

···

On Thu, Nov 9, 2017 at 12:11 AM, Ted Kremenek <kremenek@apple.com> wrote:

On Nov 8, 2017, at 12:08 PM, Kelvin Ma <kelvin13ma@gmail.com> wrote:
On Wed, Nov 8, 2017 at 1:58 PM, Ted Kremenek via swift-evolution < > swift-evolution@swift.org> wrote:

On Nov 8, 2017, at 11:40 AM, Ted Kremenek via swift-evolution < >> swift-evolution@swift.org> wrote:
On Nov 8, 2017, at 4:30 AM, Wallacy via swift-evolution < >> swift-evolution@swift.org> wrote:

As an analogy, within Apple we have various mailing lists to review APIs,
which is one mechanism used for different teams to co-review newly proposed
APIs and consider how they compose together with other APIs. It’s not
always perfect, but it does help facilitate a culture of API review so that
various APIs can be considered together and part of the same (or
compatible) design philosophies.

One of the things that resonated to me from Dave DeLong’s proposal was a
sense about having a set of libraries that are well-considered and their
efforts coordinated. While the coordination pitched in Dave’s proposal was
about a focused effort on a particular set of libraries/features,
coordination can also take the form of having a community that cares about
building good APIs and can constructively discuss them. This can be done
while also completely factoring out whether or not those APIs are part of
“core Swift”. Further, shared API review wouldn’t necessarily be about
making actual decisions — which is the case of swift-evolution when
evaluating language and standard library changes — but offering advice.
Fundamentally the library author still stays in control of their library
and APIs, but the community could help in shaping up the gestalt of what
are considered well-crafted Swift APIs in general.

Of course the big difference here with this idea compared to Apple’s
internal API review process is that for Apple the APIs it vends are
intended to be shipped together, and thus they must work together. In open
source, however, efforts on various libraries are often (usually?)
independent. Projects are usually created independently by different
authors, and while it may be desirable for APIs from various libraries to
feel natural to work with together, it’s not a requirement on their
construction in general.

i’m not one to applaud everything go does but its extended standard library
seems nice

I think there are two ideas discussed in this thread at the same time. One
is for a more extended standard library, that is developed and shipped as
part of the language. This is similar to what Python, Go and other
languages are doing. The second is for a more wide collection of packages,
developed by 3rd parties and not included with the language. This is
similar to npm, PyPI, etc.

If I understood it correctly, the original proposal is about the first
idea and I don't think an "open market" approach works well for this. As
for what should be included, I find Go to be a nice example, that could be
used as a starting point:

Standard library - Go Packages

To summarize, it includes:
- archival and compression formats (tar, zip, gzip, ...)

will be an open problem to port zlib to Swift, but it’s doable

- a few data structures

lots of incomplete implementations floating around GitHub, none really
stand out because i feel like most people have gotten used to rolling their
own since we still don’t have a Swift equivalent of std::priority_queue. at
this point i just have Queue.swift file lying around that I copy and paste
whenever i need it. which is bad.

- cryptographic functions (including hashes and random numbers)

This is being talked about right now, mostly about random number
generation, less on cryptography since it’s really easy to mess that up

- sqlite and basic database drivers support
- various data encodings (CSV, XML, JSON, ...)

I have an Linux-compatible XML library
<https://github.com/kelvin13/swiftxml&gt; in progress, last I remember the
Foundation class didn’t work on Linux

- some stuff used internally by the go tools (AST trees, debugging symbols)
- basic graphics and image support

i maintain a Swift PNG codec <https://github.com/kelvin13/maxpng&gt;, and am
developing a pure Swift JPEG codec <https://github.com/kelvin13/jpeg&gt;

- basic text and HTML templates

could really use these, don’t know of any cross-platform Swift libraries
for these

- math and bignums

peep GitHub - attaswift/BigInt: Arbitrary-precision arithmetic in pure Swift and
GitHub - xwu/NumericAnnex: Numeric facilities for Swift

- networking and HTTP support (client and server)

isn’t there a server working group?

- OS support (including path handling, command line flags etc)

YES. currently working on a new URI type <https://github.com/kelvin13/url&gt;
to replace the Foundation one which is really just a wrapper around NSURL
which is in turn a wrapper around CFURL. which is bad.

- Miscellanous stuff like dates, unicode, IO, string parsing, testing, etc.

yes

···

On Thu, Nov 9, 2017 at 2:24 AM, Nick Keets via swift-evolution < swift-evolution@swift.org> wrote:

Go considers all these packages part of the language and offers source
compatibility guarantees for all of them. I find the availability of these
packages and the fact that they are developed and maintained by the Go core
team to be very powerful.

We can debate what should be included and if these should be separate
modules or not, but I think the key insight here is that they are a curated
list of useful libraries that are developed and maintained together with
the language.

On Wed, Nov 8, 2017 at 9:37 PM, Ted Kremenek via swift-evolution < > swift-evolution@swift.org> wrote:

On Nov 8, 2017, at 4:54 AM, Karl Wagner <razielim@gmail.com> wrote:

On Nov 7, 2017, at 1:58 PM, Ted Kremenek via swift-evolution < >> swift-evolution@swift.org> wrote:

FWIW, Ben Cohen and I have been talking about possibly using Swift
packages as a way to seed out experimental ideas for extensions to the
Standard Library. This would allow ideas to be trialed by real usage (a
complaint I’ve seen about some changes we’ve made to Swift in the past).
Users could build things on top of those libraries, knowing they are
available as packages, and if an API “graduates” to being part of the
Standard Library the user can then depend upon it being available there.
If it never graduates, however, the package remains around.

Yeah this is exactly the problem that the package manager is there to
solve, right? It’s supposed to make it ridiculously easy to integrate
libraries and manage your dependencies.

The problem is that most people writing Swift code every day are doing it
to make graphical applications on iOS/macOS. SwiftPM doesn’t support those,
so if I want to test a library, it’s just a one-off thing that I play with
in a Playground.

I think that the best thing we could do to encourage people to write, use
and contribute to public libraries would be to improve the package manager.
SwiftPM is still basically a toy (or an interesting curiosity), until it
can actually be used in the projects most Swift devs get paid to work on
every day. Talking about it supporting a community is way premature; it’s
not even close to ready to taking on that responsibility, IMO.

I agree that the tooling support around SwiftPM is not sufficiently
advanced yet to support this for everybody. Further, I don’t think there
would be a need to preclude other ways to share libraries for this purpose,
even if the SwiftPM tooling support was more mature.

The primary point I wanted to make was more about the model itself. I’d
prefer the community grow up a set of libraries that trialed and used
before focusing on prematurely baking them into the core Swift distribution.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Just to comment on the idea that a forum will pull the community
together and improve this situation. A good and recent (as in young
language) example of it is Rust
announcements - The Rust Programming Language Forum . You can filter by that
category and see what the community is building, new libraries and
even calls for help to maintain them. Not saying that that specific
forum is the solution but just pointing that something different that
the mailing list would help. Also there is already a lot of people
that doesn't chime in in this discussions because they are not
interested in dealing with a mailing list (you can see it easily with
recent conversations that happen in GitHub instead of here).

I also like how the standard Go packages look, a rich set of
functionality *out of the box*. I really like that, is basically how
Apple platforms are always so nice to develop on with Foundation been
amazing. But unless the Core Team can oversee the development and
design of such tools I personally wouldn't feel confortable shipping
them with the rest of Swift. And it doesn't have to be, with
improvements on SPM there shouldn't be a big difference between
something that ships with Swift and a 3rd party package (apart from
being installed on the OS instead of coming with your binary I guess).
With that and having the community together, and a good place to
highlight them, I feel like the de facto standard libs are gonna pop
up with some time and care. Then and only ten would make sense to
consider making them part of the oficial distribution. (unless
obviously the Core Team would triple in size and they had infinite
time to do all of it, but then we also like when they improve the
language and tooling, can't have it all ^^)

Another thing that this list has to consider when not receiving
feedback on this kind of thing is that a lot of the Swift community is
more the iOS community, and that already has a lot of necessities
solved with other package manager, indexes, website that showcase
libs, newsletters, etc. The difficulty here is grow the part of the
community that is not iOS (or Apple platform) centric, which is key
for the future of world dominance.

Cheers

···

On Thu, Nov 9, 2017 at 8:54 AM, Kelvin Ma via swift-evolution <swift-evolution@swift.org> wrote:

i’m not one to applaud everything go does but its extended standard library
seems nice

On Thu, Nov 9, 2017 at 2:24 AM, Nick Keets via swift-evolution > <swift-evolution@swift.org> wrote:

I think there are two ideas discussed in this thread at the same time. One
is for a more extended standard library, that is developed and shipped as
part of the language. This is similar to what Python, Go and other languages
are doing. The second is for a more wide collection of packages, developed
by 3rd parties and not included with the language. This is similar to npm,
PyPI, etc.

If I understood it correctly, the original proposal is about the first
idea and I don't think an "open market" approach works well for this. As for
what should be included, I find Go to be a nice example, that could be used
as a starting point:

Standard library - Go Packages

To summarize, it includes:
- archival and compression formats (tar, zip, gzip, ...)

will be an open problem to port zlib to Swift, but it’s doable

- a few data structures

lots of incomplete implementations floating around GitHub, none really stand
out because i feel like most people have gotten used to rolling their own
since we still don’t have a Swift equivalent of std::priority_queue. at this
point i just have Queue.swift file lying around that I copy and paste
whenever i need it. which is bad.

- cryptographic functions (including hashes and random numbers)

This is being talked about right now, mostly about random number generation,
less on cryptography since it’s really easy to mess that up

- sqlite and basic database drivers support
- various data encodings (CSV, XML, JSON, ...)

I have an Linux-compatible XML library in progress, last I remember the
Foundation class didn’t work on Linux

- some stuff used internally by the go tools (AST trees, debugging
symbols)
- basic graphics and image support

i maintain a Swift PNG codec, and am developing a pure Swift JPEG codec

- basic text and HTML templates

could really use these, don’t know of any cross-platform Swift libraries for
these

- math and bignums

peep GitHub - attaswift/BigInt: Arbitrary-precision arithmetic in pure Swift and
GitHub - xwu/NumericAnnex: Numeric facilities for Swift

- networking and HTTP support (client and server)

isn’t there a server working group?

- OS support (including path handling, command line flags etc)

YES. currently working on a new URI type to replace the Foundation one which
is really just a wrapper around NSURL which is in turn a wrapper around
CFURL. which is bad.

- Miscellanous stuff like dates, unicode, IO, string parsing, testing,
etc.

yes

Go considers all these packages part of the language and offers source
compatibility guarantees for all of them. I find the availability of these
packages and the fact that they are developed and maintained by the Go core
team to be very powerful.

We can debate what should be included and if these should be separate
modules or not, but I think the key insight here is that they are a curated
list of useful libraries that are developed and maintained together with the
language.

On Wed, Nov 8, 2017 at 9:37 PM, Ted Kremenek via swift-evolution >> <swift-evolution@swift.org> wrote:

On Nov 8, 2017, at 4:54 AM, Karl Wagner <razielim@gmail.com> wrote:

On Nov 7, 2017, at 1:58 PM, Ted Kremenek via swift-evolution >>> <swift-evolution@swift.org> wrote:

FWIW, Ben Cohen and I have been talking about possibly using Swift
packages as a way to seed out experimental ideas for extensions to the
Standard Library. This would allow ideas to be trialed by real usage (a
complaint I’ve seen about some changes we’ve made to Swift in the past).
Users could build things on top of those libraries, knowing they are
available as packages, and if an API “graduates” to being part of the
Standard Library the user can then depend upon it being available there. If
it never graduates, however, the package remains around.

Yeah this is exactly the problem that the package manager is there to
solve, right? It’s supposed to make it ridiculously easy to integrate
libraries and manage your dependencies.

The problem is that most people writing Swift code every day are doing it
to make graphical applications on iOS/macOS. SwiftPM doesn’t support those,
so if I want to test a library, it’s just a one-off thing that I play with
in a Playground.

I think that the best thing we could do to encourage people to write, use
and contribute to public libraries would be to improve the package manager.
SwiftPM is still basically a toy (or an interesting curiosity), until it can
actually be used in the projects most Swift devs get paid to work on every
day. Talking about it supporting a community is way premature; it’s not even
close to ready to taking on that responsibility, IMO.

I agree that the tooling support around SwiftPM is not sufficiently
advanced yet to support this for everybody. Further, I don’t think there
would be a need to preclude other ways to share libraries for this purpose,
even if the SwiftPM tooling support was more mature.

The primary point I wanted to make was more about the model itself. I’d
prefer the community grow up a set of libraries that trialed and used before
focusing on prematurely baking them into the core Swift distribution.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Alejandro Martinez

Hello Ted, would you mind opening a new thread and post an update about the forum and maybe answer a few minor questions like:
- what is currently planned?
- in which timeframe we *might* see the forum finally happening (don’t have to be a promise)?
- what happens to old mailing lists? (I’d say put a public archive somewhere, maybe slightly better formatted than the mailing list is, but do not spam new categories so that we can start with a fresh and clean forum. I also wouldn’t bother with migrating old users to new accounts. If I’m interested in contribution I can take a minute and create a new forum account. I mean we’re not trying to sell here anything, so this shouldn’t be a high priority right!?)

Thank you. :)

···

Am 9. November 2017 um 07:12:10, Ted Kremenek via swift-evolution (swift-evolution@swift.org) schrieb:

On Nov 8, 2017, at 12:08 PM, Kelvin Ma <kelvin13ma@gmail.com> wrote:

On Wed, Nov 8, 2017 at 1:58 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

On Nov 8, 2017, at 11:40 AM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

On Nov 8, 2017, at 4:30 AM, Wallacy via swift-evolution <swift-evolution@swift.org> wrote:

I do not agree with Ted that only a few projects should be ranked, everyone, as it is in npm should be available. Only be graded according to recommendations.

I’m a bit confused. I’m not sure what comments of mine I’m referring to.

Clearly I’m double confused. That meant to read “I’m not sure what comments of mine *you* are referring to”.

I fully support having a broad spectrum of libraries that the community builds and uses. Any library that we decide to make part of “core Swift” — IMHO at a mature point in a library’s evolution — would need to have high value to the majority of the community and would need to feel solid enough that we can lock it in for both source and binary compatibility, high quality of implementation with sustained maintenance, etc.

i mean I don’t think these approaches are incompatible. The “swift core” could just make the process of independent libraries getting started easier. Like right now there’s really no place to say “hey I just started a library project for X, and anyone who wants to be involved should contribute at Y github repo where it lives right now”. I’ve tried sending that on this list before and it didn’t really work because mailing lists aren’t really a good medium for that and no one wants the swift-evolution list getting clogged with project-specific messages most people don’t care about.

These are great points.

FWIW, I’m getting optimistic about moving to a forum soon. Would you expect that a forum could provide a better vehicle than a mailing list to arrange communication and interest within the community around building libraries? Not just doing shout outs for projects, but also doing possible API design review, etc.?

As an analogy, within Apple we have various mailing lists to review APIs, which is one mechanism used for different teams to co-review newly proposed APIs and consider how they compose together with other APIs. It’s not always perfect, but it does help facilitate a culture of API review so that various APIs can be considered together and part of the same (or compatible) design philosophies.

One of the things that resonated to me from Dave DeLong’s proposal was a sense about having a set of libraries that are well-considered and their efforts coordinated. While the coordination pitched in Dave’s proposal was about a focused effort on a particular set of libraries/features, coordination can also take the form of having a community that cares about building good APIs and can constructively discuss them. This can be done while also completely factoring out whether or not those APIs are part of “core Swift”. Further, shared API review wouldn’t necessarily be about making actual decisions — which is the case of swift-evolution when evaluating language and standard library changes — but offering advice. Fundamentally the library author still stays in control of their library and APIs, but the community could help in shaping up the gestalt of what are considered well-crafted Swift APIs in general.

Of course the big difference here with this idea compared to Apple’s internal API review process is that for Apple the APIs it vends are intended to be shipped together, and thus they must work together. In open source, however, efforts on various libraries are often (usually?) independent. Projects are usually created independently by different authors, and while it may be desirable for APIs from various libraries to feel natural to work with together, it’s not a requirement on their construction in general.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

licenses like GPL. currently this is a huge hole in the SSCS which i am
sure will cause issues down the road

···

On Thu, Nov 9, 2017 at 12:37 PM, Wallacy via swift-evolution < swift-evolution@swift.org> wrote:

Em qui, 9 de nov de 2017 às 03:42, Ted Kremenek <kremenek@apple.com> > escreveu:

These are some really interesting analogies. How would you imagine the
community “governance” of these “plugins” (which I assume would be
libraries or packages) to be managed?

Pretty much like a TV/Games/etc community forums. The owner (In this case
Apple/Core Team) invite some people to help (Moderators etc) to maintain
the main project, look if everyone is follow the rules, etc.

Also is similar as other opensource projects.

What does it mean for the “full community” to manage them, and provide
the rough guarantees you suggest?

NPM has thousands of projects, is impossible to check all of then. And
besides be a good reference, here we have the opportunity do to something
better... Because...Because...Because I believe we can ;)

So what i expect?

Lets make some assumptions here, just to play a little:

Let assume we build a Indexer like IBM Swift Package Catalog (
https://packagecatalog.com/ ) over Swift.org.

And like Swift Source Compatibility Suite to be "indexed" the author of
the project needs to send the request to be accepted.

Lets assume the same rules as Compatibility Suite:
Acceptance Criteria

To be accepted into the Swift source compatibility test suite, a project
must:

   1. Target Linux, macOS, or iOS/tvOS/watchOS device
   2. Be an *Xcode* or *Swift Package Manager* project (Carthage and
   CocoaPods are currently unsupported but are being explored to be supported
   in the future)
   3. Support building on either Linux or macOS
   4. Be contained in a publicly accessible git repository
   5. Maintain a project branch that builds against Swift 3.0
   compatibility mode and passes any unit tests
   6. Have maintainers who will commit to resolve issues in a timely
   manner
   7. Be compatible with the latest GM/Beta versions of *Xcode* and
   *swiftpm*
   8. Add value not already included in the suite
   9. Be licensed with one of the following permissive licenses:
      - BSD
      - MIT
      - Apache License, version 2.0
      - Eclipse Public License
      - Mozilla Public License (MPL) 1.1
      - MPL 2.0
      - CDDL

i’m okay with this as long as point 9 is changed to accept copyleft

I have started a new thread.

···

On Nov 9, 2017, at 2:05 AM, Adrian Zubarev <adrian.zubarev@devandartist.com> wrote:

Hello Ted, would you mind opening a new thread and post an update about the forum and maybe answer a few minor questions like:
- what is currently planned?
- in which timeframe we *might* see the forum finally happening (don’t have to be a promise)?
- what happens to old mailing lists? (I’d say put a public archive somewhere, maybe slightly better formatted than the mailing list is, but do not spam new categories so that we can start with a fresh and clean forum. I also wouldn’t bother with migrating old users to new accounts. If I’m interested in contribution I can take a minute and create a new forum account. I mean we’re not trying to sell here anything, so this shouldn’t be a high priority right!?)

Thank you. :)

Am 9. November 2017 um 07:12:10, Ted Kremenek via swift-evolution (swift-evolution@swift.org) schrieb:

On Nov 8, 2017, at 12:08 PM, Kelvin Ma <kelvin13ma@gmail.com> wrote:

On Wed, Nov 8, 2017 at 1:58 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

On Nov 8, 2017, at 11:40 AM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

On Nov 8, 2017, at 4:30 AM, Wallacy via swift-evolution <swift-evolution@swift.org> wrote:

I do not agree with Ted that only a few projects should be ranked, everyone, as it is in npm should be available. Only be graded according to recommendations.

I’m a bit confused. I’m not sure what comments of mine I’m referring to.

Clearly I’m double confused. That meant to read “I’m not sure what comments of mine *you* are referring to”.

I fully support having a broad spectrum of libraries that the community builds and uses. Any library that we decide to make part of “core Swift” — IMHO at a mature point in a library’s evolution — would need to have high value to the majority of the community and would need to feel solid enough that we can lock it in for both source and binary compatibility, high quality of implementation with sustained maintenance, etc.

i mean I don’t think these approaches are incompatible. The “swift core” could just make the process of independent libraries getting started easier. Like right now there’s really no place to say “hey I just started a library project for X, and anyone who wants to be involved should contribute at Y github repo where it lives right now”. I’ve tried sending that on this list before and it didn’t really work because mailing lists aren’t really a good medium for that and no one wants the swift-evolution list getting clogged with project-specific messages most people don’t care about.

These are great points.

FWIW, I’m getting optimistic about moving to a forum soon. Would you expect that a forum could provide a better vehicle than a mailing list to arrange communication and interest within the community around building libraries? Not just doing shout outs for projects, but also doing possible API design review, etc.?

As an analogy, within Apple we have various mailing lists to review APIs, which is one mechanism used for different teams to co-review newly proposed APIs and consider how they compose together with other APIs. It’s not always perfect, but it does help facilitate a culture of API review so that various APIs can be considered together and part of the same (or compatible) design philosophies.

One of the things that resonated to me from Dave DeLong’s proposal was a sense about having a set of libraries that are well-considered and their efforts coordinated. While the coordination pitched in Dave’s proposal was about a focused effort on a particular set of libraries/features, coordination can also take the form of having a community that cares about building good APIs and can constructively discuss them. This can be done while also completely factoring out whether or not those APIs are part of “core Swift”. Further, shared API review wouldn’t necessarily be about making actual decisions — which is the case of swift-evolution when evaluating language and standard library changes — but offering advice. Fundamentally the library author still stays in control of their library and APIs, but the community could help in shaping up the gestalt of what are considered well-crafted Swift APIs in general.

Of course the big difference here with this idea compared to Apple’s internal API review process is that for Apple the APIs it vends are intended to be shipped together, and thus they must work together. In open source, however, efforts on various libraries are often (usually?) independent. Projects are usually created independently by different authors, and while it may be desirable for APIs from various libraries to feel natural to work with together, it’s not a requirement on their construction in general.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Is there any progress until now? If not, how should we organize ourselves to make "non-standard libraries" a reality? As a starting point I would suggest to make a new "Libraries" category and additional subcategories for each library (but this may not be the best way to do it).

In this thread there are already great suggestions what libraries could be useful.

Hi Qbyte,

As mentioned in other threads, when resurrecting an old thread, it would be nice to start a new one with a summary of the existing discussion.

This thread was very long and many ideas were brought forth. It's unclear what aspects of it you're referring to when you ask about "progress," and since it's impossible therefore to give a directed answer, you're inviting general responses that rehash the existing conversation--which it would be nice to avoid.

Hi Xiaodi,

Thank you for your remarks I will take them seriously.
But currently I have not enough time to summarize this thread and start a new one. I just wanted to ask for the current state.

By “progress” I meant: Currently I think we don’t have a concrete plan how to organize ourselves (or have an “official” repository where we can contribute to). But maybe I missed some thread, so is there a plan or only some ideas how to go on?

3 Likes

For what I understood the idea is that for a lib to become ‘official’ it first should be widely used by the community and have been ironed out for some time before the core team considers it.

I think that is what we are arguing about here. That approach works for some things, but has obviously not been working for close-to-foundation type things like basic math/geometry or a cross-platform color type. Instead we have 50 different libraries which are mostly the same, but none of them gains traction because projects are afraid to have a dependency which might not be supported in the future.

I think what some are asking is if, for a few foundational things, it might be better to make space for something to be officially blessed, and then use an evolution type process (maybe even on these forums) to cover the basics. Otherwise it seems to be a chicken and egg problem...

6 Likes

Yeha I get that, and I agree and something needs to change for this to get better. That's why I proposed a while ago to have a dedicated category on the forum for projects, but the answer is that the user category is for that also.

So, unless the Core Team changes his mind, I think that the best thing we can do as a community is start using the user category on the forum more to showoff projects and recommend them to new users. If people starts doing that some libs are gonna become defacto standard (as for example the Result type did).

I personally think that this should work, growing these libraries organically and not relying on the core team to be the judges until is the right moment. But that's my opinion ^^

There are two schools of thought here: "batteries included" and "free market". You just described the free market approach.

Batteries included is what e.g. Python and Go do, which is to include a curated set of libraries. This is usually called "Standard Library" but in Swift this means something else, hence the term "Non-Standard Libraries".

The way I see it, Foundation is the reason that neither of these approaches has moved forward. The logical place to promote a 3rd party library would be to add it to Foundation. But that has an incredibly high bar since it will end up in macOS and iOS and would have to be approved by Apple. Similarly, a batteries included approach would basically be a competitor to Foundation, which Apple seems to have no intention of supporting with resources.

So unfortunately I wouldn't expect any progress here anytime soon.

1 Like