Large Proposal: Non-Standard Libraries

Hmm. I kind of like the idea, but not really. I think it has a fundamental
flaw: centralization.

You see, the StdLib can be commanded by a central authority (the core team)
and hear the needs of the general community (through swift-evolution and
such) because, among other things, it’s small. The StdLib solves common and
well-understood problems. Most of the solutions it provides are optimal for
all use cases.

This is fundamentally different from a non-StdLib. If I understood your
idea correctly, it would be the complement of StdLib; it will be big and it
will attend problems that are not well-understood or whose solutions have
many differrying approaches depending on the users’ neccessities (think
Geometry). Therefore, a correct and complete approach would inevitably have
to:
- Know the needs of all the relevant user groups and balance their
priorities.
- Contain all the important and complementary solutions.

This is very hard to achieve in a centralized system. Not impossible, but
very resource-intensive.

You can achieve something similar by letting the community grow and by
encouraging a good environment. People will the build the tools they need,
and the important voices will index the tools people use the most. That
makes them good, as well as easily findable. It’s not perfect either, but
it’s more efficient in my opinion.

— Félix Fischer

People tend to build the tools they need, but not what other people need.
And there are many many examples from other languages of what can go wrong
when non-standard libraries compete. As for important voices indexing the
tools people use the most, I don’t see that happening.

···

On Tue, Nov 7, 2017 at 12:18 PM, Félix Fischer via swift-evolution < swift-evolution@swift.org> wrote:

Hmm. I kind of like the idea, but not really. I think it has a fundamental
flaw: centralization.

You see, the StdLib can be commanded by a central authority (the core
team) and hear the needs of the general community (through swift-evolution
and such) because, among other things, it’s small. The StdLib solves common
and well-understood problems. Most of the solutions it provides are optimal
for all use cases.

This is fundamentally different from a non-StdLib. If I understood your
idea correctly, it would be the complement of StdLib; it will be big and it
will attend problems that are not well-understood or whose solutions have
many differrying approaches depending on the users’ neccessities (think
Geometry). Therefore, a correct and complete approach would inevitably have
to:
- Know the needs of all the relevant user groups and balance their
priorities.
- Contain all the important and complementary solutions.

This is very hard to achieve in a centralized system. Not impossible, but
very resource-intensive.

You can achieve something similar by letting the community grow and by
encouraging a good environment. People will the build the tools they need,
and the important voices will index the tools people use the most. That
makes them good, as well as easily findable. It’s not perfect either, but
it’s more efficient in my opinion.

— Félix Fischer

Hmm. I kind of like the idea, but not really. I think it has a
fundamental flaw: centralization.

You see, the StdLib can be commanded by a central authority (the core
team) and hear the needs of the general community (through swift-evolution
and such) because, among other things, it’s small. The StdLib solves common
and well-understood problems. Most of the solutions it provides are optimal
for all use cases.

This is fundamentally different from a non-StdLib. If I understood your
idea correctly, it would be the complement of StdLib; it will be big and it
will attend problems that are not well-understood or whose solutions have
many differrying approaches depending on the users’ neccessities (think
Geometry). Therefore, a correct and complete approach would inevitably have
to:
- Know the needs of all the relevant user groups and balance their
priorities.
- Contain all the important and complementary solutions.

This is very hard to achieve in a centralized system. Not impossible,
but very resource-intensive.

You can achieve something similar by letting the community grow and by
encouraging a good environment. People will the build the tools they need,
and the important voices will index the tools people use the most. That
makes them good, as well as easily findable. It’s not perfect either, but
it’s more efficient in my opinion.

— Félix Fischer

People tend to build the tools they need, but not what other people need.

Fair point. Dog-feeding might be an issue, but not necessarily.

And there are many many examples from other languages of what can go wrong

when non-standard libraries compete.

Hmm. Okay, yes. But what about healthy competition? Isn’t that good as
well? In which context you get bad results? Can you change the system s.t.
you encourage good competition?

i think in open source, competition is mostly beneficial when you have one
old monopoly that’s become lethargic and a new more energetic project
upends it, i.e. gcc vs clang. You get a new improved tool, and the old
project also gets a wakeup call, which is why you’ve seen such a big
improvement in recent versions of gcc. Where there is no such incumbent,
and everyone is starting from scratch, it becomes counterproductive.

As for important voices indexing the tools people use the most, I don’t

see that happening.

This is easier than it seems. The Compatibility Suite is already an index
of important libraries.

Regarding the other problems I see with this proposal: how will you attend
the necessities of so many diverse groups? You’d need a whole...
consortium, of sorts. You can’t have just a leader: no one knows the
context of all the users.

The Compatibility Suite suffers from licensing issues which prevents a lot
of important libraries from being listed in it. GPL libraries are
effectively excluded from it.

···

On Tue, Nov 7, 2017 at 1:11 PM, Félix Fischer <felix91gr@gmail.com> wrote:

On Tue, Nov 7, 2017 at 4:01 PM Kelvin Ma <kelvin13ma@gmail.com> wrote:

On Tue, Nov 7, 2017 at 12:18 PM, Félix Fischer via swift-evolution < >> swift-evolution@swift.org> wrote:

The Compatibility Suite is a good start, but I agree that something a bit
more centralized has its benefits.

To be perfect, Compatibility Suite and Swift Package Manager need to work
"together" to offer something simple like nodejs npm and a friendly (and
central) interface to not only find these projects. Something more similar
to nuget too.

The only thing I miss using npm and nuget is some kind of "compromise" with
maintenance. And also some commitment to (avoid) rework. Several projects
remake something that another does also without explaining well the
differences between them.

Maybe we don't need to code any "Non-Standard Libraries"! Only a opt-in
project like Compatibility Suite with steroids. Not only to track those
projects, but in some level help defining some standards, documentations,
versioning etc. This can be done entirely within the community.

Not so similar, however the gstreamer keeps a list of "base", "good",
"ugly" and "bad" plugins for similar reasons.

We can do something in this line of reasoning:
- A central repository for projects (like Compatibility Suite)
- A tool to find and add each project (like SwiftPM)
- Rules for joining (like Compatibility Suite)
- A classification for each repository (like gstreamer)
- A good way to make each project as small and direct as possible (to take
advantage of cross-module inlining / specialization)
- A list of discussion (or a forum?) for people that maintain (or have an
interest in maintain) projects in this "official" list.

I vote for empowering SwiftPM and Compatibility Suite instead a
"Non-Standard Libraries".

···

Em ter, 7 de nov de 2017 às 17:19, Kelvin Ma via swift-evolution < swift-evolution@swift.org> escreveu:

On Tue, Nov 7, 2017 at 1:11 PM, Félix Fischer <felix91gr@gmail.com> wrote:

On Tue, Nov 7, 2017 at 4:01 PM Kelvin Ma <kelvin13ma@gmail.com> wrote:

On Tue, Nov 7, 2017 at 12:18 PM, Félix Fischer via swift-evolution < >>> swift-evolution@swift.org> wrote:

Hmm. I kind of like the idea, but not really. I think it has a
fundamental flaw: centralization.

You see, the StdLib can be commanded by a central authority (the core
team) and hear the needs of the general community (through swift-evolution
and such) because, among other things, it’s small. The StdLib solves common
and well-understood problems. Most of the solutions it provides are optimal
for all use cases.

This is fundamentally different from a non-StdLib. If I understood your
idea correctly, it would be the complement of StdLib; it will be big and it
will attend problems that are not well-understood or whose solutions have
many differrying approaches depending on the users’ neccessities (think
Geometry). Therefore, a correct and complete approach would inevitably have
to:
- Know the needs of all the relevant user groups and balance their
priorities.
- Contain all the important and complementary solutions.

This is very hard to achieve in a centralized system. Not impossible,
but very resource-intensive.

You can achieve something similar by letting the community grow and by
encouraging a good environment. People will the build the tools they need,
and the important voices will index the tools people use the most. That
makes them good, as well as easily findable. It’s not perfect either, but
it’s more efficient in my opinion.

— Félix Fischer

People tend to build the tools they need, but not what other people
need.

Fair point. Dog-feeding might be an issue, but not necessarily.

And there are many many examples from other languages of what can go

wrong when non-standard libraries compete.

Hmm. Okay, yes. But what about healthy competition? Isn’t that good as
well? In which context you get bad results? Can you change the system s.t.
you encourage good competition?

i think in open source, competition is mostly beneficial when you have one
old monopoly that’s become lethargic and a new more energetic project
upends it, i.e. gcc vs clang. You get a new improved tool, and the old
project also gets a wakeup call, which is why you’ve seen such a big
improvement in recent versions of gcc. Where there is no such incumbent,
and everyone is starting from scratch, it becomes counterproductive.

As for important voices indexing the tools people use the most, I don’t

see that happening.

This is easier than it seems. The Compatibility Suite is already an index
of important libraries.

Regarding the other problems I see with this proposal: how will you
attend the necessities of so many diverse groups? You’d need a whole...
consortium, of sorts. You can’t have just a leader: no one knows the
context of all the users.

The Compatibility Suite suffers from licensing issues which prevents a lot
of important libraries from being listed in it. GPL libraries are
effectively excluded from it.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

The compatibility suite is not a good model because it was designed for a
completely different purpose. Its license restrictions are so that it can
be shipped in an Apple repository and it’s not really optimized for
discovery or documentation at all. Any non-standard library index must
include GPL licensed libraries, as well as have some common documentation
format. It’d be nice if the index could actually generate documentation
from source comments.

···

On Tue, Nov 7, 2017 at 2:24 PM, Wallacy <wallacyf@gmail.com> wrote:

The Compatibility Suite is a good start, but I agree that something a bit
more centralized has its benefits.

To be perfect, Compatibility Suite and Swift Package Manager need to work
"together" to offer something simple like nodejs npm and a friendly (and
central) interface to not only find these projects. Something more similar
to nuget too.

The only thing I miss using npm and nuget is some kind of "compromise"
with maintenance. And also some commitment to (avoid) rework. Several
projects remake something that another does also without explaining well
the differences between them.

Maybe we don't need to code any "Non-Standard Libraries"! Only a opt-in
project like Compatibility Suite with steroids. Not only to track those
projects, but in some level help defining some standards, documentations,
versioning etc. This can be done entirely within the community.

Not so similar, however the gstreamer keeps a list of "base", "good",
"ugly" and "bad" plugins for similar reasons.

We can do something in this line of reasoning:
- A central repository for projects (like Compatibility Suite)
- A tool to find and add each project (like SwiftPM)
- Rules for joining (like Compatibility Suite)
- A classification for each repository (like gstreamer)
- A good way to make each project as small and direct as possible (to take
advantage of cross-module inlining / specialization)
- A list of discussion (or a forum?) for people that maintain (or have an
interest in maintain) projects in this "official" list.

I vote for empowering SwiftPM and Compatibility Suite instead a
"Non-Standard Libraries".

Em ter, 7 de nov de 2017 às 17:19, Kelvin Ma via swift-evolution < > swift-evolution@swift.org> escreveu:

On Tue, Nov 7, 2017 at 1:11 PM, Félix Fischer <felix91gr@gmail.com> >> wrote:

On Tue, Nov 7, 2017 at 4:01 PM Kelvin Ma <kelvin13ma@gmail.com> wrote:

On Tue, Nov 7, 2017 at 12:18 PM, Félix Fischer via swift-evolution < >>>> swift-evolution@swift.org> wrote:

Hmm. I kind of like the idea, but not really. I think it has a
fundamental flaw: centralization.

You see, the StdLib can be commanded by a central authority (the core
team) and hear the needs of the general community (through swift-evolution
and such) because, among other things, it’s small. The StdLib solves common
and well-understood problems. Most of the solutions it provides are optimal
for all use cases.

This is fundamentally different from a non-StdLib. If I understood
your idea correctly, it would be the complement of StdLib; it will be big
and it will attend problems that are not well-understood or whose solutions
have many differrying approaches depending on the users’ neccessities
(think Geometry). Therefore, a correct and complete approach would
inevitably have to:
- Know the needs of all the relevant user groups and balance their
priorities.
- Contain all the important and complementary solutions.

This is very hard to achieve in a centralized system. Not impossible,
but very resource-intensive.

You can achieve something similar by letting the community grow and by
encouraging a good environment. People will the build the tools they need,
and the important voices will index the tools people use the most. That
makes them good, as well as easily findable. It’s not perfect either, but
it’s more efficient in my opinion.

— Félix Fischer

People tend to build the tools they need, but not what other people
need.

Fair point. Dog-feeding might be an issue, but not necessarily.

And there are many many examples from other languages of what can go

wrong when non-standard libraries compete.

Hmm. Okay, yes. But what about healthy competition? Isn’t that good as
well? In which context you get bad results? Can you change the system s.t.
you encourage good competition?

i think in open source, competition is mostly beneficial when you have
one old monopoly that’s become lethargic and a new more energetic project
upends it, i.e. gcc vs clang. You get a new improved tool, and the old
project also gets a wakeup call, which is why you’ve seen such a big
improvement in recent versions of gcc. Where there is no such incumbent,
and everyone is starting from scratch, it becomes counterproductive.

As for important voices indexing the tools people use the most, I don’t

see that happening.

This is easier than it seems. The Compatibility Suite is already an
index of important libraries.

Regarding the other problems I see with this proposal: how will you
attend the necessities of so many diverse groups? You’d need a whole...
consortium, of sorts. You can’t have just a leader: no one knows the
context of all the users.

The Compatibility Suite suffers from licensing issues which prevents a
lot of important libraries from being listed in it. GPL libraries are
effectively excluded from it.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

The Compatibility Suite is a good start, but I agree that something a bit
more centralized has its benefits.

To be perfect, Compatibility Suite and Swift Package Manager need to work
"together" to offer something simple like nodejs npm and a friendly (and
central) interface to not only find these projects. Something more similar
to nuget too.

The only thing I miss using npm and nuget is some kind of "compromise"
with maintenance. And also some commitment to (avoid) rework. Several
projects remake something that another does also without explaining well
the differences between them.

Maybe we don't need to code any "Non-Standard Libraries"! Only a opt-in
project like Compatibility Suite with steroids. Not only to track those
projects, but in some level help defining some standards, documentations,
versioning etc. This can be done entirely within the community.

Not so similar, however the gstreamer keeps a list of "base", "good",
"ugly" and "bad" plugins for similar reasons.

We can do something in this line of reasoning:
- A central repository for projects (like Compatibility Suite)
- A tool to find and add each project (like SwiftPM)
- Rules for joining (like Compatibility Suite)
- A classification for each repository (like gstreamer)
- A good way to make each project as small and direct as possible (to take
advantage of cross-module inlining / specialization)
- A list of discussion (or a forum?) for people that maintain (or have an
interest in maintain) projects in this "official" list.

I like this approach much more. Feels more natural. And a forum
(piggybacking on the eventual Discourse perhaps). I’d only change two
things and extend one:

- Instead of “central repository”, a “central index”. It makes it more
transparent, more distributed, and closer to the current reality.

- Here:

I vote for empowering SwiftPM and Compatibility Suite instead a
"Non-Standard Libraries".

I agree with a central index, but as Kelvin says, we shouldn’t be using the
Compat Suite directly because of GPL issues.

- I’d extend on the “Rules for Joining” point: they should be as clear and
explicit as possible, to avoid drama like the one episode that happened
last year on the JS repositories with that string-padding library. That
thing broke half of the internet for some hours, and it was all about
something unclear in the rules, if I remember the case correctly.

That’s all I’d add to Wallace’s list.

On another line, the licenses point brought up by Kelvin made me thought of
this:
There could be an integration in SPM or similar, that checked your license.
That way:
- It can tell you which projects you can import (that fit with your
license). That way, if you’re using the BSD license, this integration
doesn’t offer to import GPL projects because that’d make your license
inconsistent.
- It can tell you if you need to change your license or remove a package,
in order to make your license consistent again. That way, maybe you change
to GPL because you realised that the tool you wanted to use is worth it.

This would be important from the perspective of ergonomics and
(license-consistent) discovery. It’s a detail, yes. But would be very nice.

···

On Tue, Nov 7, 2017 at 5:24 PM Wallacy <wallacyf@gmail.com> wrote:

Em ter, 7 de nov de 2017 às 17:19, Kelvin Ma via swift-evolution < > swift-evolution@swift.org> escreveu:

On Tue, Nov 7, 2017 at 1:11 PM, Félix Fischer <felix91gr@gmail.com> >> wrote:

On Tue, Nov 7, 2017 at 4:01 PM Kelvin Ma <kelvin13ma@gmail.com> wrote:

On Tue, Nov 7, 2017 at 12:18 PM, Félix Fischer via swift-evolution < >>>> swift-evolution@swift.org> wrote:

Hmm. I kind of like the idea, but not really. I think it has a
fundamental flaw: centralization.

You see, the StdLib can be commanded by a central authority (the core
team) and hear the needs of the general community (through swift-evolution
and such) because, among other things, it’s small. The StdLib solves common
and well-understood problems. Most of the solutions it provides are optimal
for all use cases.

This is fundamentally different from a non-StdLib. If I understood
your idea correctly, it would be the complement of StdLib; it will be big
and it will attend problems that are not well-understood or whose solutions
have many differrying approaches depending on the users’ neccessities
(think Geometry). Therefore, a correct and complete approach would
inevitably have to:
- Know the needs of all the relevant user groups and balance their
priorities.
- Contain all the important and complementary solutions.

This is very hard to achieve in a centralized system. Not impossible,
but very resource-intensive.

You can achieve something similar by letting the community grow and by
encouraging a good environment. People will the build the tools they need,
and the important voices will index the tools people use the most. That
makes them good, as well as easily findable. It’s not perfect either, but
it’s more efficient in my opinion.

— Félix Fischer

People tend to build the tools they need, but not what other people
need.

Fair point. Dog-feeding might be an issue, but not necessarily.

And there are many many examples from other languages of what can go

wrong when non-standard libraries compete.

Hmm. Okay, yes. But what about healthy competition? Isn’t that good as
well? In which context you get bad results? Can you change the system s.t.
you encourage good competition?

i think in open source, competition is mostly beneficial when you have
one old monopoly that’s become lethargic and a new more energetic project
upends it, i.e. gcc vs clang. You get a new improved tool, and the old
project also gets a wakeup call, which is why you’ve seen such a big
improvement in recent versions of gcc. Where there is no such incumbent,
and everyone is starting from scratch, it becomes counterproductive.

As for important voices indexing the tools people use the most, I don’t

see that happening.

This is easier than it seems. The Compatibility Suite is already an
index of important libraries.

Regarding the other problems I see with this proposal: how will you
attend the necessities of so many diverse groups? You’d need a whole...
consortium, of sorts. You can’t have just a leader: no one knows the
context of all the users.

The Compatibility Suite suffers from licensing issues which prevents a
lot of important libraries from being listed in it. GPL libraries are
effectively excluded from it.

_______________________________________________

swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I think Swift is less vulnerable to that than node.js if anything because
Swift is compiled ahead of time so someone removing their repo doesn’t
instantly break everything else.

···

On Tue, Nov 7, 2017 at 3:15 PM, Félix Fischer <felix91gr@gmail.com> wrote:

On Tue, Nov 7, 2017 at 5:24 PM Wallacy <wallacyf@gmail.com> wrote:

The Compatibility Suite is a good start, but I agree that something a
bit more centralized has its benefits.

To be perfect, Compatibility Suite and Swift Package Manager need to work
"together" to offer something simple like nodejs npm and a friendly (and
central) interface to not only find these projects. Something more similar
to nuget too.

The only thing I miss using npm and nuget is some kind of "compromise"
with maintenance. And also some commitment to (avoid) rework. Several
projects remake something that another does also without explaining well
the differences between them.

Maybe we don't need to code any "Non-Standard Libraries"! Only a opt-in
project like Compatibility Suite with steroids. Not only to track those
projects, but in some level help defining some standards, documentations,
versioning etc. This can be done entirely within the community.

Not so similar, however the gstreamer keeps a list of "base", "good",
"ugly" and "bad" plugins for similar reasons.

We can do something in this line of reasoning:
- A central repository for projects (like Compatibility Suite)
- A tool to find and add each project (like SwiftPM)
- Rules for joining (like Compatibility Suite)
- A classification for each repository (like gstreamer)
- A good way to make each project as small and direct as possible (to
take advantage of cross-module inlining / specialization)
- A list of discussion (or a forum?) for people that maintain (or have an
interest in maintain) projects in this "official" list.

I like this approach much more. Feels more natural. And a forum
(piggybacking on the eventual Discourse perhaps). I’d only change two
things and extend one:

- Instead of “central repository”, a “central index”. It makes it more
transparent, more distributed, and closer to the current reality.

- Here:

I vote for empowering SwiftPM and Compatibility Suite instead a
"Non-Standard Libraries".

I agree with a central index, but as Kelvin says, we shouldn’t be using
the Compat Suite directly because of GPL issues.

- I’d extend on the “Rules for Joining” point: they should be as clear and
explicit as possible, to avoid drama like the one episode that happened
last year on the JS repositories with that string-padding library. That
thing broke half of the internet for some hours, and it was all about
something unclear in the rules, if I remember the case correctly.

I didn’t reply to all before... um,

That’s a very good point.

Still, I’d like the rules to be as clear as possible. That only helps :)

Another point I forgot to mention, Kelvin: Jazzy (
https://github.com/realm/jazzy/blob/master/README.md\) derives documentation
from the comments. We can use that in the index :)

···

On Tue, Nov 7, 2017 at 6:18 PM Kelvin Ma <kelvin13ma@gmail.com> wrote:

On Tue, Nov 7, 2017 at 3:15 PM, Félix Fischer <felix91gr@gmail.com> wrote:

On Tue, Nov 7, 2017 at 5:24 PM Wallacy <wallacyf@gmail.com> wrote:

The Compatibility Suite is a good start, but I agree that something a
bit more centralized has its benefits.

To be perfect, Compatibility Suite and Swift Package Manager need to
work "together" to offer something simple like nodejs npm and a friendly
(and central) interface to not only find these projects. Something more
similar to nuget too.

The only thing I miss using npm and nuget is some kind of "compromise"
with maintenance. And also some commitment to (avoid) rework. Several
projects remake something that another does also without explaining well
the differences between them.

Maybe we don't need to code any "Non-Standard Libraries"! Only a opt-in
project like Compatibility Suite with steroids. Not only to track those
projects, but in some level help defining some standards, documentations,
versioning etc. This can be done entirely within the community.

Not so similar, however the gstreamer keeps a list of "base", "good",
"ugly" and "bad" plugins for similar reasons.

We can do something in this line of reasoning:
- A central repository for projects (like Compatibility Suite)
- A tool to find and add each project (like SwiftPM)
- Rules for joining (like Compatibility Suite)
- A classification for each repository (like gstreamer)
- A good way to make each project as small and direct as possible (to
take advantage of cross-module inlining / specialization)
- A list of discussion (or a forum?) for people that maintain (or have
an interest in maintain) projects in this "official" list.

I like this approach much more. Feels more natural. And a forum
(piggybacking on the eventual Discourse perhaps). I’d only change two
things and extend one:

- Instead of “central repository”, a “central index”. It makes it more
transparent, more distributed, and closer to the current reality.

- Here:

I vote for empowering SwiftPM and Compatibility Suite instead a
"Non-Standard Libraries".

I agree with a central index, but as Kelvin says, we shouldn’t be using
the Compat Suite directly because of GPL issues.

- I’d extend on the “Rules for Joining” point: they should be as clear
and explicit as possible, to avoid drama like the one episode that happened
last year on the JS repositories with that string-padding library. That
thing broke half of the internet for some hours, and it was all about
something unclear in the rules, if I remember the case correctly.

I think Swift is less vulnerable to that than node.js if anything because
Swift is compiled ahead of time so someone removing their repo doesn’t
instantly break everything else.