Swift phases and mis-timed proposals

I’m not sure documentation is sufficient… If I only wanted to use the “stable” features, would I have to read the docs on every standard library type I use? And what about unstable or pre-stable compiler features? It’s not like I can right-click on a var and get the type inference rules, for example.

Now, I wouldn’t mind the ability to vote (or at the core team’s discretion) that we accept a proposal into a separate “pre-stable” or “staging” toolchain, with the understanding that if it doesn’t cause any major issues it’ll automatically be accepted for the next major release. If issues are found, then we’d have a chance to re-review & refine the proposal before committing it to the main toolchain and its source-compatibility requirements. If, upon re-review, the proposal is rejected, it would be deprecated in the next major release’s staging toolchain and removed in the major release after that.

To clarify, the timeline would be something like:

Prior to Swift N: We accept SE-xxxx for Swift N, as well as SE-yyyy and SE-zzzz for Swift N Staging.
Release Swift N:
  - Standard: implements SE-xxxx
  - Staging: implements SE-yyyy and SE-zzzz
Prior to Swift N+1: We accept a proposal to revert SE-zzzz because it turns out to be a horrible idea
Release Swift N+1:
  - Standard: implements SE-xxxx, SE-yyyy
  - Staging: deprecates SE-zzzz
Release Swift N+2:
  - Standard: implements SE-xxxx, SE-yyyy
  - Staging: reverts SE-zzzz

This would give the people whose code relies on SE-zzzz probably between 1-2 years to find a work-around, given the current pace of major releases, and depending on exactly where in the timeline the problems get discovered. (As a last resort, you could leave the Swift N or N+1 staging toolchains installed, refactor any features relying on SE-zzzz into their own module, and move the rest of your app’s code on to Swift N+2. Certainly not ideal, but AFAIK, it's an option.) And if you’re worried about it, just use the stable toolchain.

I do have some concerns, though… My understanding is that Rust does something similar and has an “experimental” compiler release… Apparently, all the “cool” stuff is done there, which makes it the “standard" compiler, and relegates the actually-stable compiler to “old version” status. Or so I hear, anyway. I haven’t really used it much. Anyway, I wouldn’t want us to get in a position where everyone’s using some feature of the staging toolchain, but we can’t merge it into the stable toolchain because maybe it breaks some niche use-case that we need to support or something. We do have one thing going for us, though, that Rust doesn’t… a large portion of Swift’s user base needs to ship to Apple's App stores, and Apple controls what toolchains you’re allowed to use for that. OTOH, if we say "no staging toolchain for the app store”, then we have to ask if enough people will use it for the issues to be found.

- Dave Sweeris

···

On Jun 12, 2017, at 7:16 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 12, 2017, at 5:12 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jun 12, 2017, at 12:47 PM, Paul Cantrell <cantrell@pobox.com> wrote:

Concern #2 is that it’s hard to know what to do with a proposal when the ideal answer is “we need to see how it plays out in practice and then decide whether to accept it.” Theoretical discussion untempered by practical prototyping is a great way for a group to talk itself into a bad idea. (This will especially be a problem with future work to build out generics & existentials.)

I agree. Do you have specific thoughts here on what could be done differently?

I think we can partially solve this by having an extra stage of “stability” in our documentation. Right now, things are either officially part of the language or they aren’t (with a very short beta period between WWDC and the Fall). What I would like to see in the documentation is a notion of how sure we are of a particular piece of API or syntax.

Unstable - This is actively under development/design and should be expected to change. It is not safe to build frameworks on top of this.

Pre-Stable - We think we have a design which works, but we need to see how it works in the larger ecosystem before guaranteeing stability. Anything built on this should be prepared for potentially large changes in the future, and be marked “Pre-Stable” themselves.

Stable - This has been extensively tested in the actual swift ecosystem and proven itself a useful addition. Any changes to this in the future are guaranteed to either be source-compatible or have a full depreciation cycle. It is completely safe to build on.

I think being explicit about what we expect to change in the future, and having a notion of “stable” that is truly stable will give us a lot more freedom to experiment as well as room to fix our mistakes.

What if instead of a FIFO queue, we had a reddit style queue where proposals can bubble up based on community interest? It might help us figure out what to focus on in the 'phase 2' times where we allow a few out of scope proposals through... we just grab a couple off of the top.

Thanks,
Jon

···

Sent from my iPhone

On Jun 12, 2017, at 11:18 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 12, 2017, at 10:07 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org> wrote:

Perhaps the solution is not necessarily throttling proposals per se, but having some mechanism for routing a proposal to something other than either a review cycle or the freezer: “this needs manifesto-ing,” “this needs prototyping to measure impact on existing code,” “this needs to simmer and find its context before we work it into a proposal,” etc. (That’s related to Daryle’s original message.)

I feel like I’m missing some part of the motivation for this thread. Let me try to explain why:

Lets say you’re interested in proposing a new feature or idea today, and are told that it is out of scope. One of two things happen. When the next release comes around, either:

1) if you’re still engaged with swift-evolution, you can bring it back up.

2) if you’re not still engaged, it will get dropped unless someone else is interested in championing it.

What good does a “queue” of proposals in the freezer do? In practice, getting a proposal to happen frequently requires editing and iteration, not to mention active discussion about motivation. We have no shortage of important proposals to get through in each release, why should we make it easier for proposals with no active proponent? Wouldn’t that just encourage people to drop off “drive-by” proposals for ideas?

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

+1, particularly for this being a section in Discourse ;-)

···

On Mon, Jun 12, 2017 at 9:31 PM, Paul Cantrell via swift-evolution < swift-evolution@swift.org> wrote:

I support everything Jon wrote.

+1 Free-for-all brainstorming venue separate from focused proposal
discussion.

You posted this with a wink, but I think it's actually quite important. I see some value in having a "sounding board" area where people can talk through ideas that maybe aren't anywhere close to proposal yet, but actually encouraging the use of a completely different venue would be pretty unfortunate:

1. That venue would become important to the community, but it wouldn't really be a *part* of the community in some ways. For example, it would be a terrible failure if the fashionable place for (say) talking about new reflection APIs was a website where not everyone felt comfortable participating because some of the established contributors there were jerks. We really want everything in the project to be covered under the Code of Conduct.

2. Today, there's one standard venue for in-depth conversations about Swift evolution: this mailing list. Having one venue is self-reinforcing: if you have a conversation somewhere else, you're always missing people, and convincing people to join your side-conversation naturally makes them ask why you're not having it on the main venue. In contrast, telling people to have conversations elsewhere means intentionally fracturing the community, and there's no reason it would stay with just two venues. We'd probably end up with a ton of short-lived, hard-to-find, poorly-archived discussion groups with their own isolated focus and culture; that's a situation that naturally leads to different groups not realizing that they need to be talking to each other. It'd be more like the salon system than an engineering project.

3. Having a separate venue for early-stage discussions also creates an awkward transition when the proposal is ready to move on. Discussion probably doesn't totally stop in the old venue, and in the new venue the discussion has lost a lot of context, and it's annoying to constantly refer back to the old venue. In contrast, if early discussions are just threads on a different part of the same forum, they can simply be moved around as they mature.

So, in my opinion, we should just bear with the current situation until we have a proper forum.

John.

···

On Jun 13, 2017, at 1:08 AM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:
On Mon, Jun 12, 2017 at 9:31 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
I support everything Jon wrote.

+1 Free-for-all brainstorming venue separate from focused proposal discussion.

+1, particularly for this being a section in Discourse ;-)

Most proposals break down to one of:

* Adjusting the language to add features that match development ideals (usually under a manifesto umbrella)
* Fixing problems in the language that lead to inefficient compiler implementation / unnecessary complexity
* Adding features to the language to match common compilation options found elsewhere (the recent re-emergence of warning/error being a good example)
* Fixing problems in the language that lead to unsafe code

* Getting SwiftPM into shape ("ballardization")

* Editing the standard library to remove or add features for a more robust and cleaner API experience
* Adding features to the language to match common language features found in other modern programming languages

My personal interest lies in the fourth bullet ("Fixing problems in the language that lead to unsafe code"). Friends and I have found through experience a fair number of areas where the language can be adjusted to increase safety, and we've brainstormed ways to do so additively without breaking backward compatibility. There never seems to be a good time or place to introduce these ideas.

Of the remaining bullet points, the last 2 (new library features) seem like they *could* be developed, tested, and implemented independently of the standard SE process until the language is at a point (Swift 5 or Swift 6) where it becomes open to additive features.

I imagine that recent discussions like mapped keypaths, ordered sets, `count(where:)`, etc. could have a home for discussion and exploration without getting blocked by "out of scope" if there were a separate "Substandard Library" repository, mailing list, and process (potentially staffed in part or full by non-Apple personnel) that did not rely on internal Apple timelines and delivery benchmarks. Think "farm league for SE" (https://en.wikipedia.org/wiki/Farm_team\). It would lower the burden on SE but provide a way forward to discuss and develop ideas within the framework of coherent language design.

-- E

···

On Jun 13, 2017, at 2:07 AM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Tue, Jun 13, 2017 at 1:18 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jun 12, 2017, at 10:07 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Perhaps the solution is not necessarily throttling proposals per se, but having some mechanism for routing a proposal to something other than either a review cycle or the freezer: “this needs manifesto-ing,” “this needs prototyping to measure impact on existing code,” “this needs to simmer and find its context before we work it into a proposal,” etc. (That’s related to Daryle’s original message.)

I feel like I’m missing some part of the motivation for this thread. Let me try to explain why:

Lets say you’re interested in proposing a new feature or idea today, and are told that it is out of scope. One of two things happen. When the next release comes around, either:

1) if you’re still engaged with swift-evolution, you can bring it back up.

2) if you’re not still engaged, it will get dropped unless someone else is interested in championing it.

What good does a “queue” of proposals in the freezer do? In practice, getting a proposal to happen frequently requires editing and iteration, not to mention active discussion about motivation. We have no shortage of important proposals to get through in each release, why should we make it easier for proposals with no active proponent? Wouldn’t that just encourage people to drop off “drive-by” proposals for ideas?

I think this goes to the discussion above about what kind of list this is to be. We've got, on the one hand, a significant number of proposals of the following sort: how should indexing into the middle of a grapheme be handled? which integer protocols should have the bitwise operators? These require editing and iteration, as you say, and very active back-and-forth discussion. These days it seems these sorts of threads rarely gain the greatest attention. The most useful kind of feedback in such a discussion would be a detailed and robust critique; much less useful is the ever more common "+1," and storing the proposal in a freezer is the surest way to irrelevance as the detailed design can quickly become nonsensical due to even small changes elsewhere.

There are, however, a significant number of conversations that, at base, are declarations that there exists some problem or issue. It's not straightforwardly a bug that can be filed, and it's not a user knowledge issue that can be solved by a question to swift-users, but it's something else that may require a language change. This is, of course, a perfectly understandable state of affairs for which some forum should probably exist. However, the "format" of Swift Evolution requires a proposed solution and detailed design, so the author feels compelled to put down _something_ that might alleviate the problem. Frequently, later in the discussion, the author will reply that he or she isn't actually married to the proposed solution and/or does not really know if the detailed design will work at all, but wanted the community to acknowledge that a problem or issue exists and needs attention. Critiquing the placeholder proposed solution and detailed design is kind of beside the point, even though it's the ostensible purpose of this list. If anything is helpful in this scenario, it would be feedback that validates and expounds on the issue (or in the absence of that, "+1"). However, there's not an iterative process that will take you from that point to an implementable design. It might be, however, a topic that fits nicely into a "queue" of some sort.

There's another class of conversations that, from what I can tell, boil down to a request for a feature--often one that is highly desired by many people. The feature doesn't yet exist because it would require extensive design and implementation effort. Perhaps the author is unaware that the feature is already widely desired, or perhaps the author simply wants to make it known that they wish that it would take higher priority. The actual motivation for starting a thread isn't the promotion of any particular design for the desired feature, just the feature itself. Also a perfectly understandable situation, and should be accommodated in some form, somewhere. However, Swift Evolution requires a detailed design, so the author feels compelled to sketch out some sort of placeholder; sometimes, the "detailed design" recites more and more elaborate variations of the feature but makes no attempt to actually describe how it will come to be. Again, little point in critiquing it. Again, there's not an iterative process from there to implementable design. This time the topic is almost a meta-discussion on the proper prioritization of items in the "queue" of ideas, and the responses that are relevant to such a conversation (short of designing and implementing the feature) can really only be "yes, I want this feature too."

FWIW, I’ve had a few off-list discussions of that nature and found them to be incredibly valuable.

- Dave Sweeris

···

On Jun 13, 2017, at 12:18 AM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 13, 2017, at 1:08 AM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Mon, Jun 12, 2017 at 9:31 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
I support everything Jon wrote.

+1 Free-for-all brainstorming venue separate from focused proposal discussion.

+1, particularly for this being a section in Discourse ;-)

You posted this with a wink, but I think it's actually quite important. I see some value in having a "sounding board" area where people can talk through ideas that maybe aren't anywhere close to proposal yet

I'm certainly not trying to discourage anyone from having off-list discussions. Have a conversation whenever and wherever it strikes you. But I do think it would be really unfortunate to try to set somewhere else up as a permanent alternative venue.

John.

···

On Jun 13, 2017, at 3:22 AM, David Sweeris <davesweeris@mac.com> wrote:

On Jun 13, 2017, at 12:18 AM, John McCall via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jun 13, 2017, at 1:08 AM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Mon, Jun 12, 2017 at 9:31 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
I support everything Jon wrote.

+1 Free-for-all brainstorming venue separate from focused proposal discussion.

+1, particularly for this being a section in Discourse ;-)

You posted this with a wink, but I think it's actually quite important. I see some value in having a "sounding board" area where people can talk through ideas that maybe aren't anywhere close to proposal yet

FWIW, I’ve had a few off-list discussions of that nature and found them to be incredibly valuable.

+1

Not even just for development… a standard repository for “substandard” library(s) would be a great for commonly used types that aren’t quite commonly used enough to include in stdlib, and IMHO, could really help Swift’s ecosystem (I’m not claiming that it’s poor now or anything, but there’s nothing wrong with making it the best we can).

- Dave Sweeris

···

On Jun 13, 2017, at 11:46 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

I imagine that recent discussions like mapped keypaths, ordered sets, `count(where:)`, etc. could have a home for discussion and exploration without getting blocked by "out of scope" if there were a separate "Substandard Library" repository, mailing list, and process (potentially staffed in part or full by non-Apple personnel) that did not rely on internal Apple timelines and delivery benchmarks. Think "farm league for SE" (https://en.wikipedia.org/wiki/Farm_team\). It would lower the burden on SE but provide a way forward to discuss and develop ideas within the framework of coherent language design.

Oh! I was unclear… The discussions themselves were what’s valuable. The only reason they were off-list was to avoid spamming the list with premature ideas. Given the choice, I’d prefer to have them in an official “off-topic”/“sounding board”/whatever area.

- Dave Sweeris

···

On Jun 13, 2017, at 12:34 AM, John McCall <rjmccall@apple.com> wrote:

On Jun 13, 2017, at 3:22 AM, David Sweeris <davesweeris@mac.com <mailto:davesweeris@mac.com>> wrote:

On Jun 13, 2017, at 12:18 AM, John McCall via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jun 13, 2017, at 1:08 AM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Mon, Jun 12, 2017 at 9:31 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
I support everything Jon wrote.

+1 Free-for-all brainstorming venue separate from focused proposal discussion.

+1, particularly for this being a section in Discourse ;-)

You posted this with a wink, but I think it's actually quite important. I see some value in having a "sounding board" area where people can talk through ideas that maybe aren't anywhere close to proposal yet

FWIW, I’ve had a few off-list discussions of that nature and found them to be incredibly valuable.

I'm certainly not trying to discourage anyone from having off-list discussions. Have a conversation whenever and wherever it strikes you. But I do think it would be really unfortunate to try to set somewhere else up as a permanent alternative venue.

Not even just for development… a standard repository for “substandard” library(s) would be a great for commonly used types that aren’t quite commonly used enough to include in stdlib, and IMHO, could really help Swift’s ecosystem (I’m not claiming that it’s poor now or anything, but there’s nothing wrong with making it the best we can).

There have been attempts to establish exactly that (I'd describe it as "Boost for Swift"), but without at least a tiny bit of support from Core, such plans imho won't be successful:
It's way to easy to start your own "substandard"...

My favourite example why this would be beneficial are quaternions, which have several implementations in different Apple frameworks :-( — and there are a lot of really simple datatypes and functions which might be implemented over and over, all doing nearly identical things, but none of them compatible with each other.