Swift phases and mis-timed proposals

This.

I am sitting on a number of ideas that I think have merit (in a non-random use-case non-C# way) and I have no idea when the right time will be to bring them up. Several were marked as "bring forward to Swift 4" and obvious that was never going to happen for them.

I think having a queue to submit "proposals for eventually", written when the inspiration is there, and having a core team review (say once a month or even once a quarter) of their viability for future Swift directions would be amazingly valuable.

An example: Conditional Compilation Blocks proposal by erica · Pull Request #369 · apple/swift-evolution · GitHub

-- E

···

On Jun 11, 2017, at 3:41 PM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

On 11 Jun 2017, at 22:13, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I agree, this makes focusing on the right types of changes much easier and helps us avoid turning Swift into an incoherent conglomeration of random use-cases (*cough*, C#, *cough*).
Now, the question is: when will be the time to officially push the factory initializers proposal and which stage should it be targeting?

I think there's definitely a need for greater clarity; we could really do with better announcements on the list itself as to when the new phase begins and exactly what is in-scope, it also needs to be put more clearly on the swift-evolution repository.

Thing is; people are going to have ideas when they have them, and want to discuss them right away. I've been caught out numerous times with proposals that are almost immediately rejected as out of scope, and still have no idea when I'm supposed to resubmit them.

To be honest it's demoralising, as I find myself apathetic towards my own ideas as I have no idea when to revisit them, and by the time I do I've largely lost interest and moved on to other things.
_______________________________________________

I think there's definitely a need for greater clarity; we could really do with better announcements on the list itself as to when the new phase begins and exactly what is in-scope, it also needs to be put more clearly on the swift-evolution repository.

Thing is; people are going to have ideas when they have them, and want to discuss them right away. I've been caught out numerous times with proposals that are almost immediately rejected as out of scope, and still have no idea when I'm supposed to resubmit them.

To be honest it's demoralising, as I find myself apathetic towards my own ideas as I have no idea when to revisit them, and by the time I do I've largely lost interest and moved on to other things.

···

On 11 Jun 2017, at 22:13, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org> wrote:

I agree, this makes focusing on the right types of changes much easier and helps us avoid turning Swift into an incoherent conglomeration of random use-cases (*cough*, C#, *cough*).
Now, the question is: when will be the time to officially push the factory initializers proposal and which stage should it be targeting?

Everyone: this is a great thread, and I appreciate the candid thoughts here. This is something Ben Cohen and I started chatting about offline and we’ll definitely bring it up for discussion with the rest of the Core Team.

I realize there is a tension here, and a sense of frustration, because ideas sometimes feel like they don’t get air time or it isn’t clear what’s the right conduit for discussing them. I’m still mulling over the thoughts on this thread, but I’d first like to respond to Xiaodi’s point below.

···

On Jun 11, 2017, at 2:54 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

A key question here to be answered is: is this mailing list the appropriate forum for "discussing ideas right away," whatever they might be? or is it more of a working list for bringing forward in-scope proposals in the successive forms of pitch, draft, and proposal review? As far as I can tell, the aim of scoping rules is to nudge it towards the latter, and personally I think that's quite nice in terms of bringing sanity back to my inbox as well as focus to the discussions. By contrast, I have seen blogs, Twitter, and other channels used profitably for discussing ideas right away, and personally I've found off-list communications to be the best way to prepare for the much more difficult task of convincing the discerning crowd here.

I think this captures the current reality fairly well. In the Swift 3 timeframe, the swift-evolution process/venue was brand new and lots of discussion emerged on the swift-evolution list almost overnight. Further, while there was a scope of the changes in Swift 3 they were pretty vast and encompassing. For myself and other members of the Core Team the evolution discussion at times also felt a bit overwhelming. Beyond the investment in shaping the swift-evolution community, it was a struggle to balance discussions, design work, and implementation work that aligned with Swift 3.

The phases of discussion we see in Swift 4 in many ways are a direct response to trying to instill a bit more order to make swift-evolution both less chaotic but also fit with the rhythm of the release. Swift release follow a fairly predictable schedule, roughly following Apple’s yearly OS release cadence with the first “beta” of the major release of the language in June (aligning with WWDC). The engineering and design work for a Swift release pretty much orients around that. My hope is that major design discussions that define a release come earlier in the release cycle, with proposals trickling in as refinements. We saw that a bit in Swift 4 with the String Manifesto and the Ownership Manifesto. We also phrased “phase 1” as being very focused so that critical momentum on Swift 4 could be established without discussions unrelated to the release creating major distractions for the community and the Core Team. With “phase 2” the scope of what was on topic for Swift 4 was opened up a bit, but even that was hit and miss. We took some proposals very late that also landed with hot implementations. As it is, some proposals simply were beyond our capacity to implement, such as SE-0155. The fate of SE-0155 in particular will need to be re-evaluated once we look at the scope for Swift 5 (TBD).

Getting to Xiaodi’s observation, the evolution list is really the most effective as a working list for bringing forward in-scope proposals. There’s a ton of different topics people want to talk about — and their is some obvious angst about getting to those — but at the end of the day there is only so much bandwidth from everyone to pay attention to these discussions and to get seriously invested in them. Maybe moving to Discourse (which is something we still want to do, but have had limited bandwidth to implement) will provide the “off-list” communication channels analogous to the ones Xiaodi describes that keeps those discussions in the Swift.org discussion forums but clearly separates discussion for in-scope proposals versus the side discussions people want to have on topics they are interested in. Even in this setup, however, I think everyone needs to acknowledge that members of the Core Team or anyone else active in the evolution process has limited capacity to participate in only so many discussions. In reality, the discussions that are about in-scope proposals will continue to get the priority attention even if there is a place to discuss things that are out-of-scope.

I appreciate everyone’s thoughts here, and myself and other members of the Core Team are keenly interested in finding ways to improve the nature of swift-evolution for the benefits of its participants and the broader Swift community.

I am sitting on a number of ideas that I think have merit (in a non-random use-case non-C# way) and I have no idea when the right time will be to bring them up. Several were marked as "bring forward to Swift 4" and obvious that was never going to happen for them.

I think perhaps this was obvious in hindsight, but in fairness we **all** were still figuring out how swift-evolution worked at all in the Swift 3 timeframe and we are **still** figuring things out (as evident by this thread). When it is clear that something isn’t in scope for the current release we aren’t always omniscient about how it aligns with the next release. Swift 5, for example, hasn’t been scoped yet. There are certainly ideas, but it’s hard to say what the scoping should be for the next release until we see how the current active release shapes up.

A lot of ideas have merit, and I know it is frustrating to not get active discussion on them as quickly as we all would like. Establishing focus for discussions is really key not only to have themes for a release (as Ben Cohen has eloquently summarized in the past) but also just prioritization. Beyond prioritizing design points that are possibly the most pressing in Swift’s usability/success as a language, there’s only so much that can be done at once. Thus there is a real balancing act between scheduling time to talk about things and actually implementing them. I mentioned this in another reply, but SE-0155 is an example of this. We discussed it, but nobody had adequate enough time to implement it so now it isn’t in scope for Swift 4. It still probably was the right thing to discuss in in the Swift 4 timeframe, but the time we did spend on it was at a direct cost to other things we could have been doing.

I think having a queue to submit "proposals for eventually", written when the inspiration is there, and having a core team review (say once a month or even once a quarter) of their viability for future Swift directions would be amazingly valuable.

This is a good point. I think the concern about a queue is that ideas in it may still be subject to starvation if the queue gets too long. Ideas also can atrophy in their relevance as the language evolves but proposals stay in the queue. It then becomes a delicate matter when closing out old proposals. Having the Core Team review proposals in the queue on a regular basis may be the solution, but I wonder how tenable that would be in practice. The current thinking behind closing out proposals that are out-of-scope is NOT to demoralize community participation in the evolution process, but to engage everyone in thinking about what is in scope for the release and to advocate for an idea when it makes sense for the community — and not just the Core Team — to actively engage on thinking about a proposal. If an idea doesn’t make sense for the current release but does for a later one, then (hopefully) that idea will be brought up again, and possibly incorporating new context. If anything, having a queue of a bunch of written proposals (each which can take significant effort to write) not getting much traction would possibly be even more demoralizing both for authors of the proposal but also everyone else.

The point about understanding “viable for future Swift directions” is key here. Viability really comes down to trajectory for the language. None of us are fully omniscient about what is coming in future releases, but we do have a sense of some of the priorities for the language that we need to tackle, balanced with what **kind** of changes are still acceptable to take into the language depending on the kind of disruption they cause for users, the tools we have to mitigate any pain with those changes, etc. Discussion on swift-evolution does help shape those priorities in both specific ways and in broad strokes. However, it is hard for me to tell if all or any of the ideas you are sitting on are viable — as I don’t know what they are. I know that’s your point about having a queue of proposals. But I am skeptical that such a thing would scale or leave you thinking the process worked any better. What it sounds like to me is we need a better way to both air your ideas for consideration as well as a better understanding of the current thinking about future Swift releases. I don’t have specific ideas yet on either, although at least for the former there has been some interesting ideas mentioned on this thread.

···

On Jun 11, 2017, at 4:47 PM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

I have to constantly suppress all ideas (big or small) because I don't want to flood the mailing list with spam.
In the case of factory initializers, it seemed like something a lot of people yearn for (myself included), so pushing for it in the wake of Swift 4 seemed like an appropriate time.
But otherwise, It would be nice to get a more volatile mailing list specifically for out-of-place (and potentially huge, game-changing) ideas.
We would then collectively compose a monthly or quarterly (or whatever) summary of what has been going on in the volatile mailing list for the people with more timely priorities to be able to keep track of (including the core team).

···

On Jun 12, 2017, at 2:47 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 11, 2017, at 3:41 PM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 11 Jun 2017, at 22:13, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I agree, this makes focusing on the right types of changes much easier and helps us avoid turning Swift into an incoherent conglomeration of random use-cases (*cough*, C#, *cough*).
Now, the question is: when will be the time to officially push the factory initializers proposal and which stage should it be targeting?

I think there's definitely a need for greater clarity; we could really do with better announcements on the list itself as to when the new phase begins and exactly what is in-scope, it also needs to be put more clearly on the swift-evolution repository.

Thing is; people are going to have ideas when they have them, and want to discuss them right away. I've been caught out numerous times with proposals that are almost immediately rejected as out of scope, and still have no idea when I'm supposed to resubmit them.

To be honest it's demoralising, as I find myself apathetic towards my own ideas as I have no idea when to revisit them, and by the time I do I've largely lost interest and moved on to other things.
_______________________________________________

This.

I am sitting on a number of ideas that I think have merit (in a non-random use-case non-C# way) and I have no idea when the right time will be to bring them up. Several were marked as "bring forward to Swift 4" and obvious that was never going to happen for them.

I think having a queue to submit "proposals for eventually", written when the inspiration is there, and having a core team review (say once a month or even once a quarter) of their viability for future Swift directions would be amazingly valuable.

An example: Conditional Compilation Blocks proposal by erica · Pull Request #369 · apple/swift-evolution · GitHub

-- E

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

This touches on two related concerns I’ve long had about how swift-evolution has worked so far.

Concern #1 is that we consider proposals in relative isolation, on their own merits. That’s understandable and wise. Without this focus, we’d never reach conclusions about anything. However, this does tilt the process toward greedy optimization as we traverse the language space. I worry that there’s not enough attention to the big picture.

Concern #2 is that it’s hard to know what to do with a proposal when the ideal answer is “we need to see how it plays out in practice and then decide whether to accept it.” Theoretical discussion untempered by practical prototyping is a great way for a group to talk itself into a bad idea. (This will especially be a problem with future work to build out generics & existentials.)

Swift is already a very featureful language, and not always in a good way. A new feature that makes good sense in the context of a particular problem may make less sense when we consider the totality of its effect on the programmer model: its mental burden, its unforeseen interactions, its surprising pitfalls. How often do we reviewers give only a perfunctory answer to “Does this proposal fit well with the feel and direction of Swift?” How often _can_ we give anything but?

The net result is that IMO we tend to over-favor narrow solutions that solve very specific problems, and under-favor simpler, more general features that are good-enough solutions to _many_ problems at once — not as nice for any _one_ of those problems in isolation, but nicer to work with taken as a whole.

The Great Access Modifiers Wars and the recent fussing of SE-110 fallout are good examples of these problems.

The work on ABI stability is perhaps a good model for non-greedy planning guided by a coherent big picture.

Do these musings make sense? I don’t have any clear answers, but hope the observations are helpful.

Cheers,

Paul

···

On Jun 12, 2017, at 1:29 AM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 11, 2017, at 4:47 PM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

I think having a queue to submit "proposals for eventually", written when the inspiration is there, and having a core team review (say once a month or even once a quarter) of their viability for future Swift directions would be amazingly valuable.

This is a good point. I think the concern about a queue is that ideas in it may still be subject to starvation if the queue gets too long. Ideas also can atrophy in their relevance as the language evolves but proposals stay in the queue. It then becomes a delicate matter when closing out old proposals. …

The point about understanding “viable for future Swift directions” is key here. Viability really comes down to trajectory for the language. None of us are fully omniscient about what is coming in future releases, but we do have a sense of some of the priorities for the language that we need to tackle, balanced with what **kind** of changes are still acceptable to take into the language depending on the kind of disruption they cause for users, the tools we have to mitigate any pain with those changes, etc.

The Great Access Modifiers Wars and the recent fussing of SE-110 fallout

are good examples of these problems.

I want to mention that SE-0110 is part of a suite of proposals originally
asked for or proposed directly by the core team as part of an overarching
attempt to fix issues with how the type system handles function and tuple
types. It was never intended as self-contained fussing over aesthetic
purity, or pretty much anything else it's been accused as in recent posts
to the list. In this case, there is a "coherent big picture"; the problem
is that this big picture was laid out months ago and it's not really fair
to ask newcomers to the discussion to dig through mailing list archives to
try and find the relevant posts.

Regards,
Austin

···

On Mon, Jun 12, 2017 at 12:47 PM, Paul Cantrell via swift-evolution < swift-evolution@swift.org> wrote:

> On Jun 12, 2017, at 1:29 AM, Ted Kremenek via swift-evolution < > swift-evolution@swift.org> wrote:
>
>> On Jun 11, 2017, at 4:47 PM, Erica Sadun via swift-evolution < > swift-evolution@swift.org> wrote:
>>
>> I think having a queue to submit "proposals for eventually", written
when the inspiration is there, and having a core team review (say once a
month or even once a quarter) of their viability for future Swift
directions would be amazingly valuable.
>
> This is a good point. I think the concern about a queue is that ideas
in it may still be subject to starvation if the queue gets too long. Ideas
also can atrophy in their relevance as the language evolves but proposals
stay in the queue. It then becomes a delicate matter when closing out old
proposals. …
>
> The point about understanding “viable for future Swift directions” is
key here. Viability really comes down to trajectory for the language.
None of us are fully omniscient about what is coming in future releases,
but we do have a sense of some of the priorities for the language that we
need to tackle, balanced with what **kind** of changes are still acceptable
to take into the language depending on the kind of disruption they cause
for users, the tools we have to mitigate any pain with those changes, etc.

This touches on two related concerns I’ve long had about how
swift-evolution has worked so far.

Concern #1 is that we consider proposals in relative isolation, on their
own merits. That’s understandable and wise. Without this focus, we’d never
reach conclusions about anything. However, this does tilt the process
toward greedy optimization as we traverse the language space. I worry that
there’s not enough attention to the big picture.

Concern #2 is that it’s hard to know what to do with a proposal when the
ideal answer is “we need to see how it plays out in practice and then
decide whether to accept it.” Theoretical discussion untempered by
practical prototyping is a great way for a group to talk itself into a bad
idea. (This will especially be a problem with future work to build out
generics & existentials.)

Swift is already a very featureful language, and not always in a good way.
A new feature that makes good sense in the context of a particular problem
may make less sense when we consider the totality of its effect on the
programmer model: its mental burden, its unforeseen interactions, its
surprising pitfalls. How often do we reviewers give only a perfunctory
answer to “Does this proposal fit well with the feel and direction of
Swift?” How often _can_ we give anything but?

The net result is that IMO we tend to over-favor narrow solutions that
solve very specific problems, and under-favor simpler, more general
features that are good-enough solutions to _many_ problems at once — not as
nice for any _one_ of those problems in isolation, but nicer to work with
taken as a whole.

The Great Access Modifiers Wars and the recent fussing of SE-110 fallout
are good examples of these problems.

The work on ABI stability is perhaps a good model for non-greedy planning
guided by a coherent big picture.

Do these musings make sense? I don’t have any clear answers, but hope the
observations are helpful.

Cheers,

Paul

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

> The Great Access Modifiers Wars and the recent fussing of SE-110 fallout are good examples of these problems.

I want to mention that SE-0110 is part of a suite of proposals originally asked for or proposed directly by the core team as part of an overarching attempt to fix issues with how the type system handles function and tuple types. It was never intended as self-contained fussing over aesthetic purity, or pretty much anything else it's been accused as in recent posts to the list. In this case, there is a "coherent big picture"; the problem is that this big picture was laid out months ago and it's not really fair to ask newcomers to the discussion to dig through mailing list archives to try and find the relevant posts.

Right, sorry to pick on 110. I realize that proposal has taken a lot of abuse and become a bit of a boogeyman.

I do realize that there was a coherent (and good!) plan behind the sequence of SEs that lead to 110. And for the record, I DO NOT accuse anyone of “self-contained fussing over aesthetic purity!” More generally, I do not approve of the peanut gallery’s occasional degeneration into nasty ad hominems. I have the utmost trust in the expertise and motivation of the core team and proposal authors, who have done heroic, excellent work.

The problem here as I see it is that that chain of tuple-related SEs that lead to 110, once approved, had unintended consequences for dictionary-as-collection operations and RxSwift — consequences that were hard to foresee in discussion, but that prototyping could likely have revealed earlier. (My concern #2.)

We are now faced with the unpleasant prospect of working out those consequences under a tight timeline, and thus looking for a local solution within tight design constraints. (My concern #1.) It seems likely to me that we’re going to end up with yet another little language wrinkle that solves the immediate problem, but adds to language bloat.

That’s probably the best we can do for the concerns raised by 110 for now; I do not mean to derail or denegrate that debate. My argument is that there may be an underlying process problem — and thus process solution — to consider for future work.

Cheers, P

···

On Jun 12, 2017, at 3:03 PM, Austin Zheng <austinzheng@gmail.com> wrote:

Regards,
Austin

On Mon, Jun 12, 2017 at 12:47 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> On Jun 12, 2017, at 1:29 AM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
>
>> On Jun 11, 2017, at 4:47 PM, Erica Sadun via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
>>
>> I think having a queue to submit "proposals for eventually", written when the inspiration is there, and having a core team review (say once a month or even once a quarter) of their viability for future Swift directions would be amazingly valuable.
>
> This is a good point. I think the concern about a queue is that ideas in it may still be subject to starvation if the queue gets too long. Ideas also can atrophy in their relevance as the language evolves but proposals stay in the queue. It then becomes a delicate matter when closing out old proposals. …
>
> The point about understanding “viable for future Swift directions” is key here. Viability really comes down to trajectory for the language. None of us are fully omniscient about what is coming in future releases, but we do have a sense of some of the priorities for the language that we need to tackle, balanced with what **kind** of changes are still acceptable to take into the language depending on the kind of disruption they cause for users, the tools we have to mitigate any pain with those changes, etc.

This touches on two related concerns I’ve long had about how swift-evolution has worked so far.

Concern #1 is that we consider proposals in relative isolation, on their own merits. That’s understandable and wise. Without this focus, we’d never reach conclusions about anything. However, this does tilt the process toward greedy optimization as we traverse the language space. I worry that there’s not enough attention to the big picture.

Concern #2 is that it’s hard to know what to do with a proposal when the ideal answer is “we need to see how it plays out in practice and then decide whether to accept it.” Theoretical discussion untempered by practical prototyping is a great way for a group to talk itself into a bad idea. (This will especially be a problem with future work to build out generics & existentials.)

Swift is already a very featureful language, and not always in a good way. A new feature that makes good sense in the context of a particular problem may make less sense when we consider the totality of its effect on the programmer model: its mental burden, its unforeseen interactions, its surprising pitfalls. How often do we reviewers give only a perfunctory answer to “Does this proposal fit well with the feel and direction of Swift?” How often _can_ we give anything but?

The net result is that IMO we tend to over-favor narrow solutions that solve very specific problems, and under-favor simpler, more general features that are good-enough solutions to _many_ problems at once — not as nice for any _one_ of those problems in isolation, but nicer to work with taken as a whole.

The Great Access Modifiers Wars and the recent fussing of SE-110 fallout are good examples of these problems.

The work on ABI stability is perhaps a good model for non-greedy planning guided by a coherent big picture.

Do these musings make sense? I don’t have any clear answers, but hope the observations are helpful.

Cheers,

Paul

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I think having a queue to submit "proposals for eventually", written when the inspiration is there, and having a core team review (say once a month or even once a quarter) of their viability for future Swift directions would be amazingly valuable.

This is a good point. I think the concern about a queue is that ideas in it may still be subject to starvation if the queue gets too long. Ideas also can atrophy in their relevance as the language evolves but proposals stay in the queue. It then becomes a delicate matter when closing out old proposals. …

The point about understanding “viable for future Swift directions” is key here. Viability really comes down to trajectory for the language. None of us are fully omniscient about what is coming in future releases, but we do have a sense of some of the priorities for the language that we need to tackle, balanced with what **kind** of changes are still acceptable to take into the language depending on the kind of disruption they cause for users, the tools we have to mitigate any pain with those changes, etc.

This touches on two related concerns I’ve long had about how swift-evolution has worked so far.

Concern #1 is that we consider proposals in relative isolation, on their own merits. That’s understandable and wise. Without this focus, we’d never reach conclusions about anything. However, this does tilt the process toward greedy optimization as we traverse the language space. I worry that there’s not enough attention to the big picture.

I agree that some of the proposals have felt like steps of a hill-climbing algorithm.

A few thoughts here:

- In contrast to the greedy optimization analogy, the series of manifestos in the Swift 4 timeframe (ABI stability, String, Ownership) are intended to provide an umbrella of design for what will turn out to be related but separate SE proposals. The idea of these manifestos is to chart a direction for design and provide the basis for which the proposals related to the goals of the manifesto will be evaluated.

- Many SE proposals have been about language refinements, which fall into a category of topics that have been part of an ongoing design discussion for Swift for years — including long before Swift was publicly announced. While the proposals may sometimes feel disjoint, their evaluation is not necessarily so. Often for me proposals relate (directly or indirectly) to design discussions from previous versions of Swift (including pre-Swift 1) that help provide broader optics when evaluating the merit of the proposals. Sometimes when a proposal comes back for revision it incorporates those optics.

- Sometimes (often?) refinements aren’t part of a grand design. They evolve in the mind space from usage of Swift. In other words, greedy optimization is sometimes just a natural way discussion and design happens. The question in my mind is when these concerns come up should we throttle individual proposals and take more time to take a broader view? Maybe that’s the answer. Having been in discussions in the Core Team meetings I can say that sometimes it is clear there is a broader topic to be discussed, and other times it isn’t.

Concern #2 is that it’s hard to know what to do with a proposal when the ideal answer is “we need to see how it plays out in practice and then decide whether to accept it.” Theoretical discussion untempered by practical prototyping is a great way for a group to talk itself into a bad idea. (This will especially be a problem with future work to build out generics & existentials.)

I agree. Do you have specific thoughts here on what could be done differently?

Swift is already a very featureful language, and not always in a good way. A new feature that makes good sense in the context of a particular problem may make less sense when we consider the totality of its effect on the programmer model: its mental burden, its unforeseen interactions, its surprising pitfalls. How often do we reviewers give only a perfunctory answer to “Does this proposal fit well with the feel and direction of Swift?” How often _can_ we give anything but?

The net result is that IMO we tend to over-favor narrow solutions that solve very specific problems, and under-favor simpler, more general features that are good-enough solutions to _many_ problems at once — not as nice for any _one_ of those problems in isolation, but nicer to work with taken as a whole.

The Great Access Modifiers Wars and the recent fussing of SE-110 fallout are good examples of these problems.

I think the Great Access Modifier Wars benefit from a lot of hindsight 20/20. The question in my mind is how do we recognize when we are entering into such a topic again, and what steps do we take “consider the totality of its effect on the programmer model”, as you say? I have to say that we did *try* and consider the totality, or at least we thought we did, when we discussed the changes to access control. Further, it is sometimes hard to evaluate the full implications of a design change until they are unleashed. Myopia is always a risk in designing the ideal solution, with imperfect information or imperfect reasoning playing a factor in suboptimal decisions. Many domains rely on iteration and refinement as a natural part of making progress given the limited capacity to understand everything up front. Language design really is no different, even though the stakes are sometimes higher because language design choices are so fundamental.

The work on ABI stability is perhaps a good model for non-greedy planning guided by a coherent big picture.

Indeed. The same goes for Ownership and String.

Do these musings make sense? I don’t have any clear answers, but hope the observations are helpful.

They do — and I appreciate them!

I do wonder how to apply some of these suggestions. With big efforts — like revising String — we know from the onset that there is a body of problems that need to be tackled and considered together. Not everything that pops up on swift-evolution is like that. For those topics that pop up on swift-evolution as an observation that something in the language isn’t as awesome as we want it to be, should we apply resistance to taking changes to provide more time to consider the broader context? Specifically, *how* do we make design decisions on such proposals, as you put it, less greedy? Further, is this a rampant problem in your mind or a reaction to some decisions that went badly and had to be later re-considered? It’s easy to reflect on what hasn’t worked well, but any way we change the process has tradeoffs on what it yields.

···

On Jun 12, 2017, at 12:47 PM, Paul Cantrell <cantrell@pobox.com> wrote:

On Jun 12, 2017, at 1:29 AM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 11, 2017, at 4:47 PM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

Oh, no need to apologize! I didn't want to distract from the main point of
your post, which I think is very good. The fact is that SE-0110 introduced
unforeseen consequences, and that we need to deal with them one way or
another (including by reverting, if it comes to that).

The problem in this case is staging a refactor across two major versions of
Swift, in which the user experience is regressed by the first half of the
"necessary" work before it can be fixed by the second half. Throw in source
compatibility as a goal and you have a problematic situation.

That said, I've clarified what I wanted to, so I'll withdraw respectfully
from this conversation and allow others to express their views.

Austin

···

On Mon, Jun 12, 2017 at 1:48 PM, Paul Cantrell <cantrell@pobox.com> wrote:

On Jun 12, 2017, at 3:03 PM, Austin Zheng <austinzheng@gmail.com> wrote:

> The Great Access Modifiers Wars and the recent fussing of SE-110
fallout are good examples of these problems.

I want to mention that SE-0110 is part of a suite of proposals originally
asked for or proposed directly by the core team as part of an overarching
attempt to fix issues with how the type system handles function and tuple
types. It was never intended as self-contained fussing over aesthetic
purity, or pretty much anything else it's been accused as in recent posts
to the list. In this case, there is a "coherent big picture"; the problem
is that this big picture was laid out months ago and it's not really fair
to ask newcomers to the discussion to dig through mailing list archives to
try and find the relevant posts.

Right, sorry to pick on 110. I realize that proposal has taken a lot of
abuse and become a bit of a boogeyman.

I do realize that there was a coherent (and good!) plan behind the
sequence of SEs that lead to 110. And for the record, I DO NOT accuse
anyone of “self-contained fussing over aesthetic purity!” More generally,
I do not approve of the peanut gallery’s occasional degeneration into
nasty ad hominems. I have the utmost trust in the expertise and motivation of
the core team and proposal authors, who have done heroic, excellent work.

The problem here as I see it is that that chain of tuple-related SEs that
lead to 110, once approved, had unintended consequences for
dictionary-as-collection operations and RxSwift — consequences that were
hard to foresee in discussion, but that prototyping could likely have
revealed earlier. (My concern #2.)

We are now faced with the unpleasant prospect of working out those
consequences under a tight timeline, and thus looking for a local solution
within tight design constraints. (My concern #1.) It seems likely to me
that we’re going to end up with yet another little language wrinkle that
solves the immediate problem, but adds to language bloat.

That’s probably the best we can do for the concerns raised by 110 for now;
I do not mean to derail or denegrate that debate. My argument is that there
may be an underlying process problem — and thus process solution — to
consider for future work.

Cheers, P

Regards,
Austin

On Mon, Jun 12, 2017 at 12:47 PM, Paul Cantrell via swift-evolution < > swift-evolution@swift.org> wrote:

> On Jun 12, 2017, at 1:29 AM, Ted Kremenek via swift-evolution < >> swift-evolution@swift.org> wrote:
>
>> On Jun 11, 2017, at 4:47 PM, Erica Sadun via swift-evolution < >> swift-evolution@swift.org> wrote:
>>
>> I think having a queue to submit "proposals for eventually", written
when the inspiration is there, and having a core team review (say once a
month or even once a quarter) of their viability for future Swift
directions would be amazingly valuable.
>
> This is a good point. I think the concern about a queue is that ideas
in it may still be subject to starvation if the queue gets too long. Ideas
also can atrophy in their relevance as the language evolves but proposals
stay in the queue. It then becomes a delicate matter when closing out old
proposals. …
>
> The point about understanding “viable for future Swift directions” is
key here. Viability really comes down to trajectory for the language.
None of us are fully omniscient about what is coming in future releases,
but we do have a sense of some of the priorities for the language that we
need to tackle, balanced with what **kind** of changes are still acceptable
to take into the language depending on the kind of disruption they cause
for users, the tools we have to mitigate any pain with those changes, etc.

This touches on two related concerns I’ve long had about how
swift-evolution has worked so far.

Concern #1 is that we consider proposals in relative isolation, on their
own merits. That’s understandable and wise. Without this focus, we’d never
reach conclusions about anything. However, this does tilt the process
toward greedy optimization as we traverse the language space. I worry that
there’s not enough attention to the big picture.

Concern #2 is that it’s hard to know what to do with a proposal when the
ideal answer is “we need to see how it plays out in practice and then
decide whether to accept it.” Theoretical discussion untempered by
practical prototyping is a great way for a group to talk itself into a bad
idea. (This will especially be a problem with future work to build out
generics & existentials.)

Swift is already a very featureful language, and not always in a good
way. A new feature that makes good sense in the context of a particular
problem may make less sense when we consider the totality of its effect on
the programmer model: its mental burden, its unforeseen interactions, its
surprising pitfalls. How often do we reviewers give only a perfunctory
answer to “Does this proposal fit well with the feel and direction of
Swift?” How often _can_ we give anything but?

The net result is that IMO we tend to over-favor narrow solutions that
solve very specific problems, and under-favor simpler, more general
features that are good-enough solutions to _many_ problems at once — not as
nice for any _one_ of those problems in isolation, but nicer to work with
taken as a whole.

The Great Access Modifiers Wars and the recent fussing of SE-110 fallout
are good examples of these problems.

The work on ABI stability is perhaps a good model for non-greedy planning
guided by a coherent big picture.

Do these musings make sense? I don’t have any clear answers, but hope the
observations are helpful.

Cheers,

Paul

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

- Sometimes (often?) refinements aren’t part of a grand design. They evolve in the mind space from usage of Swift. In other words, greedy optimization is sometimes just a natural way discussion and design happens. The question in my mind is when these concerns come up should we throttle individual proposals and take more time to take a broader view? Maybe that’s the answer. Having been in discussions in the Core Team meetings I can say that sometimes it is clear there is a broader topic to be discussed, and other times it isn’t.

I would like to see an explicit place (maybe another list or a section of the eventual discourse) where we can ideate. Ideation requires a less critical environment than this list has been able to provide, and I think it has been contributing a great deal to our inability to deal with the larger picture. The exception to this are the manifestos provided by the core team, which I would guess are a result of off-list ideation sessions, followed by concentrated refinement.

In the absence of an official place, it seems that there are a lot of ad-hoc places where this is being done in a fragmented way. The issue with this is that, when I see “We had a long discussion on Slack”, I don’t know where this Slack channel is, how to get an invite (or even if I am eligible for an invite). Even if I have access to all of the places this discussion is happening, it is so fractured that going back to find important bits of a part of a conversation later is nearly impossible.

I would also like to see a place where ideas and known issues can be organized and referenced.

A structure that has worked well in member-driven organizations I have been a part of in the past is that of working groups. That is, we would larger identify areas of interest for the future development of swift (e.g. Strings), and then spawn a separate list or discourse section for deeper discussion (anyone who is interested could join). The group’s mandate would be to work through the big picture surrounding that issue (including an ideation stage) and develop a resulting manifesto. That manifesto should be an evolving document with known issues that need to be solved, and a record of attempted solutions which failed and the reason for their failure.

I think this structure would take some pressure off of the core team and allow deep thinking on important topics. It would free the main list to focus on specific proposals based on the guideposts provided by the manifestos. Discussions like the Great Access Modifier War would have been contained to a working group, with the main list only receiving a recommendation at the end.

Concern #2 is that it’s hard to know what to do with a proposal when the ideal answer is “we need to see how it plays out in practice and then decide whether to accept it.” Theoretical discussion untempered by practical prototyping is a great way for a group to talk itself into a bad idea. (This will especially be a problem with future work to build out generics & existentials.)

I agree. Do you have specific thoughts here on what could be done differently?

I think we can partially solve this by having an extra stage of “stability” in our documentation. Right now, things are either officially part of the language or they aren’t (with a very short beta period between WWDC and the Fall). What I would like to see in the documentation is a notion of how sure we are of a particular piece of API or syntax.

Unstable - This is actively under development/design and should be expected to change. It is not safe to build frameworks on top of this.

Pre-Stable - We think we have a design which works, but we need to see how it works in the larger ecosystem before guaranteeing stability. Anything built on this should be prepared for potentially large changes in the future, and be marked “Pre-Stable” themselves.

Stable - This has been extensively tested in the actual swift ecosystem and proven itself a useful addition. Any changes to this in the future are guaranteed to either be source-compatible or have a full depreciation cycle. It is completely safe to build on.

I think being explicit about what we expect to change in the future, and having a notion of “stable” that is truly stable will give us a lot more freedom to experiment as well as room to fix our mistakes.

Thanks,
Jon

···

On Jun 12, 2017, at 5:12 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

I think having a queue to submit "proposals for eventually", written when the inspiration is there, and having a core team review (say once a month or even once a quarter) of their viability for future Swift directions would be amazingly valuable.

This is a good point. I think the concern about a queue is that ideas in it may still be subject to starvation if the queue gets too long. Ideas also can atrophy in their relevance as the language evolves but proposals stay in the queue. It then becomes a delicate matter when closing out old proposals. …

The point about understanding “viable for future Swift directions” is key here. Viability really comes down to trajectory for the language. None of us are fully omniscient about what is coming in future releases, but we do have a sense of some of the priorities for the language that we need to tackle, balanced with what **kind** of changes are still acceptable to take into the language depending on the kind of disruption they cause for users, the tools we have to mitigate any pain with those changes, etc.

This touches on two related concerns I’ve long had about how swift-evolution has worked so far.

Concern #1 is that we consider proposals in relative isolation, on their own merits. That’s understandable and wise. Without this focus, we’d never reach conclusions about anything. However, this does tilt the process toward greedy optimization as we traverse the language space. I worry that there’s not enough attention to the big picture.

I agree that some of the proposals have felt like steps of a hill-climbing algorithm.

A few thoughts here:

- In contrast to the greedy optimization analogy, the series of manifestos in the Swift 4 timeframe (ABI stability, String, Ownership) are intended to provide an umbrella of design for what will turn out to be related but separate SE proposals. The idea of these manifestos is to chart a direction for design and provide the basis for which the proposals related to the goals of the manifesto will be evaluated.

Yes, the manifestos have been a positive and successful approach, and very helpful when present. I mentioned ABI stability, but the string & ownership manifestos also deserve their own huzzahs!

- Many SE proposals have been about language refinements, which fall into a category of topics that have been part of an ongoing design discussion for Swift for years — including long before Swift was publicly announced. While the proposals may sometimes feel disjoint, their evaluation is not necessarily so. Often for me proposals relate (directly or indirectly) to design discussions from previous versions of Swift (including pre-Swift 1) that help provide broader optics when evaluating the merit of the proposals. Sometimes when a proposal comes back for revision it incorporates those optics.

- Sometimes (often?) refinements aren’t part of a grand design. They evolve in the mind space from usage of Swift. In other words, greedy optimization is sometimes just a natural way discussion and design happens. The question in my mind is when these concerns come up should we throttle individual proposals and take more time to take a broader view? Maybe that’s the answer. Having been in discussions in the Core Team meetings I can say that sometimes it is clear there is a broader topic to be discussed, and other times it isn’t.

Perhaps the solution is not necessarily throttling proposals per se, but having some mechanism for routing a proposal to something other than either a review cycle or the freezer: “this needs manifesto-ing,” “this needs prototyping to measure impact on existing code,” “this needs to simmer and find its context before we work it into a proposal,” etc. (That’s related to Daryle’s original message.)

Jon’s idea of shunting some ideas to working groups seems like a good one.

Concern #2 is that it’s hard to know what to do with a proposal when the ideal answer is “we need to see how it plays out in practice and then decide whether to accept it.” Theoretical discussion untempered by practical prototyping is a great way for a group to talk itself into a bad idea. (This will especially be a problem with future work to build out generics & existentials.)

I agree. Do you have specific thoughts here on what could be done differently?

I don’t have any brilliant ideas (or I’d have proposed them already), so I’ll throw out some dubious ones:

1. Variation on Jon’s “unstable feature” idea: Allow a proposal to be “provisionally accepted,” which means that the implementation stays on a branch pending hands-on evaluation. That might mean testing against the source compatibility suite, doing experimental migrations of existing projects to use the new feature, performance testing if performance was the concern, etc. Provisionally accepted SEs would get separate public snapshots before merging. Depending on concerns raised during review, those snapshots could be partial implementations, e.g. non-performant or serious known bugs, but would work enough to gather developer feedback. Voting for provisional acceptance would come with the implied (or explicit?) responsibility of helping with the provisional acceptance testing. Following such testing, the proposal would come back for better informed review.

2. Related idea: do partial implementations or static analysis to determine the frequency & extent of breakage in the compatibility suite _before_ things are floated as proposals, and include results in the “impact on existing code” section. It’s conceivable that, for example, it might be relatively easy to rough out a proposal’s effect on the type system without fully implementing it. However, I don’t know how realistic this is; I get the impression that the front of the Swift compiler pipeline is where a lot of the hardest and ugliest work lives.

3. Variation / related idea: build a toy version of Swift’s type system in Swift for experimentation and consistency checking. By “toy version,” I mean an easy-to-fiddle-with model of type rules that wouldn’t be suitable for a real compiler, but is robust enough to uncover the implications of type system changes. I’m thinking especially of future generics work here. (I did something along these lines as an undergrad, writing an SML model of the type system for a toy OO language. It was surprising how many inconsistencies I spotted just having to make that toy model pass SML’s type checker.)

These ideas seem to me problematic at best … but they’re what I was able to come up with.

FWIW, I thought the Great Renaming was a success, and succeeded because of iterative prototyping. It’s the sort of thing I have in mind with #2. Simply observing the effect of proposed naming conventions on the Cocoa APIs — without even being able to write code with them! — generated lots of useful discussion, and drove the naming conventions to a much better place than they started out. (I’m still relieved that the special exception for first argument labels disappeared, and that we didn’t end up with something like moveToX(_:y:). Phew!)

Swift is already a very featureful language, and not always in a good way. A new feature that makes good sense in the context of a particular problem may make less sense when we consider the totality of its effect on the programmer model: its mental burden, its unforeseen interactions, its surprising pitfalls. How often do we reviewers give only a perfunctory answer to “Does this proposal fit well with the feel and direction of Swift?” How often _can_ we give anything but?

The net result is that IMO we tend to over-favor narrow solutions that solve very specific problems, and under-favor simpler, more general features that are good-enough solutions to _many_ problems at once — not as nice for any _one_ of those problems in isolation, but nicer to work with taken as a whole.

The Great Access Modifiers Wars and the recent fussing of SE-110 fallout are good examples of these problems.

I think the Great Access Modifier Wars benefit from a lot of hindsight 20/20. The question in my mind is how do we recognize when we are entering into such a topic again, and what steps do we take “consider the totality of its effect on the programmer model”, as you say? I have to say that we did *try* and consider the totality, or at least we thought we did, when we discussed the changes to access control. Further, it is sometimes hard to evaluate the full implications of a design change until they are unleashed. Myopia is always a risk in designing the ideal solution, with imperfect information or imperfect reasoning playing a factor in suboptimal decisions.

This is true. Any process that relies on perfect design up front is doomed. As we all well know.

Many domains rely on iteration and refinement as a natural part of making progress given the limited capacity to understand everything up front. Language design really is no different, even though the stakes are sometimes higher because language design choices are so fundamental.

This is the crux of it, isn’t it? Compilers are hard to build, thus iterations are slow and costly. And if iterations span language releases, the cost of iteration is multiplied by the language’s user base. My groping about for suggestions above boils down to “make iterations less costly.” That is complementary to Jon’s working group suggestion, which runs more along the lines of “make fewer iterations necessary.”

The work on ABI stability is perhaps a good model for non-greedy planning guided by a coherent big picture.

Indeed. The same goes for Ownership and String.

Do these musings make sense? I don’t have any clear answers, but hope the observations are helpful.

They do — and I appreciate them!

I do wonder how to apply some of these suggestions. With big efforts — like revising String — we know from the onset that there is a body of problems that need to be tackled and considered together. Not everything that pops up on swift-evolution is like that. For those topics that pop up on swift-evolution as an observation that something in the language isn’t as awesome as we want it to be, should we apply resistance to taking changes to provide more time to consider the broader context? Specifically, *how* do we make design decisions on such proposals, as you put it, less greedy?

Identifying when an isolated idea needs broader context is part of it.

Better weighing mental burden against benefit is part of it.

Part of it is mapping specific feature requests back to the _problem_ that’s motivating the request, and keeping a more globally visible pool of recurring pain points from which to identify bodies of related problems and perhaps see more cross-cutting solutions.

All of Jon’s ideas (idea generation venue, working groups, unstable features) could help with these things.

Further, is this a rampant problem in your mind or a reaction to some decisions that went badly and had to be later re-considered? It’s easy to reflect on what hasn’t worked well, but any way we change the process has tradeoffs on what it yields.

“Rampant” is much too strong. It’s more a simmering concern, a vague unease.

Although I used meddlesome past proposals as examples, they’re not the only source of that unease. I’m particularly worried about future generics & existentials work. More generally, I’m concerned about the language growing in a piecemeal fashion that causes it to bulk up with quirks and arcane features, yet simultaneously shy away from the ambitious cross-cutting rethinking that keeps software elegant. It’s what I said in my previous message:

The net result is that IMO we tend to over-favor narrow solutions that solve very specific problems, and under-favor simpler, more general features that are good-enough solutions to _many_ problems at once — not as nice for any _one_ of those problems in isolation, but nicer to work with taken as a whole.

I should clarify that I view the Swift evolution process as largely successful. Swift is evolving admirably, in leaps and bounds. Far more languages changes have been successful than not. Despite all the sound and fury on this list, and despite the fact that the core team still does most of the heavy lifting, community input has produced some great ideas, helped refine many a proposal, and caught oversights that would have been painful if released in the wild.

Swift’s mindful design is a defining feature of the language, and I wouldn’t want to mess with the core team’s process in a way that undermines that. Whatever it is that I’m trying to get at — and I’m not entirely sure myself! — it’s a light hand on the ship’s wheel and not a radical rethink of the whole evolution process.

Cheers,

Paul

···

On Jun 12, 2017, at 7:12 PM, Ted Kremenek <kremenek@apple.com> wrote:

On Jun 12, 2017, at 12:47 PM, Paul Cantrell <cantrell@pobox.com> wrote:

On Jun 12, 2017, at 1:29 AM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 11, 2017, at 4:47 PM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

I feel like I’m missing some part of the motivation for this thread. Let me try to explain why:

Lets say you’re interested in proposing a new feature or idea today, and are told that it is out of scope. One of two things happen. When the next release comes around, either:

1) if you’re still engaged with swift-evolution, you can bring it back up.

2) if you’re not still engaged, it will get dropped unless someone else is interested in championing it.

What good does a “queue” of proposals in the freezer do? In practice, getting a proposal to happen frequently requires editing and iteration, not to mention active discussion about motivation. We have no shortage of important proposals to get through in each release, why should we make it easier for proposals with no active proponent? Wouldn’t that just encourage people to drop off “drive-by” proposals for ideas?

-Chris

···

On Jun 12, 2017, at 10:07 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org> wrote:

Perhaps the solution is not necessarily throttling proposals per se, but having some mechanism for routing a proposal to something other than either a review cycle or the freezer: “this needs manifesto-ing,” “this needs prototyping to measure impact on existing code,” “this needs to simmer and find its context before we work it into a proposal,” etc. (That’s related to Daryle’s original message.)

I support everything Jon wrote.

+1 Free-for-all brainstorming venue separate from focused proposal discussion.

+1 Working groups when helpful.

+1 Longer public incubation for unstable / experimental features (but that idea executed & communicated with caution, preferably with active support from the compiler for preventing careless dependance).

P

···

On Jun 12, 2017, at 9:16 PM, Jonathan Hull <jhull@gbis.com> wrote:

On Jun 12, 2017, at 5:12 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

- Sometimes (often?) refinements aren’t part of a grand design. They evolve in the mind space from usage of Swift. In other words, greedy optimization is sometimes just a natural way discussion and design happens. The question in my mind is when these concerns come up should we throttle individual proposals and take more time to take a broader view? Maybe that’s the answer. Having been in discussions in the Core Team meetings I can say that sometimes it is clear there is a broader topic to be discussed, and other times it isn’t.

I would like to see an explicit place (maybe another list or a section of the eventual discourse) where we can ideate. Ideation requires a less critical environment than this list has been able to provide, and I think it has been contributing a great deal to our inability to deal with the larger picture. The exception to this are the manifestos provided by the core team, which I would guess are a result of off-list ideation sessions, followed by concentrated refinement.

In the absence of an official place, it seems that there are a lot of ad-hoc places where this is being done in a fragmented way. The issue with this is that, when I see “We had a long discussion on Slack”, I don’t know where this Slack channel is, how to get an invite (or even if I am eligible for an invite). Even if I have access to all of the places this discussion is happening, it is so fractured that going back to find important bits of a part of a conversation later is nearly impossible.

I would also like to see a place where ideas and known issues can be organized and referenced.

A structure that has worked well in member-driven organizations I have been a part of in the past is that of working groups. That is, we would larger identify areas of interest for the future development of swift (e.g. Strings), and then spawn a separate list or discourse section for deeper discussion (anyone who is interested could join). The group’s mandate would be to work through the big picture surrounding that issue (including an ideation stage) and develop a resulting manifesto. That manifesto should be an evolving document with known issues that need to be solved, and a record of attempted solutions which failed and the reason for their failure.

I think this structure would take some pressure off of the core team and allow deep thinking on important topics. It would free the main list to focus on specific proposals based on the guideposts provided by the manifestos. Discussions like the Great Access Modifier War would have been contained to a working group, with the main list only receiving a recommendation at the end.

Concern #2 is that it’s hard to know what to do with a proposal when the ideal answer is “we need to see how it plays out in practice and then decide whether to accept it.” Theoretical discussion untempered by practical prototyping is a great way for a group to talk itself into a bad idea. (This will especially be a problem with future work to build out generics & existentials.)

I agree. Do you have specific thoughts here on what could be done differently?

I think we can partially solve this by having an extra stage of “stability” in our documentation. Right now, things are either officially part of the language or they aren’t (with a very short beta period between WWDC and the Fall). What I would like to see in the documentation is a notion of how sure we are of a particular piece of API or syntax.

Unstable - This is actively under development/design and should be expected to change. It is not safe to build frameworks on top of this.

Pre-Stable - We think we have a design which works, but we need to see how it works in the larger ecosystem before guaranteeing stability. Anything built on this should be prepared for potentially large changes in the future, and be marked “Pre-Stable” themselves.

Stable - This has been extensively tested in the actual swift ecosystem and proven itself a useful addition. Any changes to this in the future are guaranteed to either be source-compatible or have a full depreciation cycle. It is completely safe to build on.

I think being explicit about what we expect to change in the future, and having a notion of “stable” that is truly stable will give us a lot more freedom to experiment as well as room to fix our mistakes.

Thanks,
Jon

I think this goes to the discussion above about what kind of list this is
to be. We've got, on the one hand, a significant number of proposals of the
following sort: how should indexing into the middle of a grapheme be
handled? which integer protocols should have the bitwise operators? These
require editing and iteration, as you say, and very active back-and-forth
discussion. These days it seems these sorts of threads rarely gain the
greatest attention. The most useful kind of feedback in such a discussion
would be a detailed and robust critique; much less useful is the ever more
common "+1," and storing the proposal in a freezer is the surest way to
irrelevance as the detailed design can quickly become nonsensical due to
even small changes elsewhere.

There are, however, a significant number of conversations that, at base,
are declarations that there exists some problem or issue. It's not
straightforwardly a bug that can be filed, and it's not a user knowledge
issue that can be solved by a question to swift-users, but it's something
else that may require a language change. This is, of course, a perfectly
understandable state of affairs for which some forum should probably exist.
However, the "format" of Swift Evolution requires a proposed solution and
detailed design, so the author feels compelled to put down _something_ that
might alleviate the problem. Frequently, later in the discussion, the
author will reply that he or she isn't actually married to the proposed
solution and/or does not really know if the detailed design will work at
all, but wanted the community to acknowledge that a problem or issue exists
and needs attention. Critiquing the placeholder proposed solution and
detailed design is kind of beside the point, even though it's the
ostensible purpose of this list. If anything is helpful in this scenario,
it would be feedback that validates and expounds on the issue (or in the
absence of that, "+1"). However, there's not an iterative process that will
take you from that point to an implementable design. It might be, however,
a topic that fits nicely into a "queue" of some sort.

There's another class of conversations that, from what I can tell, boil
down to a request for a feature--often one that is highly desired by many
people. The feature doesn't yet exist because it would require extensive
design and implementation effort. Perhaps the author is unaware that the
feature is already widely desired, or perhaps the author simply wants to
make it known that they wish that it would take higher priority. The actual
motivation for starting a thread isn't the promotion of any particular
design for the desired feature, just the feature itself. Also a perfectly
understandable situation, and should be accommodated in some form,
somewhere. However, Swift Evolution requires a detailed design, so the
author feels compelled to sketch out some sort of placeholder; sometimes,
the "detailed design" recites more and more elaborate variations of the
feature but makes no attempt to actually describe how it will come to be.
Again, little point in critiquing it. Again, there's not an iterative
process from there to implementable design. This time the topic is almost a
meta-discussion on the proper prioritization of items in the "queue" of
ideas, and the responses that are relevant to such a conversation (short of
designing and implementing the feature) can really only be "yes, I want
this feature too."

···

On Tue, Jun 13, 2017 at 1:18 AM, Chris Lattner via swift-evolution < swift-evolution@swift.org> wrote:

On Jun 12, 2017, at 10:07 PM, Paul Cantrell via swift-evolution < > swift-evolution@swift.org> wrote:

Perhaps the solution is not necessarily throttling proposals per se, but
having some mechanism for routing a proposal to something other than either
a review cycle or the freezer: “this needs manifesto-ing,” “this needs
prototyping to measure impact on existing code,” “this needs to simmer and
find its context before we work it into a proposal,” etc. (That’s related
to Daryle’s original message.)

I feel like I’m missing some part of the motivation for this thread. Let
me try to explain why:

Lets say you’re interested in proposing a new feature or idea today, and
are told that it is out of scope. One of two things happen. When the next
release comes around, either:

1) if you’re still engaged with swift-evolution, you can bring it back up.

2) if you’re not still engaged, it will get dropped unless someone else is
interested in championing it.

What good does a “queue” of proposals in the freezer do? In practice,
getting a proposal to happen frequently requires editing and iteration, not
to mention active discussion about motivation. We have no shortage of
important proposals to get through in each release, why should we make it
easier for proposals with no active proponent? Wouldn’t that just
encourage people to drop off “drive-by” proposals for ideas?

I’m not sure documentation is sufficient… If I only wanted to use the “stable” features, would I have to read the docs on every standard library type I use? And what about unstable or pre-stable compiler features? It’s not like I can right-click on a var and get the type inference rules, for example.

Now, I wouldn’t mind the ability to vote (or at the core team’s discretion) that we accept a proposal into a separate “pre-stable” or “staging” toolchain, with the understanding that if it doesn’t cause any major issues it’ll automatically be accepted for the next major release. If issues are found, then we’d have a chance to re-review & refine the proposal before committing it to the main toolchain and its source-compatibility requirements. If, upon re-review, the proposal is rejected, it would be deprecated in the next major release’s staging toolchain and removed in the major release after that.

To clarify, the timeline would be something like:

Prior to Swift N: We accept SE-xxxx for Swift N, as well as SE-yyyy and SE-zzzz for Swift N Staging.
Release Swift N:
  - Standard: implements SE-xxxx
  - Staging: implements SE-yyyy and SE-zzzz
Prior to Swift N+1: We accept a proposal to revert SE-zzzz because it turns out to be a horrible idea
Release Swift N+1:
  - Standard: implements SE-xxxx, SE-yyyy
  - Staging: deprecates SE-zzzz
Release Swift N+2:
  - Standard: implements SE-xxxx, SE-yyyy
  - Staging: reverts SE-zzzz

This would give the people whose code relies on SE-zzzz probably between 1-2 years to find a work-around, given the current pace of major releases, and depending on exactly where in the timeline the problems get discovered. (As a last resort, you could leave the Swift N or N+1 staging toolchains installed, refactor any features relying on SE-zzzz into their own module, and move the rest of your app’s code on to Swift N+2. Certainly not ideal, but AFAIK, it's an option.) And if you’re worried about it, just use the stable toolchain.

I do have some concerns, though… My understanding is that Rust does something similar and has an “experimental” compiler release… Apparently, all the “cool” stuff is done there, which makes it the “standard" compiler, and relegates the actually-stable compiler to “old version” status. Or so I hear, anyway. I haven’t really used it much. Anyway, I wouldn’t want us to get in a position where everyone’s using some feature of the staging toolchain, but we can’t merge it into the stable toolchain because maybe it breaks some niche use-case that we need to support or something. We do have one thing going for us, though, that Rust doesn’t… a large portion of Swift’s user base needs to ship to Apple's App stores, and Apple controls what toolchains you’re allowed to use for that. OTOH, if we say "no staging toolchain for the app store”, then we have to ask if enough people will use it for the issues to be found.

- Dave Sweeris

···

On Jun 12, 2017, at 7:16 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 12, 2017, at 5:12 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jun 12, 2017, at 12:47 PM, Paul Cantrell <cantrell@pobox.com> wrote:

Concern #2 is that it’s hard to know what to do with a proposal when the ideal answer is “we need to see how it plays out in practice and then decide whether to accept it.” Theoretical discussion untempered by practical prototyping is a great way for a group to talk itself into a bad idea. (This will especially be a problem with future work to build out generics & existentials.)

I agree. Do you have specific thoughts here on what could be done differently?

I think we can partially solve this by having an extra stage of “stability” in our documentation. Right now, things are either officially part of the language or they aren’t (with a very short beta period between WWDC and the Fall). What I would like to see in the documentation is a notion of how sure we are of a particular piece of API or syntax.

Unstable - This is actively under development/design and should be expected to change. It is not safe to build frameworks on top of this.

Pre-Stable - We think we have a design which works, but we need to see how it works in the larger ecosystem before guaranteeing stability. Anything built on this should be prepared for potentially large changes in the future, and be marked “Pre-Stable” themselves.

Stable - This has been extensively tested in the actual swift ecosystem and proven itself a useful addition. Any changes to this in the future are guaranteed to either be source-compatible or have a full depreciation cycle. It is completely safe to build on.

I think being explicit about what we expect to change in the future, and having a notion of “stable” that is truly stable will give us a lot more freedom to experiment as well as room to fix our mistakes.

What if instead of a FIFO queue, we had a reddit style queue where proposals can bubble up based on community interest? It might help us figure out what to focus on in the 'phase 2' times where we allow a few out of scope proposals through... we just grab a couple off of the top.

Thanks,
Jon

···

Sent from my iPhone

On Jun 12, 2017, at 11:18 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 12, 2017, at 10:07 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org> wrote:

Perhaps the solution is not necessarily throttling proposals per se, but having some mechanism for routing a proposal to something other than either a review cycle or the freezer: “this needs manifesto-ing,” “this needs prototyping to measure impact on existing code,” “this needs to simmer and find its context before we work it into a proposal,” etc. (That’s related to Daryle’s original message.)

I feel like I’m missing some part of the motivation for this thread. Let me try to explain why:

Lets say you’re interested in proposing a new feature or idea today, and are told that it is out of scope. One of two things happen. When the next release comes around, either:

1) if you’re still engaged with swift-evolution, you can bring it back up.

2) if you’re not still engaged, it will get dropped unless someone else is interested in championing it.

What good does a “queue” of proposals in the freezer do? In practice, getting a proposal to happen frequently requires editing and iteration, not to mention active discussion about motivation. We have no shortage of important proposals to get through in each release, why should we make it easier for proposals with no active proponent? Wouldn’t that just encourage people to drop off “drive-by” proposals for ideas?

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

+1, particularly for this being a section in Discourse ;-)

···

On Mon, Jun 12, 2017 at 9:31 PM, Paul Cantrell via swift-evolution < swift-evolution@swift.org> wrote:

I support everything Jon wrote.

+1 Free-for-all brainstorming venue separate from focused proposal
discussion.

You posted this with a wink, but I think it's actually quite important. I see some value in having a "sounding board" area where people can talk through ideas that maybe aren't anywhere close to proposal yet, but actually encouraging the use of a completely different venue would be pretty unfortunate:

1. That venue would become important to the community, but it wouldn't really be a *part* of the community in some ways. For example, it would be a terrible failure if the fashionable place for (say) talking about new reflection APIs was a website where not everyone felt comfortable participating because some of the established contributors there were jerks. We really want everything in the project to be covered under the Code of Conduct.

2. Today, there's one standard venue for in-depth conversations about Swift evolution: this mailing list. Having one venue is self-reinforcing: if you have a conversation somewhere else, you're always missing people, and convincing people to join your side-conversation naturally makes them ask why you're not having it on the main venue. In contrast, telling people to have conversations elsewhere means intentionally fracturing the community, and there's no reason it would stay with just two venues. We'd probably end up with a ton of short-lived, hard-to-find, poorly-archived discussion groups with their own isolated focus and culture; that's a situation that naturally leads to different groups not realizing that they need to be talking to each other. It'd be more like the salon system than an engineering project.

3. Having a separate venue for early-stage discussions also creates an awkward transition when the proposal is ready to move on. Discussion probably doesn't totally stop in the old venue, and in the new venue the discussion has lost a lot of context, and it's annoying to constantly refer back to the old venue. In contrast, if early discussions are just threads on a different part of the same forum, they can simply be moved around as they mature.

So, in my opinion, we should just bear with the current situation until we have a proper forum.

John.

···

On Jun 13, 2017, at 1:08 AM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:
On Mon, Jun 12, 2017 at 9:31 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
I support everything Jon wrote.

+1 Free-for-all brainstorming venue separate from focused proposal discussion.

+1, particularly for this being a section in Discourse ;-)

Most proposals break down to one of:

* Adjusting the language to add features that match development ideals (usually under a manifesto umbrella)
* Fixing problems in the language that lead to inefficient compiler implementation / unnecessary complexity
* Adding features to the language to match common compilation options found elsewhere (the recent re-emergence of #warning/#error being a good example)
* Fixing problems in the language that lead to unsafe code

* Getting SwiftPM into shape ("ballardization")

* Editing the standard library to remove or add features for a more robust and cleaner API experience
* Adding features to the language to match common language features found in other modern programming languages

My personal interest lies in the fourth bullet ("Fixing problems in the language that lead to unsafe code"). Friends and I have found through experience a fair number of areas where the language can be adjusted to increase safety, and we've brainstormed ways to do so additively without breaking backward compatibility. There never seems to be a good time or place to introduce these ideas.

Of the remaining bullet points, the last 2 (new library features) seem like they *could* be developed, tested, and implemented independently of the standard SE process until the language is at a point (Swift 5 or Swift 6) where it becomes open to additive features.

I imagine that recent discussions like mapped keypaths, ordered sets, `count(where:)`, etc. could have a home for discussion and exploration without getting blocked by "out of scope" if there were a separate "Substandard Library" repository, mailing list, and process (potentially staffed in part or full by non-Apple personnel) that did not rely on internal Apple timelines and delivery benchmarks. Think "farm league for SE" (https://en.wikipedia.org/wiki/Farm_team\). It would lower the burden on SE but provide a way forward to discuss and develop ideas within the framework of coherent language design.

-- E

···

On Jun 13, 2017, at 2:07 AM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Tue, Jun 13, 2017 at 1:18 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jun 12, 2017, at 10:07 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Perhaps the solution is not necessarily throttling proposals per se, but having some mechanism for routing a proposal to something other than either a review cycle or the freezer: “this needs manifesto-ing,” “this needs prototyping to measure impact on existing code,” “this needs to simmer and find its context before we work it into a proposal,” etc. (That’s related to Daryle’s original message.)

I feel like I’m missing some part of the motivation for this thread. Let me try to explain why:

Lets say you’re interested in proposing a new feature or idea today, and are told that it is out of scope. One of two things happen. When the next release comes around, either:

1) if you’re still engaged with swift-evolution, you can bring it back up.

2) if you’re not still engaged, it will get dropped unless someone else is interested in championing it.

What good does a “queue” of proposals in the freezer do? In practice, getting a proposal to happen frequently requires editing and iteration, not to mention active discussion about motivation. We have no shortage of important proposals to get through in each release, why should we make it easier for proposals with no active proponent? Wouldn’t that just encourage people to drop off “drive-by” proposals for ideas?

I think this goes to the discussion above about what kind of list this is to be. We've got, on the one hand, a significant number of proposals of the following sort: how should indexing into the middle of a grapheme be handled? which integer protocols should have the bitwise operators? These require editing and iteration, as you say, and very active back-and-forth discussion. These days it seems these sorts of threads rarely gain the greatest attention. The most useful kind of feedback in such a discussion would be a detailed and robust critique; much less useful is the ever more common "+1," and storing the proposal in a freezer is the surest way to irrelevance as the detailed design can quickly become nonsensical due to even small changes elsewhere.

There are, however, a significant number of conversations that, at base, are declarations that there exists some problem or issue. It's not straightforwardly a bug that can be filed, and it's not a user knowledge issue that can be solved by a question to swift-users, but it's something else that may require a language change. This is, of course, a perfectly understandable state of affairs for which some forum should probably exist. However, the "format" of Swift Evolution requires a proposed solution and detailed design, so the author feels compelled to put down _something_ that might alleviate the problem. Frequently, later in the discussion, the author will reply that he or she isn't actually married to the proposed solution and/or does not really know if the detailed design will work at all, but wanted the community to acknowledge that a problem or issue exists and needs attention. Critiquing the placeholder proposed solution and detailed design is kind of beside the point, even though it's the ostensible purpose of this list. If anything is helpful in this scenario, it would be feedback that validates and expounds on the issue (or in the absence of that, "+1"). However, there's not an iterative process that will take you from that point to an implementable design. It might be, however, a topic that fits nicely into a "queue" of some sort.

There's another class of conversations that, from what I can tell, boil down to a request for a feature--often one that is highly desired by many people. The feature doesn't yet exist because it would require extensive design and implementation effort. Perhaps the author is unaware that the feature is already widely desired, or perhaps the author simply wants to make it known that they wish that it would take higher priority. The actual motivation for starting a thread isn't the promotion of any particular design for the desired feature, just the feature itself. Also a perfectly understandable situation, and should be accommodated in some form, somewhere. However, Swift Evolution requires a detailed design, so the author feels compelled to sketch out some sort of placeholder; sometimes, the "detailed design" recites more and more elaborate variations of the feature but makes no attempt to actually describe how it will come to be. Again, little point in critiquing it. Again, there's not an iterative process from there to implementable design. This time the topic is almost a meta-discussion on the proper prioritization of items in the "queue" of ideas, and the responses that are relevant to such a conversation (short of designing and implementing the feature) can really only be "yes, I want this feature too."