In the acceptance announcement for SE–270, @John_McCall wrote:
I’m starting this thread for discussion of how the evolution process can be improved. I’ll put my personal thoughts in a separate reply to this thread.
5 Likes
My thoughts:
From a high-level overview perspective, it seems as though the “pitch” phase is for the beginning of a proposal, where the core ideas are promulgated to see if there is community support. Similarly, the “review” phase comes at the end of the process, when the community and core team decide whether to adopt a fully fledged and implemented proposal into the language.
But there is not currently any structure for the middle of the process—how to get from pitch to review.
The middle part is where counterproposals ought to arise, where all the details should be hammered out, and the implementation perfected. I will call this the “development” phase. And today, that takes place privately, by the proposal author(s), out of sight of the community.
In another thread, @John_McCall recently wrote:
This suggests that the pitch phase is the place for developing the proposal. But it also seems to indicate that serious disagreements and counterproposals shouldn’t be hashed out in the pitch thread, but instead should wait for review.
Perhaps an adjustment to the process is needed.
One idea would be to build on the “preview package” idea, to make proposals available for general use prior to their review. A standard like, “the core team is not fundamentally opposed to this proposal in principle, and an implementation has been written”, should be sufficient.
Then people could try out proposals, give feedback to the authors based on direct experience, write counterproposals and so forth all in the “development” phase, rather than attempting to cram everything into the pitch and review. In other words, to recognize and strengthen the middle section of the process, instead of just the beginning and end.
3 Likes
I think it's inevitable that new arguments and counter-proposals are going to come up during review. The audience is larger, and feedback has more urgency because it might be the last opportunity to make a change. Forbidding substantive changes after a proposal enters review would destroy a lot of the value of review: people would have to participate in the entire proposal process if they want their voice to matter, which would be hugely demanding. But if we don't forbid substantive changes, we'll still be in this basic position where major questions come out during review that need to be addressed. So while I do think it'd be good to pre-announce proposals a few weeks before their review starts, I don't think it'll have much impact on the problem here.
5 Likes
I wonder if we should be more willing to send a proposal back to the pitch phase if there are substantial concerns during review. Authors might then be able to more constructively work with the community to refine the areas of concern raised during the initial review. People who had concerns during the review may be more likely to participate in the subsequent re-pitch.
4 Likes
I’m not sure if this was addressed at me, but I hope my comment didn’t come across as advocating to forbid things during review. I certainly wouldn’t want that.
I was raising the possibility of adding more structure to the process before review, not changing the review itself.
2 Likes
This is an interesting question. The Core Team has taken this approach before, where we request broad revisions to a proposal; however, I believe we've only done this at the author's request or when making a concrete decision to reject some major aspect of the current proposal, not just when we think some new idea should be weighed by the community.
I'd say the Core Team has a couple of lines of thought here:
-
In general, we feel that a proposal should reflect the earnest and cohesive design opinions of the author. The pitch phase is there to provide advice and guidance to the proposer, not to be a formal obstacle which has to be satisfied before a proposal can run. If the proposer feels that their original design is best, they're under no obligation to change the substance of their proposal (although we do ask that the proposal's Alternatives Considered section honestly discuss any substantial counter-proposals).
-
Along the same lines, we understand that many "substantial concerns" are simply disagreements that the proposer may feel that they're already adequately responded to. (We do usually allow edits to the proposal's Alternatives Considered during the review.) The only real way to resolve such disagreements is to gather feedback and make a decision.
The upshot of both lines here is that it doesn't make sense to force something back into the pitch phase unless the Core Team requires substantive changes.
Okay. I'm just making a point about the limits of what I think that extra structure before review can achieve.
I think the review period needs to be lengthened. It's not a small thing to add something to the standard library, and 7 days is not enough to allow everybody who wants to comment to do so. Participants don't just have to read the proposal - they also need to find the time to read the discussion thread. I find that latter part can require careful thought and re-reading, and takes much more time than reading the proposal itself. To be clear: it's really great to have those discussions, but it would also be nice to give them the time they deserve.
I also agree with @Nevin about having a demo period in the preview package. That period can be announced like a review, with a discussion thread, and should also be long enough for everybody to have a say. Hopefully lots of people make use of the preview package, so just being there will be an advertisement of the proposal.
2 Likes
To be clear, it’s a goal that reviewers shouldn’t need to read the review threads, and one thing I’d like to see here is suggestions about how to draw attention to important points from review discussion that other reviewers might want to pay attention to.
Does Discourse allow us to add something like likes, but maybe with the 
or 
characters? As if to say: "I don't necessarily agree with you, but this is an interesting point".
Even without reading the review thread, I think we need longer than the formal 7 day period. Sometimes you have a busy spell or are on holiday, and later find that you've missed a review that you would have liked to participate in.
7 Likes
The idea of just extending reviews more is certainly worth considering, thank you.
Within the Evolution: Proposal Reviews folder, perhaps each review thread could be placed into its own subfolder (i.e., a folder labelled with the review title and number). If a debate or counter-proposal warrants special attention, the review manager could create a new thread, in the same folder, with a title that solicits input on the specific debate/counter-proposal.
That new, targeted thread will float to the top of the Evolution pile. By virtue of its title, it should grab attention. By virtue of its focus, it should generate the desired feedback.
3 Likes
That is what always confused me, I have seen proposals sent back to the pitch phase for re-consideration and I liked this. It allowed people to join the new thread and polish the proposal further. However recently I only always wonder, "how is this complex proposal back to another review round without being discussed in a pitch phase after it was returned?". This private re-evaluation phase is definitely not welcome for me personally and maybe for the community as well, as it does not give us any chance to provide feedback anymore.
Look at the property wrapper review. Some of the proposals part were re-evaluated privately some where made public, each review round had sneaky feature that almost only those people coughed that followed esch and every thread.
I think if a proposal is returned, then it should be announced and there should be a new pitch thread, regardless of its length (it could be short, but it could be massive like it was with property wrappers). I think this would be the golden mid which is only fair for everyone.
5 Likes
FWIW the behavior seems to be for the review to last as long as meaningful discussion is happening. That along with the typical pitch phase discourse means it's unlikely for most substantive support/criticism to not exist at the end of a review period. I see the proposal review as more of a peer review of the technical/philosophical design of a feature rather than a democratic yes-no vote. In that context it's not important that every voice is heard, so much as every idea is heard and evaluated.
That said, given the review period can often continue past the cut off anyway, some sort of official "optional extended discussion period" of an additional week seems like it would fit.
1 Like
I agree that making review periods longer would be helpful. Another piece of process that I thought was going to become standard but never actually caught on was announcing on the forums well ahead of time once a proposal had review dates it was scheduled for. Having a relatively brief review period combined with little to no warning about when that review is actually going to happen makes it difficult to plan around writing a thorough review, even for proposals that I have strong feelings about.
1 Like
Maybe 3 weeks per pitch review is better, and take voting in the last week for visual representation.
The Core Team has made it clear before that a community "vote" is not an appropriate way to think about how decisions are made for Swift, and I would oppose adding it as an explicit phase of the Evolution process. IMO, it would only make it a pain for the Core Team to accept/reject a proposal against the majority, and would result in discussions along the lines of "why do we even take a vote if the Core Team isn't going to listen to us?" It's not clear to me what value a vote would provide to the current process, so the added burden to the Core Team of justifying unpopular decisions seems to be not worth it, to me.
8 Likes
This is the crux of it. Move the task of identifying issues most worth of follow up from the proposer (and the general thread) to the review manager. I could see a couple of approaches:
- With a three week review, the review manager could do a summary at the end of week one and week two of the particular issues already brought up that deserve more attention. This could be a formal process perhaps summing up what the core team thinks most needs review.
- No idea how feasible this would be but it would be nice to be able to “subscribe” to a concept at the pitch stage so that you could get a personalized call to action when a pitch enters review. For people who participate irregularly but still have valuable perspectives that might be a nice way to draw them in.
7 Likes
I think it would be good to establish a more positive attitude towards spinning up new threads.
Afaik there's no strict real rule that says all discussion about a specific topic should stay in a single place, but it's quite common that people complain when a second thread is created.
Why "multithreading"?
Threads in Discourse are sequential lists, not trees — so discussions are often scattered and mixed with other (sub)topics. This is especially bad when people start loosing themselves in debates about details that might not even be relevant for the actual pitch. I think that's fine, but it would be nice if people created separate topics to discuss the importance of Newton for computer science, or how the world would look like if the IBM had chosen a Motorola CPU for its first PC.
Counter proposals
The Evolution forum could be a much nicer place if pitch threads would always focus on improving a proposal (without trying to wreck ideas). But when you bring together many people with conflicting goals and opinions, you end up with a "destructive" discussion easily.
Splitting could make the group of participants not only smaller, but also more homogenous, thus reducing friction.
Instead of pointing out why "the enemy proposal" is worse, the question would be "why is our idea better?" and ideally, we would end up with a compact list of advantages for each possibility, which then can be presented in the review thread.
Who owns a thread?
I don't think the author of a pitch should have special (technical) rights, but when a thread starter acknowledges a concern and solicits to move on, I'd consider to grant them the privilege of moderation. The goal should be to help, and it's up to the author to decide what is helpful.
Being supportive does not mean to stay silent when you spot a deficiency, though — I'd say the opposite is true:
You can only address shortcomings when you are aware of them.
Also, if I came to the conclusion that a pitched change could only work after someone solves the halting problem :-), mentioning that could save the author some work...
I'd be especially careful to have read the whole thread before expressing negative feedback, though.
But what if an idea is just bad?
Well, if a proposal is outright terrible, it will never pass review, won't it?
6 Likes
It's interesting that most of the suggestions here are about adding more stages to the process and making review threads run longer and receive more posts, given that this discussion was spawned by a review process that was described as “particularly drawn-out” and “tested the patience of some of the participants”. It's always easy to argue for more process, more scrutiny and more input from more people, but I don't know how that helps here. I don't feel particularly strongly about this, but perhaps in this particular case the core team could have been bolder about accepting some parts of the proposal and/or narrowing the discussion instead of sending it back to the start twice.
2 Likes