Why are these evolution proposals not moving forward?

Hi Kelvin,

You are right to flag the lack of transparency here. I apologize for my part in that. We should respond as a core team officially to each of these. Putting this official response together has been a long-standing to-do for the core team, and has ended up being pushed out in favor of the more urgent task of wrapping up the recent release.

In the short term, here is my own personal non-official feedback on why some of these proposals are currently in a holding pattern. I'll group them together, and fold in a couple of additional ones to help flesh out the narrative a little.

Previewing Mechanism and ABI considerations

This applies to most of the std lib proposals (and incidentally, is why I labelled those PRs on GitHub, so one could see just that subset). I recently pitched a new way of adding to the standard library by first previewing in a package. In my view, agreeing on this mechanism is probably the first thing that needs to be done before we make further substantial additions to the standard library. Now that we have an ABI, that ABI is forever and it increases the level of care we take in making additions.

I'd put the proposal to chunk a sequence into this bucket. (It's such a good proposal we have two :slight_smile: something that is also a cause for a PR stalling...). This is definitely a non-trivial problem to solve lazily (the preferred way to solve it for collections) but since it introduces a type, it'd be really good to get some real-world use and feedback before it's locked into the ABI.

Technical Blockage

A couple of proposals are unfortunately blocked by the requirement to mark conformances with availability. Specifically the lazy and zip proposals. These are IMO very well merited proposals that are in some way almost bug fixes, but need to wait until the availability issue is resolved. It's a good question whether the PRs should be closed in the mean-time as inactionable, or left open.

More Thought Needed

There a few proposals where the main blocker is that they need more time and consideration before they're ready to review. Perhaps more time or input in the pitch phase, or more thorough alternatives considered. You bring up the character literals proposal: that's an excellent example of a proposal where it did come to review, and in hindsight this was a mistake, because there was much about the original proposal that was just not ready for review. It's important we not make that mistake a second time.

There are other examples where what is being proposed is a fairly narrow targeted addition, where what we need instead is a more holistic survey of the problem space. I would put map sorting into this bucket. It sugars some use cases, but does not address others (such as sorting by multiple criteria, or partial sorting). Ideally, when this is the case, instead of targeted proposals, there would be a full survey of the problem space from which a series of proposals fall out. This way we can be confident that the final set of APIs relating to sorting are consistent and composable because they've all been designed together, fully cover the problem space etc.

Given the place we are in within the release cycle, now would be a perfect time to do this kind of thing. Especially now we are post-ABI stability and less constrained in terms of additions to the standard library. And again, resolving the approach to previews is key here, in order to ensure there is a real-world feedback cycle involved before they're finalized.

Trivial Additions

A large number of the additions being proposed are trivially composable. This is a topic that has come up quite a few times. We've given fairly clear guidance in the past that trivially composable functions are not something that generally make for good proposals: that proposals need to instead be targeted at solving difficult but common problems, or help resolve a serious correctness or performance issue.

Despite this, we've not seen many proposals that truly fulfill the criteria for addition. Instead, we continue to get many trivially composable proposals, but coupled with thin justifications of why they fulfill the other criteria. For example, the fold1 version of reduce has a justification on the basis of correctness: that if you are multiplying rather than summing a sequence you might supply 0 instead of 1 for the initial value by accident. This is really stretching the bounds of avoiding correctness issues beyond its breaking point, as I don't think it can be reasonably argued that this is ever a problem.

There's a real risk if only trivial proposals are reviewed and accepted that we end up with a severely lop-sided standard library that only has nice-to-have functions in it. This would lead to a very frustrating experience when hunting through looking for the harder algorithms.

I think it's a good discussion point as to how we solve this problem. Clearly informal steering advice isn't working. The core team could start to just close PRs more aggressively, and give official direction during the pitch phases that something won't make it to review, but this seems pretty heavy handed. One idea I think might work is to put together a list of "missing" more-substantial additions that need to be worked on, and steer prospective proposers towards that. Once we have this, and we start to progress through it, that would relieve the pressure to push back on useful-but-simple proposals, because the risk of a lop-sided library would lessen. @nnnnnnnn has been looking at doing this recently. But you'll have to bear with him, as that in itself is a task that takes time.

24 Likes