The role of vision documents in Swift evolution

Hello, Swift community.

"Manifestos" have historically served an important role in Swift's design. A manifesto pairs a long-form analysis of the current state of one area of the project with a high-level vision for how that area should evolve in the future. It establishes common terminology, propagates key conceptual models, and sets community expectations about the direction of the project. The impact of a good manifesto can be felt in every conversation that touches on that area in the future.

Unfortunately, manifestos also have a critical flaw, which is that it's never been clear what they really mean in the evolution process. Manifestos are not evolution proposals: they do not undergo evolution review, and they do not get formally accepted or rejected. Historically, they've often been written by active members of the Core Team, and often they've been reviewed and revised by the Core Team before being published, but this is neither guaranteed nor visible to the community. As a result, it can be hard for the community to know whether they should treat any given manifesto as speaking for more than just its author. Moreover, manifestos usually lay out a set of proposals that the author is arguing should be incorporated into Swift. If members of the Core Team seem to support a manifesto, what does that mean for those proposals when they come up for review? Are those reviews merely empty formalities?

These problems are worth fixing. The Swift project is changing how we handle these design documents by introducing a lightweight process for creating and approving them and setting clear expectations about what that approval means. To clearly distinguish documents created under the new process, we are also changing both where they are stored and what they are called. These documents are now called vision documents, and when they have been officially accepted, they will be stored in the evolution repository in the visions directory.

Vision documents usually start by being solicited by the evolution workgroup with authority for that area. For areas within the Swift language and standard library, that is the Language Workgroup. While a vision is being developed, it is called a prospective vision, and it should be clearly identified as such. In this state, the vision carries no implicit endorsement.

Eventually, the appropriate evolution workgroup may decide to officially approve a vision. This is an endorsement of the entire document, but the strength of that endorsement varies from section to section:

  • It is a strong endorsement of the vision's description of the current state of this part of the project. The evolution workgroup agrees with what the vision has to say about the problems the project has in this area.

  • It is a strong endorsement of the vision's stated goals for this part of the language. The evolution workgroup agrees that these are the right goals for evolution in this area to strive for, and it agrees that the vision prioritizes different goals appropriately.

  • It is a somewhat weaker endorsement of the overall approach laid out by the vision. The evolution workgroup agrees that this seems like the right basic approach to take; if it can successfully carried out, it should achieve the goals the vision lays out. However, the evolution workgroup is not committed to the details of the approach, and it may change substantially as the vision is distilled into concrete proposals and reviewed.

  • It is only a very weak endorsement of the concrete ideas for proposals in the vision document. The evolution workgroup thinks these sound like the right ideas in the abstract but is not committed to any of them. The proposals will all need to go through normal evolution review, and they may be rejected or substantially changed from how they appear in the vision.

Once the vision is approved, it acts as a foundation for subsequent pitches and proposals in its area. Pitches and proposals that implement or build on part of a vision should generally link back to the vision document.

Vision documents are artifacts of the design process; they are not substitutes for language or release documentation. It is not expected that authors will continually update the vision document as the proposals emerging from it change. Revision may be appropriate if the vision document is actively causing confusion, for example because of a major shift in terminology since the document's development.

We hope that this new process for vision documents will clarify their role in evolution, and we invite comments and suggestions for how they could be improved.

John McCall
Language Workgroup Chair

79 Likes

This post brought me back. Ages ago, in the olden days when swift-evolution was a mailing list and dinosaurs roamed the earth — i.e 2017 — I expressed concern about the strictly incremental nature of the SE process:

(Ted Kremenek agreed in response that “some of the proposals have felt like steps of a hill-climbing algorithm”, which I thought was wall said.)

It’s satisfying to reflect on how far the process has come since then.

I was worried about the “proposals must have implementations” requirement, but that has borne tremendous fruit: sharper, better-written proposals with a better understanding of quirks, corner cases, and tradeoffs, and far less backtracking in the evolution process.

And contrary to my concerns about the evolution process tending toward “greedy optimization,” Swift has successfully undertaken large-scale evolutions that require holistic design thinking about many interlocking proposals: concurrency, bringing much-needed generality to the generic type system, and now ownership control…the list goes on.

Manifestos played a crucial role in all of that work. Officially bringing them into the fold as vision documents feels like closing the loop on my 2017 concerns. Both my concerns in that old post are well addressed. We have not only a better language than we did 5 years ago, but a better process for evolving it.

No concerns or criticism here, just appreciation for the people who’ve brought Swift so far over the years. It has been a true community effort, and it’s heartening to see a community effort succeed.

36 Likes

Not much to add here. I believe this is the right way forward as a way to direct our focus to specific aspects of the language that ought to be improved.

I do have a couple questions:

  1. Are all member of the Swift community encouraged to submit their own vision document, as they would perhaps submit their own pitches? In other words, would vision documents submitted by anyone outside of the vision's designated workgroup be taken into consideration, or simply ignored?
  2. Will all the other informal vision documents (e.g. Variadic Generics, Macros) be retrofitted and subject to this process, or at least in some way specified as to how much in those manifestos is strongly agreed upon by the Core Team?

Vision documents are usually solicited. Community members are of course welcome to write up their own long-form designs, and if the workgroup likes the result, they might invite the author to turn it into a vision. The documents you mention were both solicited.

Those documents were actually written with this process in mind; they just haven’t been approved yet. I believe they both have boilerplate in the introduction calling themselves “prospective visions”.

Also, just to be clear, it’s the Language Workgroup soliciting and approving these specific visions, since they all relate to the parts of the project the LWG has evolution authority over. Other parts of the project subject to evolution (e.g. SwiftPM) would be managed by the Core Team (until they spin off a dedicated workgroup for that).

8 Likes

@John_McCall I also have a question about the process and a suggestion:

Is the plan to eventually revisit (via review) and move all (or some) of the existing manifestos from the Swift repo (see here) over to the Evolution repos' new visions folder?

Also, I've noticed that the prospective vision documents so far (Variadic Generics, Macros) are not "clearly identified as (prospective visions)" yet – at least the word "prospective" doesn't appear in them at all. This might be because they were written before this announcement was made.

Can we introduce a consistent and clear way of marking them as prospective visions somehow? I suggest they could be renamed to "[Prospective Vision] Variadic Generics" and "[Prospective Vision] Macros" respectively. When accepted, the [Prospective Vision] prefix could be simply removed. I'm suggesting this style over "[Prospective] Vision for Variadic Generics" (which would be closer to their current name) because I've noticed you decided against the "A Vision for" prefix entirely for the first accepted vision document. Maybe because it would be redundant to state that in a visions folder.

I don’t expect we will, no. I can see why it would be nice to have a history like that, but pragmatically it’d be a lot of work when those documents have already largely served their purpose.

2 Likes