Hi all, I've been playing around with the --experimental-documentation-coverage flag and it's super cool! I am, however, a bit confused by this and other "experimental" features which seems to skirt the evolution process. How do these experiments find their way into production releases and how can they be polished up and made "production ready"?
Assuming that there would need to be a pitch to make this a "production" feature, is there any indication that there will be a pitch for this feature in the near future? This is exactly the kind of thing that I'd love to spend spare time on (although admittedly I don't have a ton of that these days) but I don't want to invest any effort if it's just going to be superseded or rejected because some mysterious internal team has it on their backlog.
The gist of the flow described in that meeting - a simple pitch to make something non-experimental in the forums, providing the opportunity to refine and gather any feedback about it's operation. Presuming a consensus, then just the relevant PR's to convert the experimental features to default, and updates as needed to the documentation to match the feature's detail.
If it's more contentious, i'm not certain of the process, but based on past experiences, I'd guess that would keep it in experimental phase longer under a clear path forward could be built-up with consensus.
The overlap with what the internal teams may or may not be doing is unfortunate, but I don't expect Apple's team to expose their internal priorities and intended efforts on any sort of granular level. That said, a quick question in a DocC meeting of "Will my effort at .... XYZ ... run afoul of anything being worked on today?"
I found out, for example, that I'd replicated some of David's work with a 6 month delay when I was exploring extracting and rendering documentation from swift argument parser in order to come up with a scheme to document CLI options for swift commands (well, Docc and friends in particular, but it had broader applicability)
I can’t speak for any of the other features but code coverage has been in DocC since before it was open sourced. I was responsible for the original implementation—which seems mostly untouched. Which is just to say that it didn’t so much bypass evolution as it predates public evolution.
The gist of the flow described in that meeting - a simple pitch to make something non-experimental in the forums, providing the opportunity to refine and gather any feedback about it's operation. Presuming a consensus, then just the relevant PR's to convert the experimental features to default, and updates as needed to the documentation to match the feature's detail.
The overlap with what the internal teams may or may not be doing is unfortunate, but I don't expect Apple's team to expose their internal priorities and intended efforts on any sort of granular level.
These both make sense to me and are more or less the answer I expected. I think I'm just not going to worry about overlap with internal work, I'll keep this in the back of my head and if I get to it, I get to it .
code coverage has been in DocC since before it was open sourced.
Wow I didn't know it was that old, thanks for the insight!
DocC uses a less strict evolution process than Swift itself—both in terms of the pitch/proposal format itself and in terms of the review process—but new features are generally:
Talked about in the Swift Forum's before or early in the development of that feature
Introduced as "experimental" to indicate that there may still be breaking changes.
Gated behind an off-by-default feature flag, unless the feature requires an explicit action to use (for example new markup syntax).
I went through some past DocC pitches/proposals, and here's an non-exhaustive list (in no particular order) of example pitches/proposals from various people who both contribute to DocC frequently and people who are first time contributors:
Sometimes these threads have a lot of activity, sometimes they don't.
Sometimes someone—either a regular attendee or a guest—suggest these pitches as a topic for a Documentation Workgroup meeting, sometimes they don't.
Sometimes these pitches already have a draft implementation to begin with, sometimes they don't.
Sometimes there are separate threads to introduce the idea and to pitch the feature, sometimes both are done in the same thread.
For DocC, there's not a formal review process where a proposal is accepted or not. Instead, the pitches are primarily a means to seek feedback on the feature and its design and to iterate and refine based on that feedback. To date, features typically aren't rejected unless there's significant outstanding feedback. In my view the lack of explicit approvals hasn't caused any issues for DocC so far, and I enjoy the low-overhead of it, but I can also see potential flaws with it. If someone feels that the evolution of DocC would benefit from more formal process, it would be a good topic for a future Documentation Workgroup meeting. These meetings are open to anyone who is interested are usually held every two weeks; announced via a forum post in the Swift Documentation / Workgroup Updates category. For more information, contact @swift-documentation-workgroup.
Even with an iterated-upon and refined design, someone has to implement it. People are often very busy with their own projects and other responsibilities but many people are also happy to help out, answer questions, or mentor anyone who wants to contribute.
Depending on the amount of initial feedback and design refinement, features are generally introduced as "experimental" to indicate that there may still be breaking changes.
Merging experimental—or otherwise in-progress but usable—features can be a valuable tool to gather feedback from a broader group of people or to gather feedback based on actual usage. The number of people who are interested and have spare time to build a branch of Swift-DocC from source to try out a feature and run it with their real world project's data is very small.
Sometimes this additional feedback from people who use a feature in its experimental state help help refine the design, sometimes it helps verify the design, and sometimes it does neither and potential questions about the feature's design remain unanswered.
As far as I personally know, no one is already working on documentation coverage or anything related that would supersede it or otherwise make an effort into improving documentation coverage go to waste.
If you want to work on the pitch (and/or implementation) to make documentation coverage a non-experimental feature I'd be happy to help with that. We could either use this thread for that discussion or start a new thread about it.
I will say that there are a few aspects of documentation coverage that I personally would like to see revisited—while breaking changes are still accepted—before removing it's "experimental status":
The output format. Specifically, I'd love to have a discussion about:
what information would be useful to include so that other tools can read and present the coverage information
how the format could be structured so that new coverage metrics could be added and displayed without requiring further changes by those tools.
The command line options. Specifically; I think it would be a good opportunity to talk about:
Is --coverage-symbol-kind-filter <symbol-kind> option (to only emitting coverage data about a certain kind of symbol (for example enumerations but not classes) useful?
Are there other flags or options that would be useful to have? For example; would it be useful to only emitting coverage data for a certain sub-hierarchy or to only emit certain coverage metrics?
Wow, thanks for taking the time to write this lengthy response up, the insight is greatly appreciated!! Open Source governance/management seems to be quite the tricky task and I hope my confusion wasn't taken as criticism or complaint. I quite like the pitch process because it makes it easy for someone who wasn't around during the pitch to come back later and read about the implementation and decisions. Whether the pitch process is needed for DocC is a question I'll leave to those with more skin in the game.
I don't expect I'll be dedicating much, if any, time on this in the near future but I'll push any hacking I do to a fork and post here if it's anything significant.
.