[Pitch] Control severity of specific DocC warnings/diagnostics

Development Swift-DocC
1

DocC—like many other systems—makes the distinction between "warnings" and "errors" that warnings allow the build continue whereas errors fail the build without doing the remainder of the work and without outputting a "product". In practice, this has led to DocC avoiding error-severity diagnostics, even for issues where content is dropped from the output.

Different people have different opinions about which categories of issues should be considered errors and which shouldn't but the only control that they have for this today is the --warnings-as-errors flag. This raises the severity of all warnings to errors, so unless the developer has fixes all other warnings, they can't use this flag to fend against specific categories of issues.

With SE-0446 (Precise Control Flags over Compiler Warnings), Swift 6.1 gained two new -Werror <group> and -Wwarning <group> flags to control allow for precise control of the severity of individual groups of warnings/diagnostics. I propose that we do the same for DocC.

Proposed solution

Add two new --Werror and --Wwarning flags to DocC to control the severity of groups of diagnostics or individual diagnostics. For example, you can raise the severity of 2 diagnostics using:

--Werror UnresolvedLink --Werror MultipleTopicsSections

Similarly, you can raise all warnings to error-severity except a select few using:

--warnings-as-errors --Wwarning UnresolvedLink --Wwarning MultipleTopicsSections

Add a new --print-diagnostic-groups / --no-print-diagnostic-groups flag that controls whether or not DocC includes the diagnostic group identifier after the message in the console output. In the default diagnostic output format (aimed at humans) this behavior would be enabled by default a means for developers to discover which identifiers relate to which warnings. For example:

warning: 'someMethod()' is ambiguous at '/ModuleName/SomeStruct' [UnresolvedLink]
 --> Sources/ModuleName/SomeFile.swift:2:28-2:40
1 |
2 + /// An ambiguous link to ``someMethod()``
  |                                        ├─suggestion: Insert '->Int' for 'func someMethod() -> Int'
  |                                        ╰─suggestion: Insert '->String' for 'func someMethod() -> String'
3 | public struct SomeStruct {

In the diagnostic format intended for tools (configured via the --ide-console-output flag), diagnostic group identifier wouldn't be enabled by default so as to not impact any parsing logic of existing tools.

Note: the recommended ways for tools to consume diagnostic information is via the JSON file that DocC outputs to the specified --diagnostics-file <file-path> location.

Organize conceptually related diagnostics into broader groups that developers might want to control the severity of as a group. For example, DocC today uses different diagnostic IDs for issues about missing required directive parameters, extra unknown directive parameters, and incorrect values for directive parameters. It makes more sense to control the severity of these warnings as a single hypothetical "DirectiveParameterValidation" group (although that group ID spelling isn't finalized).

A draft PR that starts to implement this is available at swift-docc PR #1347.

Tools and scripts who make calls to DocC can check for the support for these flags by checking for the "precise-diagnostic-severity-control" feature in DocC's features.json file in the toolchain.

Future directions

In addition to these command line flags we could allow developers to configure specific diagnostic severities in their documentation catalog's (.docc directory) optional Info.plist file. For example, we could add two hypothetical <WarningsAsErrors> and <ErrorsAsWarnings> keys, each with their own list of string values representing individual diagnostic group identifiers.

Alternatives considered

The proposes spellings of the new flags are based on their Swift spellings with a two dash prefix instead of a single dash prefix. The idea is that developers would already be familiar with these spellings.

Alternatively, we could use a single dash prefix to be consistent with Swift (but inconsistent with DocC's other flags: -Werror and -Wwarning

Alternatively, we could use full spelled out singular-noun-phrases (warning instead of warnings) to control the severity of individual diagnostics: --warning-as-error to raise the severity and either --error-as-warning or --no-warning-as-error to lower the severity.

6 Likes
2

Future directions

In addition to these command line flags we could allow developers to configure specific diagnostic severities in their documentation catalog's (.docc directory) optional Info.plist file. For example, we could add two hypothetical <WarningsAsErrors> and <ErrorsAsWarnings> keys, each with their own list of string values representing individual diagnostic group identifiers.

+1 for this. Otherwise SPI user must use a very tediously way to specify this by custom_documentation_parameters.

version: 1
builder:
  configs:
    - documentation_targets:
        - xx
      custom_documentation_parameters:
        - --Werror UnresolvedLink --Werror MultipleTopicsSections
2 Likes
3

First off, +1 for the feature add.

While I've not yet needed this capability, I can see where it would be immensely useful. The existing GitHub workflows that are used across a variety of packages check for "soundness" of the docs using the --warnings-as-errors flag today. It would be nice to allow finer grained consideration there.

To make these diagnostic groups more useful, we should include them in the documentation for DocC, and publish them - akin to what's happening in the Swift compiler at Diagnostic Groups, with consistent URL schemes that map to the content, which can be used to provide "additional data" through development tools, if they choose to use the links.

(As an example, the Compiler diagnostic group swift-memory-safety resolves to https://docs.swift.org/compiler/documentation/diagnostics/strict-memory-safety, and provides more in depth detail than you'd expect to get some a less-than-one-line summary in tool/IDE diagnostic output.)

If we're going to the trouble of adding the diagnostic groups and providing control for them, we should include the hosting and publishing content side as part of this proposal as well.

4

Would you consider adding -Werror to match the clang spelling as well? (although, technically, you spell it as -Werror=UnresolvedLink rather than -Werror UnresolvedLink). But clang does use both forms (joined and separate) for different flags, so it isn't unreasonable to have the separate version be supported.

5

Yes, supporting both --Werror and -Werror seems like a good idea for this.

6

That sounds like a great idea. Most DocC diagnostics have an additional explanatory paragraph that’s displayed on the command line and is included in the diagnostics file but having a web page for each diagnostics group would allow us to write longer explanations with examples and further discussions on how to solve a given warning.

I would however say that I think the effort of writing that additional diagnostic documentation is best addressed as an ongoing effort over some time. However, once the first few DocC diagnostics pages are published, each new page would be a small task that could also be well suited for new contributors with some guidance from the reviewer.

1 Like