Add to swift-docc-plugin a --json-prettyprint flag to enable the output json files to be pretty-printed and ordered. This would make the output from swift-docc-plugin deterministic: if the underlaying code being documented is the same, the produced documentation is the same.
This is already supported functionality through the docc executable by adding the environment variable DOCC_JSON_PRETTYPRINT, as described in the swift-docc contribution guide.
This addition would expose this slightly hidden functionality of docc, so that other users could find it and benefit from it.
Context
The json files output by docc are optimized for space and its keys are not ordered in any deterministic manner. This causes that every time the documentation is generated, the content of all json files changes drastically, even if the underlaying code used as input has no changes.
In cases when the swift-docc-plugin is used to generate documentation files for a static site, and those files are committed to a repository, every time swift-docc-plugin is run all the json files will change, even if the content is equivalent. This aspect makes it ill suited to commit the static generated files into a repository
If through scripts or other means the json files are pretty-printed, the output of docc is indeed deterministic. It would be useful for swift-docc-plugin to expose and support the generation of deterministic output out of the box.
How it could work
The docc executable already supports json pretty-printing through the environment variable DOCC_JSON_PRETTYPRINT.
The swift-docc-plugin can expose a new --json-prettyprint flag that internally populates DOCC_JSON_PRETTYPRINT in the Process that runs docc.
This would not change current behavior of swift-docc-plugin in any other manner, even in places where DOCC_JSON_PRETTYPRINT (or other settings done through environment properties) are already set, the Process running docc would still inherit its environment and work as before.
Alternatives
This results can already be achieved through either scripts to prettify the json files produced by docc, or by enablign DOCC_JSON_PRETTYPRINT before running the swift-docc-plugin commands.
Discussion
This is already filed as Issue 127 for the swift-docc-plugin package, with a pull request being posted shorty.
This would be my first attempt at contributing to swift-docc-plugin, feedback is welcome.
Did you consider always enabling the sortedKeys option? Deterministic and reproducible builds are always a plus for a compiler, and I don’t think there is a clear reason to randomize key order here. There is also no risk of breakage because the keys are already in an arbitrary order.
If that is implemented, I would be interested to hear your motivation for enabling pretty printing as well.
Can you clarify what sortedKeys option you mean? Is this an option in the docc executable or somewhere else?
As it is currently planned, this change would only use the DOCC_JSON_PRETTYPRINT enviroment property, which modifies the behavior of docc and enables both pretty-printing and sorted keys. This change does not intend to modify the docc executable at all, all the changes would be solely in the swift-docc-plugin project.
I guess I’m asking: why not go with an approach that does not necessarily require a proposal, and improves the experience for everyone without requiring them to set a compiler flag? What risk do you see from enabling sorting by default? What benefit do you see from enabling pretty printing?
I want to second the recommendation of always enabling sortedKeys in generated JSON output. Without that, the serialised order of JSON object keys is just random as it relies on the hashes and varies between runs. The sorting hardly slows down the process at all when objects tend to be small.
Another JSONEncoder option to always enable is withoutEscapingSlashes. There's really no reason to see slashes escaped like "foo\/bar" in JSON strings.
I think that would be a good change that should not impact the docc intent of having the json files be optimized for size. However, that would be a change to the docc executable, and thus outside of scope for what is proposed here.
Because for this specific change I prefer and I am comfortable only with updating the swift-docc-plugin. I am neither comfortable nor wanting to modify the swift-docc project since that I assume requires a higher process since it gets bundled with Xcode.
It would help this proposal if it explicitly stated what problem it is solving. Right now I don't know if the issue primarily is that:
The output JSON isn't stable
For example: a hypothetical structure may sometimes be written as {"one":1,"two":2} and sometimes as {"two":2,"one":1}.
Having stable output (while still compactly formatted) would mean that pages that don't change from build to build continue to have the same single line JSON contents.
The output JSON isn't pretty-formatted
For example, the hypothetical structure from above—assuming a stable order of keys—would be output as {"one":1,"two":2} as opposed to being output as
{
"one" : 1,
"two" : 2
}
Having pretty printed output means that pages with only small content differences only need to update the lines that have differences rather than the full file (because everything would be on a single line). The tradeoff is that the pretty-formatted files are larger than the compact files.
If the problem is primarily about stable output then I personally feel that it would be a better solution to that problem to always make the JSON output use sorted keys. That way, developers who only need a stable output don't need to also pretty-format the JSON to have stable output.
Coincidentally, @snprajwal and I were talking about stable output changes very recently. We The DocC team have explored sorting the JSON keys by default is in the past but AFAIK only as a draft/work-in-progress PR that were never merged. When @snprajwal and I talked about it again today we concluded that we'd be open to sorting the JSON keys by default. It will likely have some performance impact and we do have a policy to not regress in performance, but assuming that the impact is small we'd be interested in helping out with improving performance elsewhere to make up for that performance impact.
Git always stores the entire file whenever any part of it changes in a commit. So this is really only meaningful when manually looking at diffs, or when using some other system (which one?) that stores files line-by-line.
I wouldn’t expect most people to review the JSON differences so it sounds to me that perhaps sorted keys is more important than full pretty printing.
To just +1 on the reply from @j-f1 - diffs are where the value was highest for me. It's especially relevant in two cases:
double checking that a change I made in content propagated where I expected it to into the final output.
conversely verifying that the only thing that's updated is what I expected since the last generation process.
The pretty print side of this is pleasant, but not as critical for me - when I'm hacking through the DocC JSON archive files, I'm almost universally piping them through jq to hunt down the pattern of what I'm after, or if I'm just exploring, I'll cat ... | jq | less to get it easier to read in a terminal output.
I ran DocC's benchmark script with a small change to sort the JSON keys using a few different projects and couldn't measure any statistically significant performance impact. Because I'd already made the changes to do that benchmarking I opened this DocC PR to sort the keys by default in future DocC releases.