Supporting hosting DocC archives in static environments (like GitHub Pages) by default

Development Swift-DocC
1

Hi all!

With Swift 5.6, we shipped support for building DocC archives that are compatible with static hosting environments on an opt-in basis. The response to this feature has been really positive, to the point that I think we should move forward with enabling this feature by default.

I've opened a PR here:

https://github.com/apple/swift-docc/pull/121

with the proposed change that would make the --transform-for-static-hosting flag the default and introduce a new --no-transform-for-static-hosting flag to allow for explicit opting out.

This change would mean that, by default, a DocC archive can essentially be copied into the root of most server hosts and "just work".

The drawback to this change is that, for folks not intending on hosting their documentation in a static hosting environment, docc will be adding unnecessary additional files to their DocC archive that don't provide any value. The original pitch of the feature is a good place to reference for the implications of this.

In general, I expect a ~24% increase in average DocC archive size by enabling this feature by default.

From my perspective, the benefits of a significantly improved out-of-the-box user-experience is worth the increase in DocC archive size. And, of course, the --no-transform-for-static-hosting flag will be available for scenarios where those files aren't needed and archive size is a greater concern.

I think it makes sense for DocC's defaults to cater to the more common use-case here instead of optimizing for smallest size-on-disk while creating an on-average worse user-experience.

The only other concern I had here was a significant performance hit, but, in practice, this doesn't seem to be an issue:

Detailed performance test results:

TestFramework 5

Metric Change Before After
Compiled output size (MB) +23.93% 61.05 75.66
Duration for 'bundle-registration' (sec) +0.72% 2.91 2.93
Duration for 'convert-action' (sec) +2.60% 4.19 4.30
Duration for 'navigation-index' (sec) no change 0.01 0.01
Peak memory footprint (MB) +4.86% 199.95 209.67
Topic Anchor Checksum no change 65d4ff3050cd1106a21b 65d4ff3050cd1106a21b
Topic Graph Checksum no change 0a64c740a2ed5a246836 0a64c740a2ed5a246836

TestFramework 10

Metric Change Before After
Compiled output size (MB) +24.09% 121.23 150.43
Duration for 'bundle-registration' (sec) -1.59% 5.87 5.77
Duration for 'convert-action' (sec) -1.28% 8.41 8.31
Duration for 'navigation-index' (sec) no change 0.03 0.03
Peak memory footprint (MB) +2.63% 359.21 368.66
Topic Anchor Checksum no change c8f03d4e81da8e3b6d7f c8f03d4e81da8e3b6d7f
Topic Graph Checksum no change edce22bce4f8de475a5e edce22bce4f8de475a5e

TestFramework 25

Metric Change Before After
Compiled output size (MB) +24.05% 303.45 376.43
Duration for 'bundle-registration' (sec) -1.13% 14.93 14.76
Duration for 'convert-action' (sec) +1.45% 21.48 21.79
Duration for 'navigation-index' (sec) no change 0.07 0.07
Peak memory footprint (MB) -1.78% 852.77 837.56
Topic Anchor Checksum no change 9a675b9ad6d69f8b7f0c 9a675b9ad6d69f8b7f0c
Topic Graph Checksum no change 665713509084b5131a37 665713509084b5131a37

TestFramework 50

Metric Change Before After
Compiled output size (MB) +23.95% 609.26 755.20
Duration for 'bundle-registration' (sec) -1.43% 30.26 29.82
Duration for 'convert-action' (sec) -1.40% 44.31 43.69
Duration for 'navigation-index' (sec) no change 0.14 0.14
Peak memory footprint (MB) -3.06% 1592.37 1543.62
Topic Anchor Checksum no change 08d0a84bec913460905f 08d0a84bec913460905f
Topic Graph Checksum no change 74d25c4f1ee185ad5676 74d25c4f1ee185ad5676

Looking forward to your feedback here.

- Ethan

16 Likes
2

I tend to agree -- the "by default easy and good user experience" is a valuable goal.

In the same spirit, we just accepted binaries on linux be linked statically; where similar tradeoffs were considered that the binary would be bigger, but that doesn't matter for the vast majority of cases - same here, those who care can flip the flag to not make the static hosting anymore :)

5 Likes
3

+1

I really think there is a significant value in a good out-of-the-box user experience where things just work as desired and having flags to tweak/optimize behaviour for the ones that needs it.

For all of the "lets test this and see how/if it works for us" projects, its a huge win - for the larger ones that actually runs into something like this as a real problem, they'll start looking for ways to optimise it (and then be happy to find the relevant option).

3 Likes
4

Big +1 here and thank you for including detailed performance test results. For large catalogs, the time difference does seem pretty negligible. The 24% increase in archive size is indeed quite large, but I do think the 'transform for static hosting' behavior is what most users want (especially based on the feedback the community has provided around this feature these past months). Adding the --no-transform-for-static-hosting flag for users who don't want this behavior sounds like a good way to continue supporting both workflows.

3 Likes
5

Agreed as well! It is unfortunate but if that is what we need to do to support good defaults we should do it! We could look at other ways to reduce the archive size if it becomes a problem.

1 Like
6

+1 for this.
Also one question here. Should we also soft-deprecate docc process-archive transform-for-static-hosting since transform-for-static-hosting is the default behavior? Or add a new subcommand no-transform-for-static-hosting in process-archive to remove the docs folder?

1 Like
7

Good questions! Personally I think

docc process-archive transform-for-static-hosting

is still useful both for updating archives that were produced with an older version of the compiler and for transforming new archives that were built with --no-transform-for-static-hosting.

docc process-archive no-transform-for-static-hosting

or

docc process-archive transform-for-non-static-hosting

is an interesting idea. I think in general, I'd prefer process-archive actions to be non-destructive and only add items to an archive but if there ends up being a real need for that feature I think it's definitely something we could consider.

8

Thank you all for the great feedback here!

The proposed change has been merged to main with:

- Ethan

4 Likes