CMake guidance - tweaking the process to build the docs for the standard library

Development Compiler
1

I know a bit of CMake, but I'd really like some guidance or thoughts on a potential solution I'm trying to work out:

This is all about the Swift build, and generating the documentation for the standard library (which ultimately shows up at docs.swift.org). Today it's following a simple-ish pattern: when it builds the standard library normally, it tacks on a flag that emits the symbolgraphs for all the modules that it builds, and then in a sweep later, docc is provided a top level directory where all those symbol graphs were emitted, and is uses them all together.

That's working, but it's throwing in a whole bunch of symbols that shouldn't be exposed in DocC, highlighted with Standard Library documentation includes unavailable enumerations · Issue #175 · swiftlang/docs · GitHub.

In digging through the build scripts and trying to grok them, the symbol graphs for modules are getting triggered when a module is "tagged" with IS_STDLIB, sometimes that happens indirectly through the "tag" IS_SDK_OVERLAY (such as Observation).

I think the solution I'd like to aim for would make an exclusion list - either building a list of files or tags or something, potentially in CMake - that we'd want to exclude. There's already two hard-coded as exclusions in the CMake setup today - swiftDarwin, and swiftDifferentiationUnittest. This would extract those two hard coded exceptions and start building a bigger list (so we could exclude StdlibUnittest, SwiftPrivate*, RuntimeUnittest, SwiftReflectionTest, and _RegexBuilder, among the set) and only generate symbolgraphs when those values aren't hit.

What I'm not super clear on is it doing this in CMake is the right place, or if this should be driven from elsewhere - a different layer (such as the python that we have convenience methods set up to call through it). It kind of looks like we've really leaned into CMake for most of this, so I think that make sense, but I'd like some validation.

I also want to set it up so that only the things we want to explicit exclude are dropped, so that if/when we add other modules to the standard library in the future, the process automatically includes generating those symbol graphs - which ultimately get included and built with the DocC catalog content for the standard library.

I'm thinking:

  • declaring the list - perhaps in stdlib/cmake/modules/StdlibOptions.cmake that is the exclusions that we'll use
  • update stdlib/cmake/modules/AddSwiftStdlib.cmake around 1044 to use that list instead of the hardcoded exceptions
  • add to the list to exclude _RegexBuilder, SwiftPrivate*, StdlibUnittest, RuntimeUnittest, SwiftReflectionTest, and so forth - the various modules that we just don't need the symbols for in the standard library.

Is this a reasonable approach? Am I attempting to solve this problem at too low a level?

2

It may be worth looking at switching to the new runtime build system for this. First, you can avoid needing to build the compiler at all. Second, it has tighter tracking on what flags are passed to which targets. If you want to throw something on my calendar next week, I'd like to see a walk-through of the process you're currently using and I can help you get something set up.

Anything done in the old build system now will disappear once we disable the old build system.

1 Like
3

I would say that if you want to have a proper scalable solution, then this is the wrong place to start. Instead, my recommendation would be to drop this entirely and instead look at "the new build system" instead for generating the documentation.

From what I can tell, it seems that you would like to generate documentation for swiftCore and swift_Concurrency, both of which are in core. You would have far more direct control over what you are trying to generate, and I suspect a simple SwiftCore_GENERATE_DOCUMENTATION option would suffice to allow control over the emission of the documentation.

1 Like
4

thank you (& @compnerd) - I'll definitely take you up the help, and head to a solution that builds on the new, rather than hacks the old. I'm going on holiday next week, so I'll circle back to this after the 12th, and do some reading to get up to speed in the mean time.