`--target` output clobbers `index.json`, breaks sidebar in 5.7

Over at SPI, we're hosting DocC docs for Swift packages if authors opt-in with a config file .spi.yml in their repository. Here's an example of such an opt-in:

version: 1
    - platform: macos
      scheme: ChimeKit
    - documentation_targets: [ChimeExtensionInterface, ChimeLSPAdapter]

Package authors configure a list of targets they'd like us to generate docs for. This has worked fine with all targets displaying docs when we generated the docs with Swift 5.6.

One particular issue we had to work around was that package plugin generate-documentation only has a single target flag --target. In order to generate docs for all targets, we loop over them. There's a slight hitch in that DocC clobbers the output directory when doing this. I.e. it doesn't just add file from subsequent targets but instead nukes the output directory completely on each run.

We worked around this by copying the output away between runs. I've always had the nagging feeling that this sequential processing will come to haunt us but for all the 5.6-generated docs we've seen this has been working fine.

This changed with 5.7 and its sidebar.

It seems that the file index/index.json contains information specific to a target's sidebar and all targets write to this same file. So what we're observing is that in terms of a working sidebar "last target wins".

Our hack has finally caught up with us, because of course it would :grimacing:

Best I can tell, there's no way to generate docs for a list of targets with a consolidated index.json, or is there?

It feels like our only options are to get a --targets flag into the DocC plugin or somehow merge index.json - if that's even possible, at least as workaround (that surely won't cause problems, nono) until there's a better solution.

Is our observation that index/index.json contention is the problem correct? Is there any way for us to recover multi-target sidebars with 5.7?

1 Like

I should add that the reason we're using --target in the first place (instead of just running generate-documentation without it) is to have some level of control over the amount of output being generated (it can be extensive) and to facilitate an opt-in mechanism at the same time.

I suppose we could make that a generate_documentation: true flag, but it feels like we'd be pulling in a lot of additional targets that authors don't currently select.

For additional clarification: Are you guys looking at generating a single sidebar (and associated pages) for multiple targets? If so that is currently not possible in DocC. However, we are actively looking at supporting this sort of workflow based on the proposals David put out recently (Use cases for combined documentation of multiple targets in Swift-DocC and Combined documentation of multiple targets). I think what you would be looking for is to use the proposed docc merge command once it becomes available.

1 Like

Yes, that's what I was hoping to achieve!

And I sort of have, I think, by merging the index.json files from the different processing stages (see over in the GH issue for the ugly details). Maybe that's a crutch for us to limp along until there's a proper docc merge we can use?

Thanks a lot for those links. I think I've missed those proposals completely and will have to catch up on them!

I think for simple use cases you could use this is as crutch for now, However docc merge is intended to be used a "final" build step for a group of related archives to generate a "combined" archive. Ideally this will allow people to specify how to do this in their manifests and you would then be able to build a single archive for a group of related targets. This will allow people to specify curation for their targets and have a customised landing page which would be nice enhancements.