DocC get's duplicated file checked out

Development Swift-DocC
1

We see a weird behaviour with the docc plugin and wonder if anyone else have run in to it or have any hints - quite often we'll get duplicate checkout of docc plugin source files (see below) which leads to a broken build as there are multiple source files with the same content (suffixed "2", "3", ...) removing the .build directory will give a perfectly clean compilation. In this case I had a clean build, ran tests with TSAN and then triggered the problem.

hassila@max ~/D/G/swift-storage (main) [1]> swift test
Compiling plugin Swift-DocC...
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/SwiftDocCConvert 3.swift:13:14: error: invalid redeclaration of 'SwiftDocCConvert'
@main struct SwiftDocCConvert: CommandPlugin {
             ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/SwiftDocCConvert.swift:13:14: note: 'SwiftDocCConvert' previously declared here
@main struct SwiftDocCConvert: CommandPlugin {
             ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/SwiftDocCConvert 3.swift:16:37: error: ambiguous use of 'doccExecutable'
        let doccExecutableURL = try context.doccExecutable
                                    ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SharedPackagePluginExtensions/PluginContext+doccExecutableURL 2.swift:19:9: note: found this candidate
    var doccExecutable: URL {
        ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SharedPackagePluginExtensions/PluginContext+doccExecutableURL 3.swift:19:9: note: found this candidate
    var doccExecutable: URL {
        ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SharedPackagePluginExtensions/PluginContext+doccExecutableURL.swift:19:9: note: found this candidate
    var doccExecutable: URL {
        ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/SwiftDocCConvert 3.swift:19:36: error: ambiguous use of 'extractSpecifiedTargets(in:)'
        let specifiedTargets = try argumentExtractor.extractSpecifiedTargets(in: context.package)
                                   ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SharedPackagePluginExtensions/ArgumentExtractor+extractSpecifiedTargets 3.swift:51:19: note: found this candidate
    mutating func extractSpecifiedTargets(in package: Package) throws -> [SwiftSourceModuleTarget] {
                  ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SharedPackagePluginExtensions/ArgumentExtractor+extractSpecifiedTargets.swift:51:19: note: found this candidate
    mutating func extractSpecifiedTargets(in package: Package) throws -> [SwiftSourceModuleTarget] {
                  ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/SwiftDocCConvert 3.swift:23:48: error: ambiguous use of 'allDocumentableTargets'
            swiftSourceModuleTargets = context.package.allDocumentableTargets
                                               ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SharedPackagePluginExtensions/PackageExtensions 3.swift:73:9: note: found this candidate
    var allDocumentableTargets: [SwiftSourceModuleTarget] {
        ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SharedPackagePluginExtensions/PackageExtensions.swift:73:9: note: found this candidate
    var allDocumentableTargets: [SwiftSourceModuleTarget] {
        ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/SwiftDocCConvert 3.swift:29:19: error: ambiguous use of 'packageDoesNotContainSwiftSourceModuleTargets'
            throw ArgumentParsingError.packageDoesNotContainSwiftSourceModuleTargets
                  ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SharedPackagePluginExtensions/ArgumentExtractor+extractSpecifiedTargets 3.swift:16:10: note: found this candidate
    case packageDoesNotContainSwiftSourceModuleTargets
         ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SharedPackagePluginExtensions/ArgumentExtractor+extractSpecifiedTargets.swift:16:10: note: found this candidate
    case packageDoesNotContainSwiftSourceModuleTargets
         ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/SwiftDocCConvert 3.swift:35:31: error: ambiguous use of 'init(_:)'
        let parsedArguments = ParsedArguments(argumentExtractor.remainingArguments)
                              ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SwiftDocCPluginUtilities/ParsedArguments 3.swift:154:12: note: found this candidate
    public init(_ arguments: [String]) {
           ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SwiftDocCPluginUtilities/ParsedArguments.swift:154:12: note: found this candidate
    public init(_ arguments: [String]) {
           ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SwiftDocCPluginUtilities/HelpInformation 3.swift:37:50: error: 'PluginAction' is ambiguous for type lookup in this context
    public static func forAction(_ pluginAction: PluginAction, doccExecutableURL: URL) throws -> String {
                                                 ^~~~~~~~~~~~
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SwiftDocCPluginUtilities/PluginAction 2.swift:10:13: note: found this candidate
public enum PluginAction: String {
            ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SwiftDocCPluginUtilities/PluginAction 3.swift:10:13: note: found this candidate
public enum PluginAction: String {
            ^
/Users/hassila/Documents/GitHub/swift-storage/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Convert/Symbolic Links/SwiftDocCPluginUtilities/PluginAction.swift:10:13: note: found this candidate
.......

Haven't used any other SPM plugins so can't say if it's an issue with SPM, with the DocC plugin or with our environment (we are simply using GitHub and checking out from there).

In this specific example the package manifest is fairly minimal:

// swift-tools-version: 5.6
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "swift-storage",
    platforms: [
        .macOS(.v12),
    ],
    products: [
        .library(
            name: "Storage",
            targets: ["Storage"]
        ),
    ],
    dependencies: [
        .package(url: "https://github.com/apple/swift-lmdb", branch: "main"),
        .package(url: "https://github.com/apple/swift-argument-parser", .upToNextMajor(from: "1.1.0")),
        .package(url: "https://github.com/apple/swift-log", .upToNextMajor(from: "1.0.0")),
        .package(url: "https://github.com/apple/swift-system", .upToNextMajor(from: "1.0.0")),
        .package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0"),
    ],
    targets: [
        .target(
            name: "Storage",
            dependencies: [
                .product(name: "CLMDB", package: "swift-lmdb"),
                .product(name: "Logging", package: "swift-log"),
                .product(name: "SystemPackage", package: "swift-system"),
            ]
        ),
        .executableTarget(
            name: "StorageExecutable",
            dependencies: [
                .product(name: "ArgumentParser", package: "swift-argument-parser"),
                .product(name: "Logging", package: "swift-log"),
                .product(name: "SystemPackage", package: "swift-system"),
                "Storage",
            ]
        ),
        .testTarget(
            name: "StorageTests",
            dependencies: ["Storage",
                           .product(name: "Logging", package: "swift-log")]
        ),
    ]
)

Any thoughts? Is it a bug or? (unfortunately haven't pinned down the steps to reproduce it, but happens once per day or so)

2

Hi @hassila! Thanks for bringing this up. This doesn't seem like something specific to the Swift-DocC Plugin at first look- would you mind filing a GitHub Issue on the Swift Package Manager repo so we can have the SwiftPM folks take a look?

If it turns out to be a DocC Plugin issue we can transfer the issue from there.

3

Thanks @ethankusters - I filed Sometimes SPM seems to duplicate checked out files · Issue #5589 · apple/swift-package-manager · GitHub for this (for future people running into this thread that wants to follow up).

1 Like