Plugin command not running when package is indirectly included in Xcode project

I've been experimenting with the new plugin feature introduced in Swift 5.6 and have run into a strange issue.

My project uses a single Package.swift which contains ~20 local package targets. I have a target that is using an experimental SwiftGenPlugin which includes a pre-build command. I am using the plugin along with a swiftgen.yml file to generate a file to help with localization.

The following scenarios work fine:

  • Build that package directly
  • Build a package that depends on that package

I can monitor that location in derived-data and see that it is built almost immediately.

However, when I include one of the top level packages that depends on the package using SwiftGen inside an iOS project as a normal dependency, I will get build failures from other packages that depend on SwiftGen already having run. It's like it stops building the graph correctly once it's included in an actual Xcode project.

Is this expected? Does anyone have a workaround?

3 Likes

Here is the relevant portion of the build log for the iOS Xcode project target:
Screen Shot 2022-05-05 at 3.44.39 PM

And here is the relevant portion of the build log for the package that the iOS Xcode project target depends on (which includes the BTLocalization package as a dependency):
Screen Shot 2022-05-05 at 4.10.54 PM

Notice how the failed build never mentions SwiftGen while the successful case has it as the first step. The failed build also makes no reference to "prebuild commands".

Are these plugins intended to run when triggered from building an actual project?

Here is a project that reproduces the issue.

Notice that it fails if you try to build the "iOS App" target alone. Once you build either the "iOSAppFeature" or "Localization" package, future attempts to build the "iOS App" target will succeed.

I have been unable to make any progress on this issue. Can anyone tell me if this is a bug or functioning as designed?

I have tested this with Xcode 13.4 and have confirmed that the issue is still present.

I created the experimental plugin @rzulkoski mentioned in the original post.

I created this plugin in preparation for integration with our app's SPM micro-feature architecture where we are currently running SwiftGen from commit hooks.

I can also replicate the issue described using Ryan's simplified example, and I'll admit I hadn't gotten this far with integration on the plugin that I built - for reasons I'll explain.

I forked the above reproducing example to play with this a bit myself, and in doing so I stripped down the example as much as possible. To make sure the issue is not in the SwiftGen executable or anywhere else I just created a simple executable binary with the sole purpose of creating a file with the current reference time interval as it's filename in Derived Data.

It is incredibly difficult to debug anything for these plugins.

Sometimes swift build will give an error when Xcode does not
Xcode will say

:white_check_mark: Applying plugin "TouchPlugin" to target "iOSAppFeature"

But if you run swift build you will receive

error: failed: PrebuildCommand(configuration: SPMBuildCore.BuildToolPluginInvocationResult.CommandConfiguration(displayName: Optional("Create a file"), executable: AbsolutePath:"/Users/nicolas.richard/Projects/PluginNotRunning/SwiftPackages/.build/arm64-apple-macosx/debug/Touch", arguments: ["674575448.053521"], environment: [:], workingDirectory: nil), outputFilesDirectory: AbsolutePath:"/Users/nicolas.richard/Projects/PluginNotRunning/SwiftPackages/.build/plugins/outputs/swiftpackages/iOSAppFeature/TouchPlugin")

No output from the executable is included anywhere in Xcode
It would obviously be extremely useful to have the executable's output for debugging

You are allowed to violate the rules of the package plugins and there are no errors at all
SE-0305 proposes that plugins cannot depend on an executable target inside the package for technical reasons. But there are no errors or warnings if you do so.

Just try pointing your plugin to an executableTarget instead of a binaryTarget. There are no errors or warnings. But the command does not run.

You can do this in my example fork by just switching out the comments from binaryTarget to executableTarget.

There is some kind of synchronization issue with FileManager for plugins
Related to why I haven't continued integration of the plugin into our apps.

I've tried to do some things like clear out the plugin folder before writing new files to it and the behaviour seems to be that the deletion can occur at any time before/after the write.

This causes issues if a filename changes, or if a generated file is meant to be deleted. Extra garbage is left in DerivedData and developers have an inconsistent experience.

I know that one potential solution is to work on a build command which explicitly states its inputs and outputs but I haven't had the time to try this yet.


I hope my perspective helps in looking at this issue / improving the functionality.

A big help to me and my pursuit would be information for how to obtain the output from the command my plugin is running and/or any additional debug information for the build tool plugin.

3 Likes

Good news, this has been fixed in Xcode 14 beta 4! :tada:

1 Like

Apologies in advance for the long post (and reviving this thread)...

@rzulkoski @nicorichard I'm trying to do a very similar (if not the same) thing, but am running into the same issues as above. The difference in my use case is that I'm trying to use SwiftGenPlugin in a pre-commit hook for a server-side Vapor application. You can see my Package.swift here:

// swift-tools-version:5.6
import PackageDescription

let package = Package(
  name: "wishlist-server",
  platforms: [
    .macOS(.v12)
  ],
  dependencies: [
    // đź’§ A server-side Swift web framework.
    .package(url: "https://github.com/SwiftGen/SwiftGenPlugin", from: "6.6.2"),
    .package(url: "https://github.com/vapor/vapor.git", from: "4.0.0"),
  ],
  targets: [
    .target(
      name: "App",
      dependencies: [
        .product(name: "Vapor", package: "vapor")
      ],
      resources: [.process("Resources")],
      swiftSettings: [
        .unsafeFlags(["-cross-module-optimization"], .when(configuration: .release))
      ],
      plugins: [
        .plugin(name: "SwiftGenPlugin", package: "SwiftGenPlugin")
      ]
    ),
    .executableTarget(name: "Run", dependencies: [.target(name: "App")]),
    .testTarget(
      name: "AppTests",
      dependencies: [
        .target(name: "App"),
        .product(name: "XCTVapor", package: "vapor"),
      ]
    )
  ]
)

And the relevant portion of my pre-commit file is here (SwiftGen should be run during swift test, as that also builds):

#!/bin/sh

XCPRETTY=$(which xcpretty)
if [[ -e "${XCPRETTY}" ]]; then
    echo "[SWIFT TEST] Testing..."
    swift test 2>&1 | $XCPRETTY
else
    echo "[SWIFT TEST] xcpretty does not exist, download from https://github.com/xcpretty/xcpretty"
    swift test 2>&1
fi

Building in Xcode works and properly generates my Strings.generated.swift file. This was the expected behavior and I'm glad this is working.

However, running swift build or swift test in the CLI (ex: pre-commit hook, or manually via Terminal) does not work as expected.

Here is the error I'm getting in my pre-commit hook (formatted for readability):

[SWIFT TEST] Testing...

❌  error: failed: PrebuildCommand(
	configuration: SPMBuildCore.BuildToolPluginInvocationResult.CommandConfiguration(
		displayName: Optional("SwiftGen BuildTool Plugin"),
		executable: <AbsolutePath:"path/to/project/.build/artifacts/swiftgenplugin/swiftgen.artifactbundle/swiftgen/bin/swiftgen">,
		arguments: [
			"config",
			"run",
			"--verbose",
			"--config",
			"/path/to/project/swiftgen.yml"
		],
		environment: [
			"TARGET_NAME": "App",
			"DERIVED_SOURCES_DIR": "/path/to/project/.build/plugins/outputs/wishlist-server/App/SwiftGenPlugin",
			"PRODUCT_MODULE_NAME": "App",
			"PROJECT_DIR": "/path/to/project"
		],
		workingDirectory: nil
	),
	outputFilesDirectory: <AbsolutePath:"/path/to/project/.build/plugins/outputs/wishlist-server/App/SwiftGenPlugin">
)

As you can see (and have experienced yourselves), It's not particularly informative. But, if I now run swift build manually via CLI, I get the same error as above, but fortunately with a little more info:

Executing configuration file /path/to/project/swiftgen.yml
 $ swiftgen strings --templatePath ./swiftgen-structured-swift5.stencil --param publicAccess --output ./Sources/App/Enums/Strings.generated.swift ./Resources/Localizable.strings ./Resources/Localizable.stringsdict
Error: You don’t have permission to save the file “Strings.generated.swift” in the folder “Enums”.

I was also able to find this thread that may be related (also with no resolution).

Would love any assistance getting this working!

@lewisgodowski
Looks like you may be trying to generate files outside of sandbox plugin directory.
That was bug in Xcode 14 (allowing to write outside plugin sandbox) which is now fixed in Xcode 15 (you will get same error in Xcode now). Most probably CLI handles that in different way and has no such bug.
You can have a look here for more details about similar SwiftGenPlugin issue and how it was solved here: Error using SwiftGen as SPM plugin in Xcode 15 · Issue #1075 · SwiftGen/SwiftGen · GitHub

And back to main topic:
I just experienced similar issue as original author but for remote plugin which depends on SwiftGenPlugin.
It looks like Xcode 14 does not run preBuildCommnad Swift Plugins for remote Swift Packages. It does however for local ones (so If I pull my remote package to project by dragging it and then re-adding to target libraries it works).
Generally speaking it looks like Xcode is not seeing such plugin at all.
This issue is fixed on Xcode 15 but before we migrate it may be handy to solve that for Xcode 14(.3.1).
Anyone else experienced that and maybe found solution?

I am running into a similar situation. I put together a small reproducer project here (see the README for instructions):

This builds fine when I run it via XCode. However, when trying to run via xcrun xcodebuild... , I consistently get the following:

  • xcrun version: 67
  • Xcode 15.0.1, Build version 15A507
  • xcode-select version 2397.
sandbox-exec: execvp() of '//Users/corymosiman/Library/Developer/Xcode/DerivedData/ABigFoo-gpktfsxtunvztkbtgwzooryjhwss/Build/Products/Debug/FooGeneratorExecutable' failed: No such file or directory
Command PhaseScriptExecution failed with a nonzero exit code
note: Using global toolchain override 'Xcode Default'.
note: Using global toolchain override 'Xcode Default'. (in target 'FooPackageTests' from project 'FooPackage')
note: Using global toolchain override 'Xcode Default'. (in target 'FooGeneratorExecutable' from project 'FooPackage')
note: Using global toolchain override 'Xcode Default'. (in target 'FooGeneratorExecutable' from project 'FooPackage')
note: Using global toolchain override 'Xcode Default'. (in target 'FooPackageApp' from project 'FooPackage')
note: Run script build phase 'Foo the Plugin' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'FooPackageApp' from project 'FooPackage')
note: Using global toolchain override 'Xcode Default'. (in target 'FooPlugin' from project 'FooPackage')
Testing failed:
        Command PhaseScriptExecution failed with a nonzero exit code
        Testing cancelled because the build failed.

Navigating to DD path, there is only a Debug-iphonesimulator directory:

image

When run via XCode (which succeeds), the Debug directory exists alongside the other:

image

A potential workaround is that if I can successfully run the build plugin separately (to produce the Debug directory), then run the scheme I actually want, it may work.

However, in the real project I'm working on, the build plugin doesn't exist in the local package dependency, it is an external package that the local package dependency relies on, so the proposed workaround wouldn't work for what I actually need.

In general, I guess I would expect xcodebuild to correctly run all necessary build plugins.

Looking for any guidance / help folks can offer.

Following up here. We attempted:

  1. add the test targets to the XCode Project scheme and then update the build command to reference that scheme instead of one of the SPM test schemes. No success.
  2. Installed XCode15.1-beta-3 and attempt to run with those Command Line Tools. No success.

[ UPDATE]

I have now narrowed it down to the use of the -sdk argument that is getting passed to xcodebuild.