Build of SPM CLI tool without changes sometimes takes ~30 sec

Using Swift
1

We’ve got a swift package for a cli tool in our repo with our iOS app. It seems like sometimes when we build the tool, it will take ~30 seconds even if no files for the swift package changed. When this happens, there's no console output, so it appears to confirm that it isn't recompiling any of the dependencies or our code. Rebuilding again immediately after takes usually a fraction of a second which is what we would expect when there have been no changes to the package.

Running the build command with -v shows a bunch of commands (shown below) that seem to be rebuilding each of the dependancies' Package.swift file.

We've been trying to narrow down when it happens but so far it feels a little random. It frequently seems to happen when switching between git branches with no changes to the swift package, other say it happens if a certain amount of time has past between builds. Any suggestions for what the cause could be or further steps we could try to troubleshoot it?

➜ xcrun -sdk macosx swift build --package-path "./scripts/neo-tools" -c release -v
/usr/bin/xcrun --sdk macosx --show-sdk-platform-path
/usr/bin/xcrun --sdk macosx --find xctest
/Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -print-target-info
/Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -print-target-info
xcrun vtool -show-build /Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2/libPackageDescription.dylib
/Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -L /Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -lPackageDescription -Xlinker -rpath -Xlinker /Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -target x86_64-apple-macosx10.15 -swift-version 5 -I /Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -sdk /Applications/Xcode-12-4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -package-description-version 5.3.0 /Users/craig/code/work/neo-ios/scripts/neo-tools/Package.swift -o /var/folders/hf/wtv9mp5d2ysdhz_ndpx3jf7c0000gn/T/TemporaryDirectory.F3UF3n/neo-tools-manifest
sandbox-exec -p '(version 1)
(deny default)
(import "system.sb")
(allow file-read*)
(allow process*)
' /var/folders/hf/wtv9mp5d2ysdhz_ndpx3jf7c0000gn/T/TemporaryDirectory.F3UF3n/neo-tools-manifest -fileno 8
/Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -L /Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -lPackageDescription -Xlinker -rpath -Xlinker /Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -target x86_64-apple-macosx10.15 -swift-version 5 -I /Applications/Xcode-12-4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -sdk /Applications/Xcode-12-4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -package-description-version 5.2.0 /Users/craig/code/work/neo-ios/scripts/neo-tools/.build/checkouts/swift-argument-parser/Package.swift -o /var/folders/hf/wtv9mp5d2ysdhz_ndpx3jf7c0000gn/T/TemporaryDirectory.6VFJcz/swift-argument-parser-manifest
sandbox-exec -p '(version 1)
(deny default)
(import "system.sb")
(allow file-read*)
(allow process*)
(allow sysctl*)
(allow file-write*
    (regex #"^/private/var/tmp/org\.llvm\.clang.*")
    (regex #"^/var/folders/hf/wtv9mp5d2ysdhz_ndpx3jf7c0000gn/T/org\.llvm\.clang.*")
    (regex #"^/private/var/folders/hf/wtv9mp5d2ysdhz_ndpx3jf7c0000gn/T/org\.llvm\.clang.*")
    (regex #"^/private/var/folders/hf/wtv9mp5d2ysdhz_ndpx3jf7c0000gn/C/org\.llvm\.clang.*")
    (subpath "/Users/craig/code/work/neo-ios/scripts/neo-tools/.build")
)
' /var/folders/hf/wtv9mp5d2ysdhz_ndpx3jf7c0000gn/T/TemporaryDirectory.6VFJcz/swift-argument-parser-manifest -fileno 8

... truncated due to post size limit
2

My guess would be that this is reloading the package graph.

3

Do you happen to know why it sometimes reloads the package graph, but other times it doesn't need to? I'd guess there's a cached version somewhere that maybe expires after a certain amount of time, or is somehow invalidated when switching branches.

One workaround I through of would be to use a makefile so that it only rebuilds when Package.swift or Package.resolved change or the compiled binary is missing. I was hoping SPM would be able to tell that the package was unchanged since the last build and skip doing any work.

4

Time shouldn't matter, but switching branches might since it could be changing the modification time of the package manifests.

1 Like
5

Hmm... doesn't seem to be the modification date of any files. If I run the following commands, then diff the text files, they're the same.

find ./scripts/neo-tools -exec stat -f '%m %N' "{}" \; > ~/Desktop/branch1.txt
git co branch2
find ./scripts/neo-tools -exec stat -f '%m %N' "{}" \; > ~/Desktop/branch2.txt

I'm kinda grasping at straws here but our build command is using xcrun --sdk macosx swift build... any chance that could be the cause? That used to be needed but it it seems we don't need it anymore.

My other thought was to setup a makefile to run the build command that depends on the Package.swift and Package.resolved files so if those don't change and the binary exists, it wont rebuild. But I was hoping the swift build command would be doing that already.

1 Like
6

I couldn't figure out why swift was deciding rebuild all the Package.swift files for the dependencies. As far as I could tell, no files were having their modifiedAt time changed.

My workaround was to use a makefile. It's setup so it should only rebuild if Package.swift, Package.resolved, or any Sources/**/*.swift changes. Here's what it looks like if anyone comes across this in the future.

SWIFT_PACKAGE_DIR = ./scripts/cli-tools
SWIFT_PACKAGE_SOURCES = $(shell find $(SWIFT_PACKAGE_DIR)/Sources -type f -name "*.swift")
SWIFT_PACKAGE_BINARY = $(SWIFT_PACKAGE_DIR)/.build/release/cli-tools
cli-tools: $(SWIFT_PACKAGE_DIR)/Package.swift $(SWIFT_PACKAGE_DIR)/Package.resolved $(SWIFT_PACKAGE_SOURCES)
	swift build --package-path "$(SWIFT_PACKAGE_DIR)" -c release
	ln -Ffsv "$(SWIFT_PACKAGE_BINARY)" cli-tools
	touch "$(SWIFT_PACKAGE_BINARY)"

Though if there's a better solution, please let me know :smiley:.