# Incremental builds with Swift Package Manager and Linux / Docker **URL:** \ **Category:** Using Swift\ **Tags:** spm\ **Created:** [July 2, 2019, 4:05am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538 "2019-07-02T04:05:20Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Bridger\_Maxwell](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/bridger_maxwell/32/6895_2.png) [@Bridger\_Maxwell](https://forums.swift.org/u/Bridger_Maxwell)\ **Post date:** [July 2, 2019, 4:05am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/1 "2019-07-02T04:05:20Z")
I'm using Swift Package Manager to build my Swift server app in a Linux Docker container. Each time I rebuild it fetches all of the dependencies and rebuilds them all. Recently I started using Swift-NIO as a dependency and it takes a while to compile. Are incremental builds supported on Linux? I tried to get incremental builds working by having my Package.swift + some dummy files copied into docker -\> build -\> copy in the real source files -\> build. I was hoping the dependencies of the first build would be saved for the second build. Unfortunately, even if I just do two identical build commands one after another it still rebuilds all dependencies from scratch. Are incremental builds on Linux supported? This is the docker file I'm using: ``` FROM bridger/swift-ubuntu:latest MAINTAINER Bridger LABEL Description="Docker image for building and running the Scribble server." EXPOSE 9080 RUN mkdir /root/Scribble-Server ADD Tests /root/Scribble-Server/Tests ADD Package.swift /root/Scribble-Server ADD Package.resolved /root/Scribble-Server # Add a dummy file for resolving dependencies and pre-building first ADD Sources/DummyForDependencies/main.swift /root/Scribble-Server/Sources/ScribbleServer/main.swift ADD Sources/DummyForDependencies/DummyModel.swift /root/Scribble-Server/Sources/CanvasModel/DummyModel.swift # Resolve the dependencies by building using the dummy sources RUN cd /root/Scribble-Server && export KITURA_NIO=1 && swift build -c release # Delete the dummy sources RUN rm /root/Scribble-Server/Sources/ScribbleServer/main.swift RUN rm /root/Scribble-Server/Sources/CanvasModel/DummyModel.swift # Add in the real sources ADD Sources /root/Scribble-Server/Sources RUN rm -rf /root/Scribble-Server/Sources/DummyForDependencies # Build again. This unfortunately rebuilds all dependencies, even though they haven't changed RUN cd /root/Scribble-Server && export KITURA_NIO=1 && swift build -c release ``` ---
**Author:** ![stephand](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/stephand/32/2506_2.png) [@stephand](https://forums.swift.org/u/stephand)\ **Post date:** [July 2, 2019, 9:03am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/2 "2019-07-02T09:03:30Z")
Haven't played with that in a while, but previously I did something like this, and that seemed to work: ``` # Install swift deps to use cache COPY ./Package.swift ./ COPY ./Package.resolved ./ RUN swift package resolve ``` Then copy the sources over and build the final product. ---
**Author:** ![Bridger\_Maxwell](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/bridger_maxwell/32/6895_2.png) [@Bridger\_Maxwell](https://forums.swift.org/u/Bridger_Maxwell)\ **Post date:** [July 3, 2019, 1:41am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/3 "2019-07-03T01:41:26Z")
Running `swift package resolve` as a separate step does cache the package resolution (so future build steps don't re-download the packages from git). Unfortunately, the packages are all compiled from scratch each time. The build stage is still not incremental. ---
**Author:** ![MrMage](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/mrmage/32/3829_2.png) [@MrMage](https://forums.swift.org/u/MrMage)\ **Post date:** [October 31, 2019, 6:16pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/4 "2019-10-31T18:16:29Z")
@Bridger_Maxwell I have run into exactly the same problem — the results of `swift package resolve` can be cached, but the dependencies will be rebuilt every time. My suspicion is that SwiftPM for some reason considers build artifacts in lower layers stale and thinks that it needs to rebuild them. Did you ever find a solution to this? ---
**Author:** ![SDGGiesbrecht](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sdggiesbrecht/32/4768_2.png) [@SDGGiesbrecht](https://forums.swift.org/u/SDGGiesbrecht)\ **Post date:** [October 31, 2019, 7:29pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/5 "2019-10-31T19:29:56Z")
Incremental builds are supported on Linux, just like macOS. Run `swift build` twice and the second time it will be virtually a no‐op. However, all continuous integration environments I have ever used consistently cause the cache to be rejected as stale, and this happens on all platforms. I believe something like the environment or the file modification dates come into consideration when deciding whether the cache is still valid, such that the strategy of copying the build cache onto a new virtual machine runs afoul of it. ---
**Author:** ![Aciid](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/aciid/32/429_2.png) [@Aciid](https://forums.swift.org/u/Aciid)\ **Post date:** [October 31, 2019, 8:40pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/6 "2019-10-31T20:40:10Z")
The build system uses [stat](https://github.com/apple/swift-llbuild/blob/master/lib/BuildSystem/BuildSystem.cpp#L570) to determine a file modified since it last saw it. Relaunching the docker container most likely invalidates the stat information that was previously obtained. ---
**Author:** ![Aciid](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/aciid/32/429_2.png) [@Aciid](https://forums.swift.org/u/Aciid)\ **Post date:** [October 31, 2019, 8:58pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/7 "2019-10-31T20:58:02Z")
Hm, it looks like the stat information of cached files don't change in a docker container. Do you mind filing a JIRA for this? It might be possible to make this work. ---
**Author:** ![MrMage](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/mrmage/32/3829_2.png) [@MrMage](https://forums.swift.org/u/MrMage)\ **Post date:** [November 11, 2019, 1:28pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/8 "2019-11-11T13:28:54Z")
Thank you for looking into this! I have filed [[SR-11760] SwiftPM builds inside Docker do not re-use existing build artifacts · Issue #4651 · apple/swift-package-manager · GitHub](https://bugs.swift.org/browse/SR-11760). ---
**Author:** ![nugmanoff](https://avatars.discourse-cdn.com/v4/letter/n/e480ec/32.png) [@nugmanoff](https://forums.swift.org/u/nugmanoff)\ **Post date:** [April 18, 2020, 3:58pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/9 "2020-04-18T15:58:14Z")
Hey there! Is there any updates on this one? ---
**Author:** ![rnantes](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/rnantes/32/5653_2.png) [@rnantes](https://forums.swift.org/u/rnantes)\ **Post date:** [December 9, 2020, 2:22am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/10 "2020-12-09T02:22:06Z")
I'm commenting for visibility. This is still an issue and as a result is making Vapor builds very slow as it needs to recompile all of swift-nio each time. [Incremental builds inside Docker · Issue #2325 · vapor/vapor · GitHub](https://github.com/vapor/vapor/issues/2325) ---
**Author:** ![rnantes](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/rnantes/32/5653_2.png) [@rnantes](https://forums.swift.org/u/rnantes)\ **Post date:** [December 15, 2020, 2:46am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/11 "2020-12-15T02:46:49Z")
@codafi I was wondering if the recent [Cross-Module Incremental Builds PR](https://github.com/apple/swift/pull/35036) would have any effect on this issue. ---
**Author:** ![codafi](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/codafi/32/791_2.png) [@codafi](https://forums.swift.org/u/codafi)\ **Post date:** [December 15, 2020, 3:52am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/12 "2020-12-15T03:52:15Z")
All incremental builds will improve wrt 5.3(.1) as long as you ensure that as many targets as possible pass `-enable-experimental-cross-module-incremental-build`. ---
**Author:** ![mufumade](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/mufumade/32/18299_2.png) [@mufumade](https://forums.swift.org/u/mufumade)\ **Post date:** [June 21, 2021, 2:19pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/13 "2021-06-21T14:19:55Z")
Any updates on this? I ran into the same issue when using aws codebuild. ---
**Author:** ![codafi](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/codafi/32/791_2.png) [@codafi](https://forums.swift.org/u/codafi)\ **Post date:** [June 21, 2021, 7:36pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/14 "2021-06-21T19:36:45Z")
Cross-module incremental builds shipped with Xcode 13 and Swift 5.5. [The latest development toolchains](https://swift.org/download/#snapshots) will have these features available for users on Linux and Windows. ---
**Author:** ![amraboelela](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/amraboelela/32/12898_2.png) [@amraboelela](https://forums.swift.org/u/amraboelela)\ **Post date:** [January 15, 2022, 9:41pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/16 "2022-01-15T21:41:55Z")
For me I found it was rebuilding everything when I do `swift build` then `swift run`, but if I do `swift run` only, then it would only build the changed files in my project and that's it. So maybe there is a bug in `swift build` ? I am using SPM 5.5 ---
**Author:** ![doozMen](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/doozmen/32/7803_2.png) [@doozMen](https://forums.swift.org/u/doozMen)\ **Post date:** [August 29, 2023, 8:32am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/17 "2023-08-29T08:32:41Z")
Did you ever found a solution for this? ---
**Author:** ![SDGGiesbrecht](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sdggiesbrecht/32/4768_2.png) [@SDGGiesbrecht](https://forums.swift.org/u/SDGGiesbrecht)\ **Post date:** [August 29, 2023, 9:41pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/18 "2023-08-29T21:41:21Z")
No, but I never tried to fix it; I only noticed it was happening. ---
**Author:** ![featherless](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/featherless/32/36808_2.png) [@featherless](https://forums.swift.org/u/featherless)\ **Post date:** [September 12, 2023, 3:55am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/19 "2023-09-12T03:55:33Z")
I just ran into this as well. 10+ minutes to build a Vapor server in a Dockerfile which makes iteration super slow for building an APNS server ☹ fwiw I tried following [Mounts | Docker Docs](https://docs.docker.com/build/guide/mounts/) and it didn't seem to make any difference either, I think because I'm using the kaniko builder which doens't support BuildKit. I also tried setting `--build-path /build/.artifacts` on each swift build invocation, but as mentioned above this doesn't seem to address the underlying problem of timestamps changing across steps. May need to resign to just waiting 10-15 minutes between deploys for now ☹ ---
**Author:** ![sliemeobn](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sliemeobn/32/27698_2.png) [@sliemeobn](https://forums.swift.org/u/sliemeobn)\ **Post date:** [September 12, 2023, 5:55am UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/20 "2023-09-12T05:55:43Z")
FWIW, I use `--mount=type=cache` in my docker builds and had zero issues with it so far. I was worried that I would need to clear the caches all the time because `swift build` would get confused and require a clean build - but so far it has been quite the soldier. after ~300 changes/builds I had to clear the caches exactly once (after switching git branches) here is an excerpt of what I use: ```swift ARG SWIFT_VERSION=5.8 FROM swift:$SWIFT_VERSION AS builder WORKDIR /build # Resolve all dependencies if there was a change COPY ./Package.* ./ RUN --mount=type=cache,target=/build/.build swift package resolve # Copy all sources and build all COPY . . # build with caching and copy app out of cached folder RUN --mount=type=cache,target=/build/.build \ swift build -c release \ && cp -a .build/release/ /release FROM swift:$SWIFT_VERSION-slim COPY --from=builder /release/${SERVICE_NAME} ./app ENTRYPOINT ["./app"] ``` ---
**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\ **Post date:** [February 18, 2024, 10:20pm UTC](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538/21 "2024-02-18T22:20:47Z")
> [@sliemeobn](#): > > FWIW, I use `--mount=type=cache` in my docker builds and had zero issues with it so far. is there a way to get this to work with VSCode devcontainers? [Next page](https://forums.swift.org/t/incremental-builds-with-swift-package-manager-and-linux-docker/26538.md?page=2)