Missing dependency in testing

Hi, swift-build-dev. I recently hit this failure <https://ci.swift.org/job/swift-PR-Linux-smoke-test/2159/&gt; on a pull request test that did nothing but change Swift’s test/lit.cfg—i.e. a test-configuration-only change in the compiler repo.

/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test@2/branch-master/swiftpm/Sources/swift-test/main.swift:11:8: error: module file was created by an older version of the compiler; rebuild 'Commands' and try again: /home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test@2/branch-master/buildbot_linux/swiftpm-linux-x86_64/debug/Commands.swiftmodule

import Commands
       ^

(and so on for Basic and TestSupport. Note that this was in the self-host stage of the CI build.)

This kind of failure implies that the compiler and standard library were rebuilt but the support modules of SwiftPM were not, which in turn seems like a missing dependency to me. Yes, a clean build will fix this, and yes, I’m not sure it matters to end users (who aren’t replacing their compiler and standard library arbitrarily), but it seems like there’s an easy answer here: add Swift.swiftmodule as a build dependency for this build. (Since it builds pretty quickly, maybe that’s just used to clean the build folder.) That way, you’ll always be up-to-date on the module format and the contents of the standard library.

There’s still a dependency trap here: if the only thing that changed was the Darwin or Glibc overlay, SwiftPM might not think it needs to rebuild. The actual correct answer is that your build depends on all swiftmodules you use. (And if static linking, all static archives as well, which may not always change in sync with the swiftmodule.)

What do you want to do here, SwiftPM folks?

Jordan

We currently have a built-in dependency edge on the Swift compiler which is supposed to catch this (we don't have one on just the standard libraries, but it doesn't seem like that was getting hit here?).

It looks like that didn't work here? Can you file a bug with as much info as you have?

I don't remember the details, but shouldn't we be getting all the .swiftmodules as dependencies reported by the compiler?

- Daniel

···

On Nov 1, 2016, at 9:40 AM, Jordan Rose via swift-build-dev <swift-build-dev@swift.org> wrote:

Hi, swift-build-dev. I recently hit this failure <https://ci.swift.org/job/swift-PR-Linux-smoke-test/2159/&gt; on a pull request test that did nothing but change Swift’s test/lit.cfg—i.e. a test-configuration-only change in the compiler repo.

/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test@2/branch-master/swiftpm/Sources/swift-test/main.swift:11:8: error: module file was created by an older version of the compiler; rebuild 'Commands' and try again: /home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test@2/branch-master/buildbot_linux/swiftpm-linux-x86_64/debug/Commands.swiftmodule

import Commands
       ^

(and so on for Basic and TestSupport. Note that this was in the self-host stage of the CI build.)

This kind of failure implies that the compiler and standard library were rebuilt but the support modules of SwiftPM were not, which in turn seems like a missing dependency to me. Yes, a clean build will fix this, and yes, I’m not sure it matters to end users (who aren’t replacing their compiler and standard library arbitrarily), but it seems like there’s an easy answer here: add Swift.swiftmodule as a build dependency for this build. (Since it builds pretty quickly, maybe that’s just used to clean the build folder.) That way, you’ll always be up-to-date on the module format and the contents of the standard library.

There’s still a dependency trap here: if the only thing that changed was the Darwin or Glibc overlay, SwiftPM might not think it needs to rebuild. The actual correct answer is that your build depends on all swiftmodules you use. (And if static linking, all static archives as well, which may not always change in sync with the swiftmodule.)

What do you want to do here, SwiftPM folks?

Jordan
_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

I'm sorry, I filed [SR-3107] SwiftPM's dependency on swiftc wasn't sufficient to prevent out-of-date modules in self-host build · Issue #5184 · apple/swift-package-manager · GitHub with the full build log, but I don't really have any more information. Yes, the compiler should be including dependencies on the swiftmodules it uses in the .d file—there's a test for it (test/Frontend/dependencies.swift).

Jordan

···

On Nov 1, 2016, at 9:53, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

We currently have a built-in dependency edge on the Swift compiler which is supposed to catch this (we don't have one on just the standard libraries, but it doesn't seem like that was getting hit here?).

It looks like that didn't work here? Can you file a bug with as much info as you have?

I don't remember the details, but shouldn't we be getting all the .swiftmodules as dependencies reported by the compiler?

- Daniel

On Nov 1, 2016, at 9:40 AM, Jordan Rose via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:

Hi, swift-build-dev. I recently hit this failure <https://ci.swift.org/job/swift-PR-Linux-smoke-test/2159/&gt; on a pull request test that did nothing but change Swift’s test/lit.cfg—i.e. a test-configuration-only change in the compiler repo.

/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test@2/branch-master/swiftpm/Sources/swift-test/main.swift:11:8: error: module file was created by an older version of the compiler; rebuild 'Commands' and try again: /home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test@2/branch-master/buildbot_linux/swiftpm-linux-x86_64/debug/Commands.swiftmodule

import Commands
       ^

(and so on for Basic and TestSupport. Note that this was in the self-host stage of the CI build.)

This kind of failure implies that the compiler and standard library were rebuilt but the support modules of SwiftPM were not, which in turn seems like a missing dependency to me. Yes, a clean build will fix this, and yes, I’m not sure it matters to end users (who aren’t replacing their compiler and standard library arbitrarily), but it seems like there’s an easy answer here: add Swift.swiftmodule as a build dependency for this build. (Since it builds pretty quickly, maybe that’s just used to clean the build folder.) That way, you’ll always be up-to-date on the module format and the contents of the standard library.

There’s still a dependency trap here: if the only thing that changed was the Darwin or Glibc overlay, SwiftPM might not think it needs to rebuild. The actual correct answer is that your build depends on all swiftmodules you use. (And if static linking, all static archives as well, which may not always change in sync with the swiftmodule.)

What do you want to do here, SwiftPM folks?

Jordan
_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev