[SwiftPM] Proposal: Add support for test-only dependencies


(thislooksfun) #1

(I think this is the right place for this suggestion, but please let me know if I'm mistaken)

There is currently no supported way to have some dependencies only used for testing (`swift test`), and the workarounds for it, while workable, are unnecessary crude.

Currently, what a lot of projects are doing is defining a `.Package.test.swift` file that is then copied over `Package.swift` when tests are to be run (on external CI, for example). While this works, it adds a extra step and another point of failure (if you forget to add a new dependency to both `Package` files, for example.

What I propose is a new section of `Package.swift`, labelled either `testDependencies` or `devDependencies`, and the modules referenced within are only loaded/compiled when running `swift test`

// Package.swift
import PackageDescription
let package = Package(
    name: "Project",
    targets: [
      Target(name: "BoxioDebug", dependencies: ["Core"]),
    ],
    dependencies: [
  //Same as before
    ],
    testDependencies: [
        .Package(url: "https://github.com/FooBar/PackageOnlyNeededForTesting.git", majorVersion: 1),
    ]
)

This solves having to manually switch out Package files or `.Package` statements inside one Package file when trying to run tests, and keeps all dependencies neatly organized in one file.

This change is purely additive and optional, no existing code needs to change for this to be added.

A real world example is a dependency on Quick (or another such testing framework), that itself uses XCTest, and thus crashes when being run outside of `swift test` (add Quick to Package.swift, run `swift build` and run the compiled binary). With the new `test[dev]Dependencies`, running `swift build` wouldn't even see Quick, and thus would compile and run correctly, but `swift test` would both see and use Quick for running the test suite.

Other solutions considered:
Another potential solution is to only compile the dependencies that were actually used when running `swift build`, since that seems to be the cause of the above problem.

-thislooksfun (tlf)


(Derrick Ho) #2

We need this.

···

On Mon, Dec 26, 2016 at 11:57 AM thislooksfun via swift-evolution < swift-evolution@swift.org> wrote:

(I think this is the right place for this suggestion, but please let me
know if I'm mistaken)

There is currently no supported way to have some dependencies only used
for testing (`swift test`), and the workarounds for it, while workable, are
unnecessary crude.

Currently, what a lot of projects are doing is defining a
`.Package.test.swift` file that is then copied over `Package.swift` when
tests are to be run (on external CI, for example). While this works, it
adds a extra step and another point of failure (if you forget to add a new
dependency to both `Package` files, for example.

What I propose is a new section of `Package.swift`, labelled either
`testDependencies` or `devDependencies`, and the modules referenced within
are only loaded/compiled when running `swift test`

// Package.swift
import PackageDescription
let package = Package(
    name: "Project",
    targets: [
      Target(name: "BoxioDebug", dependencies: ["Core"]),
    ],
    dependencies: [ //Same as before
    ],
    testDependencies: [
        .Package(url: "https://github.com/FooBar/PackageOnlyNeededForTesting.git", majorVersion: 1),
    ]
)

This solves having to manually switch out Package files or `.Package`
statements inside one Package file when trying to run tests, and keeps all
dependencies neatly organized in one file.

This change is purely additive and optional, no existing code needs to
change for this to be added.

A real world example is a dependency on Quick (or another such testing
framework), that itself uses XCTest, and thus crashes when being run
outside of `swift test` (add Quick to Package.swift, run `swift build` and
run the compiled binary). With the new `test[dev]Dependencies`, running
`swift build` wouldn't even see Quick, and thus would compile and run
correctly, but `swift test` would both see and use Quick for running the
test suite.

*Other solutions considered:*
Another potential solution is to only compile the dependencies that were
actually used when running `swift build`, since that seems to be the cause
of the above problem.

-thislooksfun (tlf)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(thislooksfun) #3

Does anyone else have any thoughts on this, or should I go ahead an assemble a PR?

-thislooksfun (tlf)

···

On Dec 26, 2016, at 1:56 PM, thislooksfun <thislooksfun@repbot.org> wrote:

(I think this is the right place for this suggestion, but please let me know if I'm mistaken)

There is currently no supported way to have some dependencies only used for testing (`swift test`), and the workarounds for it, while workable, are unnecessary crude.

Currently, what a lot of projects are doing is defining a `.Package.test.swift` file that is then copied over `Package.swift` when tests are to be run (on external CI, for example). While this works, it adds a extra step and another point of failure (if you forget to add a new dependency to both `Package` files, for example.

What I propose is a new section of `Package.swift`, labelled either `testDependencies` or `devDependencies`, and the modules referenced within are only loaded/compiled when running `swift test`

// Package.swift
import PackageDescription
let package = Package(
    name: "Project",
    targets: [
      Target(name: "BoxioDebug", dependencies: ["Core"]),
    ],
    dependencies: [
  //Same as before
    ],
    testDependencies: [
        .Package(url: "https://github.com/FooBar/PackageOnlyNeededForTesting.git", majorVersion: 1),
    ]
)

This solves having to manually switch out Package files or `.Package` statements inside one Package file when trying to run tests, and keeps all dependencies neatly organized in one file.

This change is purely additive and optional, no existing code needs to change for this to be added.

A real world example is a dependency on Quick (or another such testing framework), that itself uses XCTest, and thus crashes when being run outside of `swift test` (add Quick to Package.swift, run `swift build` and run the compiled binary). With the new `test[dev]Dependencies`, running `swift build` wouldn't even see Quick, and thus would compile and run correctly, but `swift test` would both see and use Quick for running the test suite.

Other solutions considered:
Another potential solution is to only compile the dependencies that were actually used when running `swift build`, since that seems to be the cause of the above problem.

-thislooksfun (tlf)


(Xiaodi Wu) #4

The need for a feature such as this was pointed out some time ago. A
proposal to allow dependencies to be declared with increased granularity
has already been written, reviewed, and accepted! Here it is:
https://github.com/apple/swift-evolution/blob/master/proposa
ls/0146-package-manager-product-definitions.md

According to the authors, the proposal was meant to be a more general
solution that enables specifying test dependencies as well as other use
cases:
https://lists.swift.org/pipermail/swift-build-dev/Week-of-Mon-20161114/000735.html

The proposal came up for review, received no feedback, and was accepted
as-is.

···

On Tue, Dec 27, 2016 at 5:41 PM, thislooksfun via swift-evolution < swift-evolution@swift.org> wrote:

Does anyone else have any thoughts on this, or should I go ahead an
assemble a PR?

-thislooksfun (tlf)

On Dec 26, 2016, at 1:56 PM, thislooksfun <thislooksfun@repbot.org> wrote:

(I think this is the right place for this suggestion, but please let me
know if I'm mistaken)

There is currently no supported way to have some dependencies only used
for testing (`swift test`), and the workarounds for it, while workable, are
unnecessary crude.

Currently, what a lot of projects are doing is defining a
`.Package.test.swift` file that is then copied over `Package.swift` when
tests are to be run (on external CI, for example). While this works, it
adds a extra step and another point of failure (if you forget to add a new
dependency to both `Package` files, for example.

What I propose is a new section of `Package.swift`, labelled either
`testDependencies` or `devDependencies`, and the modules referenced within
are only loaded/compiled when running `swift test`

// Package.swift
import PackageDescription
let package = Package(
    name: "Project",
    targets: [
      Target(name: "BoxioDebug", dependencies: ["Core"]),
    ],
    dependencies: [ //Same as before
    ],
    testDependencies: [
        .Package(url: "https://github.com/FooBar/PackageOnlyNeededForTesting.git", majorVersion: 1),
    ]
)

This solves having to manually switch out Package files or `.Package`
statements inside one Package file when trying to run tests, and keeps all
dependencies neatly organized in one file.

This change is purely additive and optional, no existing code needs to
change for this to be added.

A real world example is a dependency on Quick (or another such testing
framework), that itself uses XCTest, and thus crashes when being run
outside of `swift test` (add Quick to Package.swift, run `swift build` and
run the compiled binary). With the new `test[dev]Dependencies`, running
`swift build` wouldn't even see Quick, and thus would compile and run
correctly, but `swift test` would both see and use Quick for running the
test suite.

*Other solutions considered:*
Another potential solution is to only compile the dependencies that were
actually used when running `swift build`, since that seems to be the cause
of the above problem.

-thislooksfun (tlf)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Ankit Aggarwal) #5

- swift-evolution, + swift-build-dev

It might be worth discussing this feature separately from product proposal because IMO the test-only (or local) dependencies should not be considered in dependency resolution. With product proposal we will *not* compile the targets which are not vended by a package but we will still clone and resolve all the dependencies. This means two packages can not depend on conflicting versions of Quick (which might not make sense for local dependencies).

···

On 28-Dec-2016, at 5:31 AM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

The need for a feature such as this was pointed out some time ago. A proposal to allow dependencies to be declared with increased granularity has already been written, reviewed, and accepted! Here it is:
https://github.com/apple/swift-evolution/blob/master/proposals/0146-package-manager-product-definitions.md

According to the authors, the proposal was meant to be a more general solution that enables specifying test dependencies as well as other use cases:
https://lists.swift.org/pipermail/swift-build-dev/Week-of-Mon-20161114/000735.html

The proposal came up for review, received no feedback, and was accepted as-is.

On Tue, Dec 27, 2016 at 5:41 PM, thislooksfun via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Does anyone else have any thoughts on this, or should I go ahead an assemble a PR?

-thislooksfun (tlf)

On Dec 26, 2016, at 1:56 PM, thislooksfun <thislooksfun@repbot.org <mailto:thislooksfun@repbot.org>> wrote:

(I think this is the right place for this suggestion, but please let me know if I'm mistaken)

There is currently no supported way to have some dependencies only used for testing (`swift test`), and the workarounds for it, while workable, are unnecessary crude.

Currently, what a lot of projects are doing is defining a `.Package.test.swift` file that is then copied over `Package.swift` when tests are to be run (on external CI, for example). While this works, it adds a extra step and another point of failure (if you forget to add a new dependency to both `Package` files, for example.

What I propose is a new section of `Package.swift`, labelled either `testDependencies` or `devDependencies`, and the modules referenced within are only loaded/compiled when running `swift test`

// Package.swift
import PackageDescription
let package = Package(
    name: "Project",
    targets: [
      Target(name: "BoxioDebug", dependencies: ["Core"]),
    ],
    dependencies: [
  //Same as before
    ],
    testDependencies: [
        .Package(url: "https://github.com/FooBar/PackageOnlyNeededForTesting.git", majorVersion: 1),
    ]
)

This solves having to manually switch out Package files or `.Package` statements inside one Package file when trying to run tests, and keeps all dependencies neatly organized in one file.

This change is purely additive and optional, no existing code needs to change for this to be added.

A real world example is a dependency on Quick (or another such testing framework), that itself uses XCTest, and thus crashes when being run outside of `swift test` (add Quick to Package.swift, run `swift build` and run the compiled binary). With the new `test[dev]Dependencies`, running `swift build` wouldn't even see Quick, and thus would compile and run correctly, but `swift test` would both see and use Quick for running the test suite.

Other solutions considered:
Another potential solution is to only compile the dependencies that were actually used when running `swift build`, since that seems to be the cause of the above problem.

-thislooksfun (tlf)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(thislooksfun) #6

Does anyone over here have any feedback into this, or should I start writing up a PR?

-thislooksfun (tlf)

···

On Dec 28, 2016, at 11:33 PM, Ankit Aggarwal <ankit_aggarwal@apple.com> wrote:

- swift-evolution, + swift-build-dev

It might be worth discussing this feature separately from product proposal because IMO the test-only (or local) dependencies should not be considered in dependency resolution. With product proposal we will *not* compile the targets which are not vended by a package but we will still clone and resolve all the dependencies. This means two packages can not depend on conflicting versions of Quick (which might not make sense for local dependencies).

On 28-Dec-2016, at 5:31 AM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The need for a feature such as this was pointed out some time ago. A proposal to allow dependencies to be declared with increased granularity has already been written, reviewed, and accepted! Here it is:
https://github.com/apple/swift-evolution/blob/master/proposals/0146-package-manager-product-definitions.md

According to the authors, the proposal was meant to be a more general solution that enables specifying test dependencies as well as other use cases:
https://lists.swift.org/pipermail/swift-build-dev/Week-of-Mon-20161114/000735.html

The proposal came up for review, received no feedback, and was accepted as-is.

On Tue, Dec 27, 2016 at 5:41 PM, thislooksfun via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Does anyone else have any thoughts on this, or should I go ahead an assemble a PR?

-thislooksfun (tlf)

On Dec 26, 2016, at 1:56 PM, thislooksfun <thislooksfun@repbot.org <mailto:thislooksfun@repbot.org>> wrote:

(I think this is the right place for this suggestion, but please let me know if I'm mistaken)

There is currently no supported way to have some dependencies only used for testing (`swift test`), and the workarounds for it, while workable, are unnecessary crude.

Currently, what a lot of projects are doing is defining a `.Package.test.swift` file that is then copied over `Package.swift` when tests are to be run (on external CI, for example). While this works, it adds a extra step and another point of failure (if you forget to add a new dependency to both `Package` files, for example.

What I propose is a new section of `Package.swift`, labelled either `testDependencies` or `devDependencies`, and the modules referenced within are only loaded/compiled when running `swift test`

// Package.swift
import PackageDescription
let package = Package(
    name: "Project",
    targets: [
      Target(name: "BoxioDebug", dependencies: ["Core"]),
    ],
    dependencies: [
  //Same as before
    ],
    testDependencies: [
        .Package(url: "https://github.com/FooBar/PackageOnlyNeededForTesting.git", majorVersion: 1),
    ]
)

This solves having to manually switch out Package files or `.Package` statements inside one Package file when trying to run tests, and keeps all dependencies neatly organized in one file.

This change is purely additive and optional, no existing code needs to change for this to be added.

A real world example is a dependency on Quick (or another such testing framework), that itself uses XCTest, and thus crashes when being run outside of `swift test` (add Quick to Package.swift, run `swift build` and run the compiled binary). With the new `test[dev]Dependencies`, running `swift build` wouldn't even see Quick, and thus would compile and run correctly, but `swift test` would both see and use Quick for running the test suite.

Other solutions considered:
Another potential solution is to only compile the dependencies that were actually used when running `swift build`, since that seems to be the cause of the above problem.

-thislooksfun (tlf)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


#7

Agree with Ankit's thought of `testDependencies` being a weird misnomer in
certain cases -- + 1 to something like `devDependencies` (Rust/Cargo/NPM
use this term) or `localDependencies`

Another thing I think that's important is the ability to clone/fetch all
your deps without building or without having to run a test process -- right
now we have `swift package fetch` -- it'd be great if this included these
local dependencies (or had an option flag to include them)


(Ankit Aggarwal) #8

A good place to start would be writing the proposal in swift evolution format and posting it here.

I think we should call this feature localDependencies (instead of devDependencies) since these deps are local to a package. We should also mention the behaviour of local dependencies around pinning and editing.

···

On 07-Jan-2017, at 9:48 AM, thislooksfun <thislooksfun@repbot.org> wrote:

Does anyone over here have any feedback into this, or should I start writing up a PR?

-thislooksfun (tlf)

On Dec 28, 2016, at 11:33 PM, Ankit Aggarwal <ankit_aggarwal@apple.com> wrote:

- swift-evolution, + swift-build-dev

It might be worth discussing this feature separately from product proposal because IMO the test-only (or local) dependencies should not be considered in dependency resolution. With product proposal we will *not* compile the targets which are not vended by a package but we will still clone and resolve all the dependencies. This means two packages can not depend on conflicting versions of Quick (which might not make sense for local dependencies).

On 28-Dec-2016, at 5:31 AM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

The need for a feature such as this was pointed out some time ago. A proposal to allow dependencies to be declared with increased granularity has already been written, reviewed, and accepted! Here it is:
https://github.com/apple/swift-evolution/blob/master/proposals/0146-package-manager-product-definitions.md

According to the authors, the proposal was meant to be a more general solution that enables specifying test dependencies as well as other use cases:
https://lists.swift.org/pipermail/swift-build-dev/Week-of-Mon-20161114/000735.html

The proposal came up for review, received no feedback, and was accepted as-is.

On Tue, Dec 27, 2016 at 5:41 PM, thislooksfun via swift-evolution <swift-evolution@swift.org> wrote:
Does anyone else have any thoughts on this, or should I go ahead an assemble a PR?

-thislooksfun (tlf)

On Dec 26, 2016, at 1:56 PM, thislooksfun <thislooksfun@repbot.org> wrote:

(I think this is the right place for this suggestion, but please let me know if I'm mistaken)

There is currently no supported way to have some dependencies only used for testing (`swift test`), and the workarounds for it, while workable, are unnecessary crude.

Currently, what a lot of projects are doing is defining a `.Package.test.swift` file that is then copied over `Package.swift` when tests are to be run (on external CI, for example). While this works, it adds a extra step and another point of failure (if you forget to add a new dependency to both `Package` files, for example.

What I propose is a new section of `Package.swift`, labelled either `testDependencies` or `devDependencies`, and the modules referenced within are only loaded/compiled when running `swift test`

// Package.swift
import PackageDescription
let package = Package(
    name: "Project",
    targets: [
      Target(name: "BoxioDebug", dependencies: ["Core"]),
    ],
    dependencies: [
  //Same as before
    ],
    testDependencies: [
        .Package(url: "https://github.com/FooBar/PackageOnlyNeededForTesting.git", majorVersion: 1),
    ]
)

This solves having to manually switch out Package files or `.Package` statements inside one Package file when trying to run tests, and keeps all dependencies neatly organized in one file.

This change is purely additive and optional, no existing code needs to change for this to be added.

A real world example is a dependency on Quick (or another such testing framework), that itself uses XCTest, and thus crashes when being run outside of `swift test` (add Quick to Package.swift, run `swift build` and run the compiled binary). With the new `test[dev]Dependencies`, running `swift build` wouldn't even see Quick, and thus would compile and run correctly, but `swift test` would both see and use Quick for running the test suite.

Other solutions considered:
Another potential solution is to only compile the dependencies that were actually used when running `swift build`, since that seems to be the cause of the above problem.

-thislooksfun (tlf)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution