Proposal for Package Manager Testing Infrastruture


(Max Howell) #1

Testing is an essential part of modern software development.
Tight integration of testing into the Swift Package Manager
will help ensure a stable and reliable packaging ecosystem.

Here is our proposal for our integration: https://github.com/apple/swift-evolution/pull/51

Please reply all or comment on the PR, thanks!

Max


(Jed Lewison) #2

RE: "Any additional dependencies or dependencies that could not be automatically determined would need to be specified in a package manifest separately.”

Being able to establish test-only dependencies like Quick/Nimble in the test-module package manifest is great.

···

On Dec 14, 2015, at 10:48 AM, Max Howell via swift-evolution <swift-evolution@swift.org> wrote:

Testing is an essential part of modern software development.
Tight integration of testing into the Swift Package Manager
will help ensure a stable and reliable packaging ecosystem.

Here is our proposal for our integration: https://github.com/apple/swift-evolution/pull/51

Please reply all or comment on the PR, thanks!

Max
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Thomas Guthrie) #3

No mention of testing an individual module, something for a later proposal?

(Or individual test cases, though Xcode doesn’t seem to be able to do this for Swift tests last I checked so I’m not sure if thats an issue with XCTest and Swift?).

···

On 14 Dec 2015, at 18:48, Max Howell via swift-evolution <swift-evolution@swift.org> wrote:

Testing is an essential part of modern software development.
Tight integration of testing into the Swift Package Manager
will help ensure a stable and reliable packaging ecosystem.

Here is our proposal for our integration: https://github.com/apple/swift-evolution/pull/51

Please reply all or comment on the PR, thanks!

Max
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Max Howell) #4

Perhaps it is not clear but specifically the proposal is: one test-module per module.

···

No mention of testing an individual module, something for a later proposal?

(Or individual test cases, though Xcode doesn’t seem to be able to do this for Swift tests last I checked so I’m not sure if thats an issue with XCTest and Swift?).

On 14 Dec 2015, at 18:48, Max Howell via swift-evolution <swift-evolution@swift.org> wrote:

Testing is an essential part of modern software development.
Tight integration of testing into the Swift Package Manager
will help ensure a stable and reliable packaging ecosystem.

Here is our proposal for our integration: https://github.com/apple/swift-evolution/pull/51

Please reply all or comment on the PR, thanks!

Max
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Thomas Guthrie) #5

Perhaps it is not clear but specifically the proposal is: one test-module per module.

No that’s clear!

I think I was the one not being clear, was mostly curious about whether you had discussed (with a flag) **running** individual module’s tests (or more fine grained as in Xcode) as there is no mention of it like Debug/Release configuration and other future things.

(Clarifying the question a bit makes me think this is probably outside the scope of the proposal though.)

— Thomas


(Drew Crawford) #6

I'm wondering what the next steps for this proposal are.

I see Max's comment from the 16th <https://github.com/apple/swift-evolution/pull/51#issuecomment-165221854> advising to pull back discussion to the list, but I haven't seen any further discussion in the past 10 days. Does someone feel that there are outstanding problems to address? Even if there is some further discussion we need to have, IMO it would be appropriate to have that discussion in the context of a swift-evolution review.

I personally back the proposal essentially in its entirety. I think it is vague on some points we will want to take up as separate proposals, but I don't think they are worth holding up the ability to test packages at all, which is the present situation.

I am not familiar with what exactly the criteria is for that PR to be accepted into swift-evolution and begin a review, which IIRC is the next stage in the process. I do notice that it is the second oldest of the 12 open PRs in the queue.


(Max Howell) #7

OK, I see, yes this should be added to the proposal.

···

On Dec 15, 2015, at 3:19 PM, Thomas Guthrie <tomguthrie@gmail.com> wrote:

Perhaps it is not clear but specifically the proposal is: one test-module per module.

No that’s clear!

I think I was the one not being clear, was mostly curious about whether you had discussed (with a flag) **running** individual module’s tests (or more fine grained as in Xcode) as there is no mention of it like Debug/Release configuration and other future things.

(Clarifying the question a bit makes me think this is probably outside the scope of the proposal though.)

— Thomas


(Rick Ballard) #8

My apologies for the delay on this – this has been waiting on me to accept the pull request and schedule the formal review. Apple is on holiday shutdown at the moment so things are moving slowly right now. I will accept the PR soon and expect to schedule the review for the first week of January.

Cheers,

  - Rick

···

On Dec 26, 2015, at 11:33 PM, Drew Crawford via swift-evolution <swift-evolution@swift.org> wrote:

I'm wondering what the next steps for this proposal are.

I see Max's comment from the 16th <https://github.com/apple/swift-evolution/pull/51#issuecomment-165221854> advising to pull back discussion to the list, but I haven't seen any further discussion in the past 10 days. Does someone feel that there are outstanding problems to address? Even if there is some further discussion we need to have, IMO it would be appropriate to have that discussion in the context of a swift-evolution review.

I personally back the proposal essentially in its entirety. I think it is vague on some points we will want to take up as separate proposals, but I don't think they are worth holding up the ability to test packages at all, which is the present situation.

I am not familiar with what exactly the criteria is for that PR to be accepted into swift-evolution and begin a review, which IIRC is the next stage in the process. I do notice that it is the second oldest of the 12 open PRs in the queue.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution