[Draft] SwiftPM: Adding development package as a dependency


(Ankit Agarwal) #1

SwiftPM: Adding development package as a dependency

   - Proposal: SE-NNNN
   <https://github.com/apple/swift-evolution/blob/master/proposals/NNNN-name.md>
   - Author(s): Ankit Aggarwal <https://github.com/aciidb0mb3r>
   - Status: *Awaiting review*
   - Review manager: TBD

Introduction

I propose to enable SwiftPM to use a package that is still under
development as a dependency for another package during testing and
development.
Motivation

During development of libraries, developers commonly want to try out their
module package as a dependency. This emulates the typical library use-case
and is currently not possible in SwiftPM without first checking in and
tagging that library. These extra steps, while reasonable for an
already-built package, are an unnecessary burden for a package that remains
in development.

Forcing the user to modify the library package inside Packages to continue
development or continuously reclone the package after recommiting and
retagging strain the process of building the library in the first place.
Detail Design

Under this proposal, the *root* package will be allowed to specify a
DevPackage dependency. This dependency will *not *clone the package inside
Packages/ or require the dependency to be under version control. This will
free the developer to continue iterative testing, expansion, and
enhancements without being tied to the current dependency system.

This approach limits DevPackage dependencies to local file systems. Remote
repositories cannot be used with this keyword.

The following example demonstrates what a manifest file would look like. In
this example, the DevPackage is specified using a local path and the
majorVersion is used as is for this DevPackage.

import PackageDescription
let package = Package(
    name: "MyLibraryTester",
    dependencies: [
        .Package(url:
"https://github.com/apple/example-package-fisheryates.git",
majorVersion: 1),
        .DevPackage(localPath: "../MyAwesomeLibrary", majorVersion: 1),
    ]
)

Under this design:

   - DevPackage is limited strictly to the root package. The manifest of
   any dependency containing a DevPackage will fail to build.
   - A DevPackage is not copied inside Packages/ and does not require
   version control.
   - SwiftPM uses the DevPackage's source directory for building,
   permitting in-place development on the local file system.
   - SwiftPM disallows non-local DevPackage sources. To use a remote
   package, the developer must first clone a package and then specify the
   local path.
   - Version numbers are specified for DevPackage entries within the
   manifest file.
   - Should the DevPackage version not be selected after resolving the
   dependency graph, the build will succeed with a warning.

Impact on existing code

This proposal does not impact existing code.
Alternatives considered

I propose two possible alternatives to this problem:

   1. Create a executable target within the library package for development
   testing.
   2. Use XCTest to test the library.

Both alternate approaches permit testing a library module but they will not
simulate a full SwiftPM package.
Acknowledgements

Thanks to Erica Sadun <https://github.com/erica> for inputs.

···

------------------------------------------------------------------------

Github link:
https://github.com/aciidb0mb3r/swift-evolution/blob/swiftpm_dev_package/proposals/NNNN-swiftpm-development-package-as-dependency.md

--
Ankit


(Max Howell) #2

This was more or less what I’ve been thinking, but lately I’ve been also thinking it would be great if:

You didn’t have to modify Package.swift to get this feature.

Because:

1) You have to remember to modify it back at some point, and if you are iterating frequently this is tedious and error-prone
2) We don’t want any chance that DevPackage gets into the package graph and thus the ecosystem.

Thoughts?

···

On Mar 29, 2016, at 11:33 AM, Ankit Agarwal via swift-build-dev <swift-build-dev@swift.org> wrote:

SwiftPM: Adding development package as a dependency
Proposal: SE-NNNN <https://github.com/apple/swift-evolution/blob/master/proposals/NNNN-name.md>
Author(s): Ankit Aggarwal <https://github.com/aciidb0mb3r>
Status: Awaiting review
Review manager: TBD
Introduction

I propose to enable SwiftPM to use a package that is still under development as a dependency for another package during testing and development.

Motivation

During development of libraries, developers commonly want to try out their module package as a dependency. This emulates the typical library use-case and is currently not possible in SwiftPM without first checking in and tagging that library. These extra steps, while reasonable for an already-built package, are an unnecessary burden for a package that remains in development.

Forcing the user to modify the library package inside Packages to continue development or continuously reclone the package after recommiting and retagging strain the process of building the library in the first place.

Detail Design

Under this proposal, the root package will be allowed to specify a DevPackage dependency. This dependency will not clone the package inside Packages/ or require the dependency to be under version control. This will free the developer to continue iterative testing, expansion, and enhancements without being tied to the current dependency system.

This approach limits DevPackage dependencies to local file systems. Remote repositories cannot be used with this keyword.

The following example demonstrates what a manifest file would look like. In this example, the DevPackage is specified using a local path and the majorVersion is used as is for this DevPackage.

import PackageDescription

let package = Package(
    name: "MyLibraryTester",
    dependencies: [
        .Package(url: "https://github.com/apple/example-package-fisheryates.git", majorVersion: 1),
        .DevPackage(localPath: "../MyAwesomeLibrary", majorVersion: 1),
    ]
)
Under this design:

DevPackage is limited strictly to the root package. The manifest of any dependency containing a DevPackage will fail to build.
A DevPackage is not copied inside Packages/ and does not require version control.
SwiftPM uses the DevPackage's source directory for building, permitting in-place development on the local file system.
SwiftPM disallows non-local DevPackage sources. To use a remote package, the developer must first clone a package and then specify the local path.
Version numbers are specified for DevPackage entries within the manifest file.
Should the DevPackage version not be selected after resolving the dependency graph, the build will succeed with a warning.
Impact on existing code

This proposal does not impact existing code.

Alternatives considered

I propose two possible alternatives to this problem:

Create a executable target within the library package for development testing.
Use XCTest to test the library.
Both alternate approaches permit testing a library module but they will not simulate a full SwiftPM package.

Acknowledgements

Thanks to Erica Sadun <https://github.com/erica> for inputs.

------------------------------------------------------------------------

Github link: https://github.com/aciidb0mb3r/swift-evolution/blob/swiftpm_dev_package/proposals/NNNN-swiftpm-development-package-as-dependency.md

--
Ankit

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(James Richard) #3

It's be nice to include a bit about how these interact with generated Xcodeproj files. In Cocoapods those end up in a separate grouping, which I find to be beneficial.

···

Sent from my iPhone

On Mar 29, 2016, at 11:33 AM, Ankit Agarwal via swift-evolution <swift-evolution@swift.org> wrote:

SwiftPM: Adding development package as a dependency
Proposal: SE-NNNN
Author(s): Ankit Aggarwal
Status: Awaiting review
Review manager: TBD
Introduction

I propose to enable SwiftPM to use a package that is still under development as a dependency for another package during testing and development.

Motivation

During development of libraries, developers commonly want to try out their module package as a dependency. This emulates the typical library use-case and is currently not possible in SwiftPM without first checking in and tagging that library. These extra steps, while reasonable for an already-built package, are an unnecessary burden for a package that remains in development.

Forcing the user to modify the library package inside Packages to continue development or continuously reclone the package after recommiting and retagging strain the process of building the library in the first place.

Detail Design

Under this proposal, the root package will be allowed to specify a DevPackage dependency. This dependency will not clone the package inside Packages/ or require the dependency to be under version control. This will free the developer to continue iterative testing, expansion, and enhancements without being tied to the current dependency system.

This approach limits DevPackage dependencies to local file systems. Remote repositories cannot be used with this keyword.

The following example demonstrates what a manifest file would look like. In this example, the DevPackage is specified using a local path and the majorVersion is used as is for this DevPackage.

import PackageDescription

let package = Package(
    name: "MyLibraryTester",
    dependencies: [
        .Package(url: "https://github.com/apple/example-package-fisheryates.git", majorVersion: 1),
        .DevPackage(localPath: "../MyAwesomeLibrary", majorVersion: 1),
    ]
)
Under this design:

DevPackage is limited strictly to the root package. The manifest of any dependency containing a DevPackage will fail to build.
A DevPackage is not copied inside Packages/ and does not require version control.
SwiftPM uses the DevPackage's source directory for building, permitting in-place development on the local file system.
SwiftPM disallows non-local DevPackage sources. To use a remote package, the developer must first clone a package and then specify the local path.
Version numbers are specified for DevPackage entries within the manifest file.
Should the DevPackage version not be selected after resolving the dependency graph, the build will succeed with a warning.
Impact on existing code

This proposal does not impact existing code.

Alternatives considered

I propose two possible alternatives to this problem:

Create a executable target within the library package for development testing.
Use XCTest to test the library.
Both alternate approaches permit testing a library module but they will not simulate a full SwiftPM package.

Acknowledgements

Thanks to Erica Sadun for inputs.

------------------------------------------------------------------------

Github link: https://github.com/aciidb0mb3r/swift-evolution/blob/swiftpm_dev_package/proposals/NNNN-swiftpm-development-package-as-dependency.md

--
Ankit

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Ankit Agarwal) #4

This was more or less what I’ve been thinking, but lately I’ve been also
thinking it would be great if:

You didn’t have to modify Package.swift to get this feature.

Because:

1) You have to remember to modify it back at some point, and if you are
iterating frequently this is tedious and error-prone

One way I can think of to avoid Package.swift is to place DevPackages
inside some special folder (perhaps: DevPackages/) inside the root package
which sounds good if I am developing some patch to some package but it
might be awkward if I am starting to write a library and want to create a
package to try it because I'll already have created the library package
(though maybe minimal) but then I'll have to move the library inside
DevPackages/

This would be a bit awkward but sounds good enough to me

2) We don’t want any chance that DevPackage gets into the package graph
and thus the ecosystem.

what if one of the dependencies depend on a package inside DevPackage then
should the DevPackage be preferred?


(Ankit Agarwal) #5

It should do the same as it does for other dependencies ie create a target
and group it under "dependencies". If you're thinking about workspaces and
subprojects that is not in the scope of this proposal.

···

On Wed, Mar 30, 2016 at 6:05 AM, James Richard <ketzu@me.com> wrote:

It's be nice to include a bit about how these interact with generated
Xcodeproj files. In Cocoapods those end up in a separate grouping, which I
find to be beneficial.

--
Ankit


(Max Howell) #6

1) You have to remember to modify it back at some point, and if you are iterating frequently this is tedious and error-prone

One way I can think of to avoid Package.swift is to place DevPackages inside some special folder (perhaps: DevPackages/) inside the root package which sounds good if I am developing some patch to some package but it might be awkward if I am starting to write a library and want to create a package to try it because I'll already have created the library package (though maybe minimal) but then I'll have to move the library inside DevPackages/

Well, it seems to me part of the utility here is having a package be a local clone in an entirely different directory. So this sounds a bit tedious.

This would be a bit awkward but sounds good enough to me

2) We don’t want any chance that DevPackage gets into the package graph and thus the ecosystem.

what if one of the dependencies depend on a package inside DevPackage then should the DevPackage be preferred?

Don’t understand.


(Ankit Agarwal) #7

1) You have to remember to modify it back at some point, and if you are

iterating frequently this is tedious and error-prone

One way I can think of to avoid Package.swift is to place DevPackages
inside some special folder (perhaps: DevPackages/) inside the root package
which sounds good if I am developing some patch to some package but it
might be awkward if I am starting to write a library and want to create a
package to try it because I'll already have created the library package
(though maybe minimal) but then I'll have to move the library inside
DevPackages/

Well, it seems to me part of the utility here is having a package be a
local clone in an entirely different directory. So this sounds a bit
tedious.

Any other way you can think of to avoid Package.swift?

This would be a bit awkward but sounds good enough to me

2) We don’t want any chance that DevPackage gets into the package graph
and thus the ecosystem.

what if one of the dependencies depend on a package inside DevPackage then
should the DevPackage be preferred?

Don’t understand.

RootPackage
Dependency: APackage
DevPackage: BPackage

APackage
Dependency: BPackage

If DevPackages don't specify version in Package.swift, in case of a
collision should the DevPackage be always preferred?

···

On Wed, Mar 30, 2016 at 1:01 AM, Max Howell <max.howell@apple.com> wrote:

--
Ankit


(Max Howell) #8

1) You have to remember to modify it back at some point, and if you are iterating frequently this is tedious and error-prone

One way I can think of to avoid Package.swift is to place DevPackages inside some special folder (perhaps: DevPackages/) inside the root package which sounds good if I am developing some patch to some package but it might be awkward if I am starting to write a library and want to create a package to try it because I'll already have created the library package (though maybe minimal) but then I'll have to move the library inside DevPackages/

Well, it seems to me part of the utility here is having a package be a local clone in an entirely different directory. So this sounds a bit tedious.

Any other way you can think of to avoid Package.swift?

This seems pretty similar to standard overrides as part of the lockfile proposal to me, so we could add it on there, maybe with
a convenience command line.

This would be a bit awkward but sounds good enough to me

2) We don’t want any chance that DevPackage gets into the package graph and thus the ecosystem.

what if one of the dependencies depend on a package inside DevPackage then should the DevPackage be preferred?

Don’t understand.

RootPackage
Dependency: APackage
DevPackage: BPackage

APackage
Dependency: BPackage

If DevPackages don't specify version in Package.swift, in case of a collision should the DevPackage be always preferred?

Root package always has override precedence.


(Ankit Agarwal) #9

Okay, does this sound good :

`swift build --dev-pkg=../MyLib --dev-pkg=../MyLib2`

···

On Wed, Mar 30, 2016 at 1:10 AM, Max Howell <max.howell@apple.com> wrote:

1) You have to remember to modify it back at some point, and if you are

iterating frequently this is tedious and error-prone

One way I can think of to avoid Package.swift is to place DevPackages
inside some special folder (perhaps: DevPackages/) inside the root package
which sounds good if I am developing some patch to some package but it
might be awkward if I am starting to write a library and want to create a
package to try it because I'll already have created the library package
(though maybe minimal) but then I'll have to move the library inside
DevPackages/

Well, it seems to me part of the utility here is having a package be a
local clone in an entirely different directory. So this sounds a bit
tedious.

Any other way you can think of to avoid Package.swift?

This seems pretty similar to standard overrides as part of the lockfile
proposal to me, so we could add it on there, maybe with
a convenience command line.

--
Ankit


(Ankit Agarwal) #10

If this looks good can we move ahead for review ?

···

--
Ankit


(Ankit Agarwal) #11

Updated the proposal to commandline override here:
https://github.com/aciidb0mb3r/swift-evolution/blob/swiftpm_dev_package/proposals/NNNN-swiftpm-development-package-as-dependency.md

···

On Wed, Mar 30, 2016 at 1:17 AM, Ankit Agarwal <ankit@ankit.im> wrote:

On Wed, Mar 30, 2016 at 1:10 AM, Max Howell <max.howell@apple.com> wrote:

1) You have to remember to modify it back at some point, and if you are

iterating frequently this is tedious and error-prone

One way I can think of to avoid Package.swift is to place DevPackages
inside some special folder (perhaps: DevPackages/) inside the root package
which sounds good if I am developing some patch to some package but it
might be awkward if I am starting to write a library and want to create a
package to try it because I'll already have created the library package
(though maybe minimal) but then I'll have to move the library inside
DevPackages/

Well, it seems to me part of the utility here is having a package be a
local clone in an entirely different directory. So this sounds a bit
tedious.

Any other way you can think of to avoid Package.swift?

This seems pretty similar to standard overrides as part of the lockfile
proposal to me, so we could add it on there, maybe with
a convenience command line.

Okay, does this sound good :

`swift build --dev-pkg=../MyLib --dev-pkg=../MyLib2`

--
Ankit

--
Ankit


(Max Howell) #12

I’m wondering if the dev package should be inside Packages in some manner, perhaps as a symlink.

Otherwise when you looks inside Packages you don't see all your dependencies.

Also since this is related to the lockfile work, I’d like to wait until that is done, probably we could use very similar command line UX.

If you don’t mind?

I’ll revise the lockfile proposal today and get it moving.

Thanks for all your great work :+1:t2:

···

If this looks good can we move ahead for review ?


(Ankit Agarwal) #13

I’m wondering if the dev package should be inside Packages in some manner,
perhaps as a symlink.

Otherwise when you looks inside Packages you don't see all your
dependencies.

Since as per current design dev packages will only be included if it is in
the `swift build` command I don't think its ever a concrete dependency to
be inside `Packages/` in any way.

Also since this is related to the lockfile work, I’d like to wait until
that is done, probably we could use very similar command line UX.

If you don’t mind?

Cool.

···

On Fri, Apr 1, 2016 at 10:45 PM, Max Howell <max.howell@apple.com> wrote:

--
Ankit