SPM support for branches and commits

Said_Sikira · September 6, 2016, 1:41pm

Hi,

Our code is almost always developed and pushed in small incremental
changes. When we implement critical amount of changes in our code, we push
new version.

When adding dependencies to Package.swift file, we supply their repository
url and version we want to use. However, differentiating code only by it’s
version is not enough in some cases. When writing new code features, we use
branches. They enable all the mechanisms one needs to create, test and
deploy new features without polluting production environment.

I think that Swift Package Manager should have support for branches (and
commits). There are several reasons why this feature would greatly improve
developer workflow:

   1. Writing new features. Being able to specify branch in Package.swift
   would make creating and testing new features easier. You wouldn’t need to
   push new version to be able to use it in your Swift program. You would just
   specify the branch you’re working on.
   2. Differentiating between new Swift versions. This problem comes from
   the current Swift 2.2 -> Swift 3.0 migration. Many framework developers use
   specific branches (swift–3, swift3.0) to work on migration of their API’s
   to Swift 3. However, you can’t use them in your Swift projects because they
   don’t live in the master branch in the repository. I’m sure this will also
   happen when Swift 3 starts migration to the Swift 4, until ABI becomes
   stable.

SPM should also have support for specifying commits. Specifying which
commit you want to use in your project dependency is not always a good
idea, but it’s necessary in some cases.

This shouldn’t be very hard to implement. We would need to update
PackageDescription and Get source from swift-package-manager repository to
enable specifying branches or commits. Pulling the branch source would just
be another parameter in git instruction.

Example:

// Specifying branch
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git", branch: "new-feature")
  ]
)

// Specifying commits
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git", commit:
"c336664020v4f94ed78cbe7447a39ae5ca0b6c11")
  ]
)

What are your thoughts on this subject?

Guillaume_DIDIER · September 6, 2016, 1:49pm

I think that the ability to fetch a branch or a commit may be interesting
(basically anything that git understands), would be nice,
but with the caveat that for production use it is far better to use à fixed version (i.e. a tag).
We do not want that most packages are used by specifying the master branch.

Guillaume DIDIER

···

—
ÉCOLE POLYTECHNIQUE
91128 PALAISEAU CEDEX
guillaume.didier@polytechnique.edu <mailto:guillaume.didier@polytechnique.edu?subject=>
www.polytechnique.edu <http://www.polytechnique.edu/>
—

PS : I am new here, do not hesitate to correct me.

Le 6 sept. 2016 à 15:41, Said Sikira via swift-evolution <swift-evolution@swift.org> a écrit :

Hi,

Our code is almost always developed and pushed in small incremental changes. When we implement critical amount of changes in our code, we push new version.

When adding dependencies to Package.swift file, we supply their repository url and version we want to use. However, differentiating code only by it’s version is not enough in some cases. When writing new code features, we use branches. They enable all the mechanisms one needs to create, test and deploy new features without polluting production environment.

I think that Swift Package Manager should have support for branches (and commits). There are several reasons why this feature would greatly improve developer workflow:

Writing new features. Being able to specify branch in Package.swift would make creating and testing new features easier. You wouldn’t need to push new version to be able to use it in your Swift program. You would just specify the branch you’re working on.
Differentiating between new Swift versions. This problem comes from the current Swift 2.2 -> Swift 3.0 migration. Many framework developers use specific branches (swift–3, swift3.0) to work on migration of their API’s to Swift 3. However, you can’t use them in your Swift projects because they don’t live in the master branch in the repository. I’m sure this will also happen when Swift 3 starts migration to the Swift 4, until ABI becomes stable.
SPM should also have support for specifying commits. Specifying which commit you want to use in your project dependency is not always a good idea, but it’s necessary in some cases.

This shouldn’t be very hard to implement. We would need to update PackageDescription and Get source from swift-package-manager repository to enable specifying branches or commits. Pulling the branch source would just be another parameter in git instruction.

Example:

// Specifying branch
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git <https://repo-source.git/>", branch: "new-feature")
  ]
)
// Specifying commits
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git <https://repo-source.git/>", commit: "c336664020v4f94ed78cbe7447a39ae5ca0b6c11")
  ]
)
What are your thoughts on this subject?

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Said_Sikira · September 6, 2016, 2:08pm

Yes, I agree that tags are far better option for production use, but
development process stage would certainly benefit from possibility to track
specific branch or commit. Having to rely completely on versioned tags in
development stage would not be easy nor very productive.

Said

···

On September 6, 2016 at 3:49:06 PM, Guillaume DIDIER ( guillaume.didier.2014@polytechnique.org) wrote:

I think that the ability to fetch a branch or a commit may be interesting
(basically anything that git understands), would be nice,
but with the caveat that for production use it is far better to use à fixed
version (i.e. a tag).
We do not want that most packages are used by specifying the master branch.

*Guillaume DIDIER *
—
*ÉCOLE POLYTECHNIQUE*
91128 PALAISEAU CEDEX
guillaume.didier@polytechnique.edu
<guillaume.didier@polytechnique.edu?subject=>
www.polytechnique.edu
—

PS : I am new here, do not hesitate to correct me.

Le 6 sept. 2016 à 15:41, Said Sikira via swift-evolution < swift-evolution@swift.org> a écrit :

Hi,

Our code is almost always developed and pushed in small incremental
changes. When we implement critical amount of changes in our code, we push
new version.

When adding dependencies to Package.swift file, we supply their repository
url and version we want to use. However, differentiating code only by it’s
version is not enough in some cases. When writing new code features, we use
branches. They enable all the mechanisms one needs to create, test and
deploy new features without polluting production environment.

I think that Swift Package Manager should have support for branches (and
commits). There are several reasons why this feature would greatly improve
developer workflow:

   1. Writing new features. Being able to specify branch in Package.swift
   would make creating and testing new features easier. You wouldn’t need to
   push new version to be able to use it in your Swift program. You would just
   specify the branch you’re working on.
   2. Differentiating between new Swift versions. This problem comes from
   the current Swift 2.2 -> Swift 3.0 migration. Many framework developers use
   specific branches (swift–3, swift3.0) to work on migration of their API’s
   to Swift 3. However, you can’t use them in your Swift projects because they
   don’t live in the master branch in the repository. I’m sure this will also
   happen when Swift 3 starts migration to the Swift 4, until ABI becomes
   stable.

SPM should also have support for specifying commits. Specifying which
commit you want to use in your project dependency is not always a good
idea, but it’s necessary in some cases.

This shouldn’t be very hard to implement. We would need to update
PackageDescription and Get source from swift-package-manager repository to
enable specifying branches or commits. Pulling the branch source would just
be another parameter in git instruction.

Example:

// Specifying branch
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git", branch: "new-feature")
  ]
)

// Specifying commits
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git", commit:
"c336664020v4f94ed78cbe7447a39ae5ca0b6c11")
  ]
)

What are your thoughts on this subject?
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Daniel_Leping · September 6, 2016, 2:31pm

+1 here. It becomes much more crucial when you work with finely grained
projects with each module in its own repo. Making a new release when you
push a fix becomes too much of job to do.

On Tue, Sep 6, 2016 at 5:08 PM, Said Sikira via swift-evolution <

···

swift-evolution@swift.org> wrote:

Yes, I agree that tags are far better option for production use, but
development process stage would certainly benefit from possibility to track
specific branch or commit. Having to rely completely on versioned tags in
development stage would not be easy nor very productive.

Said

On September 6, 2016 at 3:49:06 PM, Guillaume DIDIER ( > guillaume.didier.2014@polytechnique.org) wrote:

I think that the ability to fetch a branch or a commit may be interesting
(basically anything that git understands), would be nice,
but with the caveat that for production use it is far better to use à
fixed version (i.e. a tag).
We do not want that most packages are used by specifying the master branch.

*Guillaume DIDIER *
—
*ÉCOLE POLYTECHNIQUE*
91128 PALAISEAU CEDEX
guillaume.didier@polytechnique.edu
<guillaume.didier@polytechnique.edu?subject=>
www.polytechnique.edu
—

PS : I am new here, do not hesitate to correct me.

Le 6 sept. 2016 à 15:41, Said Sikira via swift-evolution < > swift-evolution@swift.org> a écrit :

Hi,

Our code is almost always developed and pushed in small incremental
changes. When we implement critical amount of changes in our code, we push
new version.

When adding dependencies to Package.swift file, we supply their repository
url and version we want to use. However, differentiating code only by it’s
version is not enough in some cases. When writing new code features, we use
branches. They enable all the mechanisms one needs to create, test and
deploy new features without polluting production environment.

I think that Swift Package Manager should have support for branches (and
commits). There are several reasons why this feature would greatly improve
developer workflow:

   1. Writing new features. Being able to specify branch in Package.swift
   would make creating and testing new features easier. You wouldn’t need to
   push new version to be able to use it in your Swift program. You would just
   specify the branch you’re working on.
   2. Differentiating between new Swift versions. This problem comes from
   the current Swift 2.2 -> Swift 3.0 migration. Many framework developers use
   specific branches (swift–3, swift3.0) to work on migration of their API’s
   to Swift 3. However, you can’t use them in your Swift projects because they
   don’t live in the master branch in the repository. I’m sure this will also
   happen when Swift 3 starts migration to the Swift 4, until ABI becomes
   stable.

SPM should also have support for specifying commits. Specifying which
commit you want to use in your project dependency is not always a good
idea, but it’s necessary in some cases.

This shouldn’t be very hard to implement. We would need to update
PackageDescription and Get source from swift-package-manager repository to
enable specifying branches or commits. Pulling the branch source would just
be another parameter in git instruction.

Example:

// Specifying branch
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git", branch: "new-feature")
  ]
)

// Specifying commits
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git", commit: "c336664020v4f94ed78cbe7447a39ae5ca0b6c11")
  ]
)

What are your thoughts on this subject?
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

ddunbar · September 6, 2016, 4:06pm

(+swift-build-dev, which is more focused for SwiftPM work)

Hi Said,

I agree, this is something we need.

One of the things that a complete proposal for adding this needs is a precise definition for the workflow and exploration about the impact on other features. Some of my questions are:

1. When will those commits be resolved and packages updated? For example, we might find it useful for `swift build` to automatically fetch packages and prompt when new versions are available when using tagged dependencies -- is that behavior appropriate when tracking a branch?

2. How do we manage the presence of these attributes w.r.t. producing reliable package graphs. For example, should this be allowed in all circumstances, or should it only be allowed when accessing a package via an "untagged" path? If I depend on B, and B updates and pushes a tag to start referencing a branch, do I see an error, or a warning, to let me know my product is no longer being built from tagged repositories?

3. We anticipate needing an explicit workflow for development against an untagged group of packages (we've been colloquially calling this "master"-style development). If we had that feature, which let you just use a bunch of packages in whatever state they were in on the local filesystem, would that change how you felt about wanting this in the `Package.swift` manifest?

I am in favor of getting a complete proposal for this feature on the books, and agree with you that implementing it should be relatively easy. The proposal doesn't have to be elaborate, but I think it does need to work through some of the impact of this feature and ways we can mitigate potential problems.

Cheers,
- Daniel

···

On Sep 6, 2016, at 7:31 AM, Daniel Leping via swift-evolution <swift-evolution@swift.org> wrote:

+1 here. It becomes much more crucial when you work with finely grained projects with each module in its own repo. Making a new release when you push a fix becomes too much of job to do.

On Tue, Sep 6, 2016 at 5:08 PM, Said Sikira via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Yes, I agree that tags are far better option for production use, but development process stage would certainly benefit from possibility to track specific branch or commit. Having to rely completely on versioned tags in development stage would not be easy nor very productive.

Said

On September 6, 2016 at 3:49:06 PM, Guillaume DIDIER (guillaume.didier.2014@polytechnique.org <mailto:guillaume.didier.2014@polytechnique.org>) wrote:

I think that the ability to fetch a branch or a commit may be interesting
(basically anything that git understands), would be nice,
but with the caveat that for production use it is far better to use à fixed version (i.e. a tag).
We do not want that most packages are used by specifying the master branch.

Guillaume DIDIER
—
ÉCOLE POLYTECHNIQUE
91128 PALAISEAU CEDEX
guillaume.didier@polytechnique.edu <mailto:guillaume.didier@polytechnique.edu?subject=>
www.polytechnique.edu <http://www.polytechnique.edu/>
—

PS : I am new here, do not hesitate to correct me.

Le 6 sept. 2016 à 15:41, Said Sikira via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

Hi,

Our code is almost always developed and pushed in small incremental changes. When we implement critical amount of changes in our code, we push new version.

When adding dependencies to Package.swift file, we supply their repository url and version we want to use. However, differentiating code only by it’s version is not enough in some cases. When writing new code features, we use branches. They enable all the mechanisms one needs to create, test and deploy new features without polluting production environment.

I think that Swift Package Manager should have support for branches (and commits). There are several reasons why this feature would greatly improve developer workflow:

Writing new features. Being able to specify branch in Package.swift would make creating and testing new features easier. You wouldn’t need to push new version to be able to use it in your Swift program. You would just specify the branch you’re working on.
Differentiating between new Swift versions. This problem comes from the current Swift 2.2 -> Swift 3.0 migration. Many framework developers use specific branches (swift–3, swift3.0) to work on migration of their API’s to Swift 3. However, you can’t use them in your Swift projects because they don’t live in the master branch in the repository. I’m sure this will also happen when Swift 3 starts migration to the Swift 4, until ABI becomes stable.
SPM should also have support for specifying commits. Specifying which commit you want to use in your project dependency is not always a good idea, but it’s necessary in some cases.

This shouldn’t be very hard to implement. We would need to update PackageDescription and Get source from swift-package-manager repository to enable specifying branches or commits. Pulling the branch source would just be another parameter in git instruction.

Example:

// Specifying branch
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git <https://repo-source.git/>", branch: "new-feature")
  ]
)
// Specifying commits
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git <https://repo-source.git/>", commit: "c336664020v4f94ed78cbe7447a39ae5ca0b6c11")
  ]
)
What are your thoughts on this subject?

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Cavelle_Benjamin · September 6, 2016, 6:15pm

+1
My first time saying anything on the distribution.

Working on Swift 3 migration, a lot of github projects have an non-versioned swift3 branch. I've had to fork projects and create my own version tags... A lot of unnecessary work.

Cheers,
CB

···

On Sep 6, 2016, at 12:06, Daniel Dunbar via swift-evolution <swift-evolution@swift.org> wrote:

(+swift-build-dev, which is more focused for SwiftPM work)

Hi Said,

I agree, this is something we need.

One of the things that a complete proposal for adding this needs is a precise definition for the workflow and exploration about the impact on other features. Some of my questions are:

1. When will those commits be resolved and packages updated? For example, we might find it useful for `swift build` to automatically fetch packages and prompt when new versions are available when using tagged dependencies -- is that behavior appropriate when tracking a branch?

2. How do we manage the presence of these attributes w.r.t. producing reliable package graphs. For example, should this be allowed in all circumstances, or should it only be allowed when accessing a package via an "untagged" path? If I depend on B, and B updates and pushes a tag to start referencing a branch, do I see an error, or a warning, to let me know my product is no longer being built from tagged repositories?

3. We anticipate needing an explicit workflow for development against an untagged group of packages (we've been colloquially calling this "master"-style development). If we had that feature, which let you just use a bunch of packages in whatever state they were in on the local filesystem, would that change how you felt about wanting this in the `Package.swift` manifest?

I am in favor of getting a complete proposal for this feature on the books, and agree with you that implementing it should be relatively easy. The proposal doesn't have to be elaborate, but I think it does need to work through some of the impact of this feature and ways we can mitigate potential problems.

Cheers,
- Daniel

On Sep 6, 2016, at 7:31 AM, Daniel Leping via swift-evolution <swift-evolution@swift.org> wrote:

+1 here. It becomes much more crucial when you work with finely grained projects with each module in its own repo. Making a new release when you push a fix becomes too much of job to do.

On Tue, Sep 6, 2016 at 5:08 PM, Said Sikira via swift-evolution <swift-evolution@swift.org> wrote:

Yes, I agree that tags are far better option for production use, but development process stage would certainly benefit from possibility to track specific branch or commit. Having to rely completely on versioned tags in development stage would not be easy nor very productive.

Said

On September 6, 2016 at 3:49:06 PM, Guillaume DIDIER (guillaume.didier.2014@polytechnique.org) wrote:

I think that the ability to fetch a branch or a commit may be interesting
(basically anything that git understands), would be nice,
but with the caveat that for production use it is far better to use à fixed version (i.e. a tag).
We do not want that most packages are used by specifying the master branch.

Guillaume DIDIER
—
ÉCOLE POLYTECHNIQUE
91128 PALAISEAU CEDEX
guillaume.didier@polytechnique.edu
www.polytechnique.edu
—

PS : I am new here, do not hesitate to correct me.

Le 6 sept. 2016 à 15:41, Said Sikira via swift-evolution <swift-evolution@swift.org> a écrit :

Hi,

Our code is almost always developed and pushed in small incremental changes. When we implement critical amount of changes in our code, we push new version.

When adding dependencies to Package.swift file, we supply their repository url and version we want to use. However, differentiating code only by it’s version is not enough in some cases. When writing new code features, we use branches. They enable all the mechanisms one needs to create, test and deploy new features without polluting production environment.

I think that Swift Package Manager should have support for branches (and commits). There are several reasons why this feature would greatly improve developer workflow:

Writing new features. Being able to specify branch in Package.swift would make creating and testing new features easier. You wouldn’t need to push new version to be able to use it in your Swift program. You would just specify the branch you’re working on.
Differentiating between new Swift versions. This problem comes from the current Swift 2.2 -> Swift 3.0 migration. Many framework developers use specific branches (swift–3, swift3.0) to work on migration of their API’s to Swift 3. However, you can’t use them in your Swift projects because they don’t live in the master branch in the repository. I’m sure this will also happen when Swift 3 starts migration to the Swift 4, until ABI becomes stable.
SPM should also have support for specifying commits. Specifying which commit you want to use in your project dependency is not always a good idea, but it’s necessary in some cases.

This shouldn’t be very hard to implement. We would need to update PackageDescription and Get source from swift-package-manager repository to enable specifying branches or commits. Pulling the branch source would just be another parameter in git instruction.

Example:

// Specifying branch
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git", branch: "new-feature")
  ]
)
// Specifying commits
let package = Package(
  name: "SomePackage",
  dependencies: [
    .Package(url: "https://repo-source.git", commit: "c336664020v4f94ed78cbe7447a39ae5ca0b6c11")
  ]
)
What are your thoughts on this subject?

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Paul_Cantrell · September 6, 2016, 9:10pm

Working on Swift 3 migration, a lot of github projects have an non-versioned swift3 branch. I've had to fork projects and create my own version tags... A lot of unnecessary work.

A thousand times this.

Clearly using carefully versioned releases is preferable, but the lack of branch support in SwiftPM is a point of pain.

My several cents on Daniel’s questions:

One of the things that a complete proposal for adding this needs is a precise definition for the workflow and exploration about the impact on other features. Some of my questions are:

1. When will those commits be resolved and packages updated? For example, we might find it useful for `swift build` to automatically fetch packages and prompt when new versions are available when using tagged dependencies -- is that behavior appropriate when tracking a branch?

If possible, the semantics should be parallel to those of open-ended specifiers like “majorVersion: 3”. Whatever updating & pinning mechanisms apply to one should apply to the other.

2. How do we manage the presence of these attributes w.r.t. producing reliable package graphs. For example, should this be allowed in all circumstances, or should it only be allowed when accessing a package via an "untagged" path? If I depend on B, and B updates and pushes a tag to start referencing a branch, do I see an error, or a warning, to let me know my product is no longer being built from tagged repositories?

Emit a warning.

If B has pinned a version, use that version. If this creates a conflict between transitive dependencies pinned to different commits:

• if one commit is the ancestor of the other, emit a warning;

• otherwise, emit an error.

I give these rules thinking of the common case: several packages doing new development in tandem — as in CB4’s Swift 3 example.

Where there are branches running around in the dependency graph, developers should expect some instability and unexpected breakage; however, it’s crucial to be able to detect that breakage without have to cut new versions.

3. We anticipate needing an explicit workflow for development against an untagged group of packages (we've been colloquially calling this "master"-style development). If we had that feature, which let you just use a bunch of packages in whatever state they were in on the local filesystem, would that change how you felt about wanting this in the `Package.swift` manifest?

No.

Again, this should support multiple packages developing new features in tandem — packages that may have different owners.

Cheers,

Paul

···

On Sep 6, 2016, at 1:15 PM, The CB4 via swift-evolution <swift-evolution@swift.org> wrote:
On Sep 6, 2016, at 12:06, Daniel Dunbar via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote: