Swift Package Manager Manifesto

During some discussion in this thread about SPM, it was brought up how there is no manifesto for the Swift Package Manager. This lack of a manifesto leads to some confusion about the direction we should/will take as well as what features are planned or even commonly rejected.

Full Quote

So I'd like to start working on an SPM Manifesto in order to give important information to the community at least about some weak points of SPM, what is currently planned, the priority of the planned items, and items that are commonly rejected. This will hopefully help people know where they contribute to SPM and will help curb the frequent requests for certain items that are either already on the roadmap or have been rejected.

This post contains a list of needed items in the foreseeable future of SPM, but there is no priority associated with any of the items.

Another commonly requested item would be a package index of SPM libraries/programs.

I'm not aware of what other items may have been rejected in the past or even any other items that may have been requested in the past. So I'd like to get some community input about what people want as well as how important it is to see it in SPM in the future. I'll use the discussion sparked from this post to draft a proposal which I will pitch at a later time.

1 Like

Hi Jacob,

While we may not have a clear "manifesto", we do have some pretty good definitions of what kinds of features we think make a lot of sense.

I would suggest if you want to contribute, try picking one of the features that you are strongly interested in and develop it out a little more... for example trying to find pieces that could be turned into concrete proposals for us to discuss. I'd be more than happy to help review proposals or even proposals for proposals.

In my experience, open source projects usually are driven mostly by what contributions people are interested in making, versus up front declared "priorities".

HTH!

  • Daniel
5 Likes

Daniel,

I appreciate your willingness to help me out and I would love to contribute in other ways as well, but I still think a manifesto should be drafted. Many of the pitches/proposals/discussions in other areas of swift will often reference a manifesto as motivationfor why to accept/reject a proposal. The manifesto helps give a clear direction of where a project is trying to go.

Right now it's difficult to even know what SPM wants/needs, because it's basically my own opinion. I linked to a post with some of the known items, but until yesterday I had no idea that post even existed and I'm willing to bet I'm not the only one. A manifesto in the same place as all the others or in the swift-package-manager repo would definitely be a good idea for anyone wondering about where SPM is headed.

I definitely agree that open source contributions are driven by what people are interested in, but they are also driven by a community need for something. A developer may wish to add a feature they want/need, but not have any idea how/where to begin (a frequent problem, especially when dealing with a large codebase like swift/spm). They sometimes choose an item off a todo list that seems more simple in order to familiarize themselves with the codebase. Afterwards, they are better prepared to tackle the larger, more interesting feature that they originally wanted to work on.

At least this is how I often contribute to open source projects, because it can be a daunting task to dive off the deep end into a huge project like swift/spm. Low hanging fruit can be a great way to get started as I work up to the beefier todo items.

  • Jacob
3 Likes

It would be extremely useful for the community to know what those definitions are and in which priority order do the main SPM developers think they should be implemented.

Also, a clear roadmap or something alike would be very interesting to avoid the sense of being a rogue or unmaintained project.

5 Likes

Gotcha, that certainly makes sense.

I'll chat with Ankit and see how we can potentially make this more obvious. I don't think it makes sense to be in the Swift repo, but maybe we should look at taking Ankit's post, trying to roughly prioritize it from the point of view of the (current) primary contributors, and then putting it into the SwiftPM repo.

Does that sound like a monotonic improvement?

3 Likes

That sounds like an excellent improvement and it's basically what I was hoping to get out of this thread. Thank you

I'd also like to point out that the evolution ideas post is already a sort of SwiftPM manifesto or roadmap. The post was created by collecting the ideas that we always wanted to do and the feature requests that I've seen over the past year or so. It also tries to address the topics that frequently come up for the package manager (package index is already listed there). It should give people a good idea of where the project is headed. I plan on expanding that list as I see more feature requests that makes sense. I am also happy to apply suggestions one might have for improving that post.

You make a good point about the discovery problem. We can certainly do things to improve that situation, maybe by linking the post from SwiftPM's repository.

3 things i’d like to see are

  • source-only packages
  • being able to specify linker flags,, right now my swift build invocation looks like this! swift build -c debug -Xswiftc "-D" -Xswiftc "DEBUG" -Xcc -I/usr/local/include -Xlinker -L/usr/local/lib
  • being able to build only some targets instead of all of them
1 Like

I've moved the evolution ideas document to SwiftPM repository: https://github.com/apple/swift-package-manager/blob/master/Documentation/EvolutionIdeas.md

1 Like

So when can we expect real Xcode integration? The current form is to much hassle to use.

The ability to handle resource files is a prerequisite for that I guess.

I would also be interested to think what the team prioritizes. Some items on the wishlist are sugar, while others (like the resources thing) are hard requirements and showstoppers at the moment. It would be nice to get an idea about when those issues will be tackled approximately.

I for one can not comprehend how features like "Dependency mirroring" or "Manifest caching" are approached before showstoppers like "Handle resources". Those are nice to have but not blocking in my opinion. (I can of course only speak for myself and am fully aware that my opinion might not reflect majority)

Much of my frustration with SPM's development stems from this subjective discrepancy.

As Daniel said, we'll talk about what we can do to make the project's immediate priorities more clear to the community. I think everyone absolutely agrees that resources support is a very important feature. A draft proposal is in works and is expected to be posted on the forums for comments in the fall.

I would ask you to look at this from a different perspective. The two things you mentioned are individual contributions. The package manager has no restrictions on what kind of features are allowed to be proposed. If you look at the recent commits and PRs, there has been a lot of different kinds of bug fixes, enhancements, and feature additions by different contributors. It'll be a bit unfair to ask contributors to work on a things that are regarded to be more important than the thing they want to contribute on.

Manifest caching is actually just a quick side-effect of the much larger effort of moving SwiftPM onto library based use of llbuild. This will enable us to build a lot of features and enhancements in the package manager like extensible build tools support, better progress reporting, more parallelization across the project, easily implementable build rules, etc. It will also directly help in the implementation of the resources build rule.

3 Likes

That is why I don't like open source projects. They miss dedicated control of project targets.

Well, I absolutely see your point here and just hope that my comments were not seen as bashing work that is being done. I cannot stress enough that every interaction I had with the people working on SPM was absolutely fabulous and helpful. But I just wanted to point out where my frustration comes from. This is highly subjective and is nobody's fault.

I do get the feeling that the groundwork being done on SPM is magnificent but I also fear it leads down a path that I can only describe as "Always improve the foundation and never getting to the roof" syndrome. I hope that makes sense.

:heart:

I honestly don't think that is a problem of open source in this case but the lack of resources.

@Aciid how many people are working on SPM full time?

1 Like

As with the Swift language, SwiftPM has seen a fair amount of changes in the Swift 3 period. I think its worth spending the time getting the foundation solid now before we rush ahead on some of those bigger features. Moreover, as Ankit said, some of that groundwork like moving to using llbuild as a library is mandatory for implementing some of those higher level features.

That's a good step, thanks!

As a side note: I would like to add to the idea list the option to use package library products from the REPL. I know it is not a priority but it seems simple to implement.

1 Like

It may not be a priority for people who are used to this and have found ways to work around this, but for many newcomers to the language (who come from other programming languages), I suspect that this is quite a deal-breaker, so I agree that this should be done.

2 Likes

Thanks for the suggestions. The evolution idea list is for the features that will require a proposal before they can be implemented. REPL support can be implemented without a proposal. It is tracked by SR-1573.

2 Likes