Use of SwiftPM for internal modules?

Using Swift
1

How can I create and add a Swift Package to my app, where the source folder for the package is a subfolder within the same repo? How can I add it to my app in a way that is unversioned? (I.e. we only have the app's version; we want to use Swift Packages purely to define modules but don't need any of SPM's "dependency management" trappings.)

Here's some background if it matters:

My org's app is broken about 40 frameworks that comprise various features and shared code. Recently we migrated to a monorepo, so none of our own projects rely on any kind of per-framework versioning or dependency management system anymore.

Still though, having 40 frameworks in the workspace means lots of Xcode project files and so, refactors than move files around for example can cause nasty merge conflicts in the .pbproj files. And when someone makes a new project there is always some setting in the project that gets set wrong and we have build problems.

So now that project resources can be included to a Swift Package as of Swift 5.3, we are beginning to consider if it makes sense to migrate away from using frameworks and use Swift Packages for everything instead.

However when I was trying to convert one framework into a Swift Package, it seems that Swift Packages are really only intended for a situation where the Swift Package comes from its own git repo (and not just some local subfolder in the same repo as the app).

As well it seems like the "dependency management" aspect is rather baked into the concept of a Swift Package—it seems to expect that rebuilding only needs to happen if the package version changed.

But we don't want these packages to be versioned, and we don't want them to exist in another repo or for SPM/Xcode to look for them anywhere on a network. We just want to use them as modules, that's it.

How can we?

2 Likes
2

If you create another directory and put the Package.swift and all of its source files in there you can drag that into your app. You will need to manually link the libraries when you do it this way. It has worked well overall for my current project.

3

I do this all the time for small tools and utilities I spin up. Swift Packages make it really easy to declare separate modules for things. You would have a root Package.swift manifest that can reference other packages by relative file path instead of a git repository URL. Those subpackages can be in the same repository (or completely outside of source control if you want).

In fact, you could merely have a single root Package.swift file and just declare separate modules in there and a single root product that depends on them. You don't even need to have a tree of local packages to get this up and running quickly.

With the latest versions of Xcode, if you double click on the Package.swift file, it will open up as a project you can manipulate.

2 Likes
4

Wow, thanks for the prompt replies! You guys are awesome.

I will try this. Maybe what was confusing me was Xcode's UI for adding packages.

5

Maybe what was confusing me was Xcode's UI for adding packages.

You’re not the first to say that.

  1. As @Braden_Scothern said, when you want to use a local unversioned package, simply drag its directory into the file list on the left side in your client Xcode project. (Remember to drag the directory containing Package.swift, not the Package.swift file itself. That trips a lot of people.)
  2. Xcode will ask to create a workspace if you haven’t already. (Edit: Only if you drop it in the global scope.)
  3. After that, you’ll be able to add any of the package libraries to any target with the “+” under “Frameworks, Libraries, and Embedded Content”.

That’s all there is to it.

2 Likes
6

It's not necessary to create a workspace, local packages can also be referenced from projects.

3 Likes
7

Trying it now, it appears you are right and it depends on where precisely in the list your drag ends. I guess it never occurred to me to drop it anywhere besides the very top or bottom of the list, so I always got the prompt to create a workspace.

2 Likes
8

Yah, I think it is a little finicky and unless you know both are possible it's easy to miss. I'm going to think about how this could be made more obvious.

4 Likes
9

When you realize that you can add local packages similar to nested project files it is easy. I think Apple could clear this up by adding a local package option in the “Swift Packages” project settings since I think that is what someone would try first intuitively. It is unintuitive that there are two completely different workflows for adding packages based on local or remote. Once you get it, it works great. You can even run tests as usual.

2 Likes
10

This has been working great so far.

Only issue we have is that if we include a package's test target in a different scheme with other test targets, then @testable import Package never works from inside its own tests. No idea why not.

11

I tried to use local packages to organize our codebase, but found a limitation that it is not possible to mix local and remote dependencies (versionBasedDependencyContainsUnversionedDependency). It makes it impossible to use SPM for internal modules and expose something outside with "umbrella" Package.swift. Can someone comment if this is something fundamental or not?

12

This is a limitation that I would like to see us lift in the future. As far as I know, it is just policy.

2 Likes
13

I relation to using SPM or nested Xcode Projects. It seems the SPM is evolving to a preferred approach. What are the benefits of sticking with Xcode projects now?

14

It is not yet possible to run a custom build tool as a build phase in a Swift package.
However, there is an accepted proposal SE-0303 for that being worked on!

15

There are a ton of bugs around using Swift Packages (SPs) to declare internal modules and Apple is extremely slow at addressing them. Apple does not even try to test this use case or listen to feedback on how horrible of an experience it is.

Trust me if your team does not want headaches, don't do it. If you do it anyway then you will find out why I said it's a headache.

If your app is Swift-only and ALL your internal modules are declared as packages and NONE of them need to import a prebuilt binary from a folder on disk and you don't need a way to remotely cache third-party dependencies and you won't be using Carthage or Cocoapods and you have a skilled developer who can dedictate themselves to resolving weird SP issues and you are OK with using ugly hacks to solve problems and you don't mind waiting for years to get bugfixes then maybe you'll be fine, but heed my warning, it won't be pretty.

2 Likes
16

@vitamin your post is very much wrong and full of hyperbole.

We have fully embraced SwiftPM for internal app modules when SwiftPM was first added to Xcode and it works great. You can still use non SwiftPM dependencies you just have to inject the dependency over into your package modules.

17

Although @vitamin's post is maybe overly aggressive, I have to agree with him. We use Swift packages for internal modules and also have some external dependencies. We don't use anything else for dependencies and still we have multiple issues:

  • Xcode 13 crashes when resolving Swift Packages and building the project is not possible at all in Xcode 13 even with clearing everything and trying other workarounds (it builds in Xcode 12.5)
  • In Xcode 12.5 building only works after two tries (it doesn't work with a clean build directory) and even then sometime it randomly doesn't build
  • This all although we already implemented weird workarounds for Swift Packages with binary dependencies
  • When switching branches, Xcode needs to be restarted and the SPM cache needs to be cleared, otherwise it won't build

All these issues are discussed in depth in other threads in this forum and multiple bug reports / radars have been filed but they are not fixed in Xcode 13.0.

1 Like
18

Oh it for sure has issues and can be greatly improved. I more than often than not also have to restart it when swapping branches. I haven’t run into not being able to build issues for a long time with it, that said I know a lot of people do experience that issue.

19

Please tell me what facts in that post were wrong or hyperbolic and I'll apologize. I'm just speaking from personal experience. Our enterprise app has a ton of local packages now, but it's been a regular source of build problems that have required ugly hacks to work around.

For example, Xcode 12.5 added a "feature" where if you don't specify ".dynamic" or ".static" then Xcode will decide which one to use automatically. This was supposed to be a solution for the bug where if multiple targets in the same package depend on the same other target in the same package, you get "unexpected duplicate" errors because Xcode will only use static linking between targets in the same package even if they're each declared as dynamic.

However not only did removing explicit ".dynamic" not fix that duplicate target issue, it made it far worse by removing your ability to specify whether to embed a dynamic Swift library product in frameworks that link to it, resulting in App Store submission failures due to many copies of the same package automatically being embedded everywhere.

So we've had to implement a build script that goes back through the build folder and cleans out all these package frameworks. What the actual F.

To make matters worse, dynamic library framework bundles generated from Swift packages don't get a "Headers" folder like normal frameworks made from Xcode projects, even if they contain @objc Swift code. That has led to a situation where importing a Swift package in an Xcode project framework's Obj. C headers always breaks the build. This is frankly an inexcusable bug that should have never made it to prod had Apple used TDD to make sure package frameworks aren't malformed.

Another issue is that these auto-created package frameworks have bizarre names like NetworkStuff_157B50DE_PackageFramework.framework, and they always get put in a subfolder of the normal built products directory called PackageFrameworks. And there's no way to override this. Why? This is a minor annoyance, granted, but if you don't realize it and don't add special framework search paths to account for this, then you might spend quite some time trying to figure out why your Xcode projects cannot import Swift packages even when you link everything correctly.

And that's just the tip of the iceberg of what we've had to deal with since starting to use packages for local modules.

Should I also tell you about how if you use an unsafe flag in a local package to hack a solution to a problem like packages' inability to be linked to non-system binaries, now you cannot link that package to an Xcode project at all? This is clearly a bug, because "unsafe" is only meant to protect you from a foreign package being able to do bad things on your computer (since a package manifest is actual code that gets run when Xcode opens)—but "unsafe" was not supposed to affect what you can do with local packages. Yet another bug with Xcode's SPM implementation that should never have reached prod had the Xcode team used TDD to make sure that using a local package would be functionally equivalent to using an Xcode project to accomplish the same ends.

Or should I tell you about the intentional limitations of Swift packages, like their inability to handle mixed-source targets? Their lack of support for customized build configuration names? (Woe be unto you if you use any build configuration not named "Debug" or "Release.")

Or perhaps you want me to explain the horrific way that asset catalogs were implemented for SPM where other modules cannot see the bundle or catalog? Compounded with the other bugs mentioned above this makes local Swift packages useless for managing assets in mixed-source targets because you will wind up with two copies of the same catalog, one for the package that serves them to Swift and one for the packahe that serves them to Obj. C.

Maybe there is some other hack workaround for the assets issue, but that's not the point—the point is it doesn't just work. So you wind up wasting a lot of time.

So have we. Maybe for some small or medium projects, especially if they're all Swift, and especially if you are also using SPM to manage all your third-party dependencies, it might not be so bad, especially if you have bandwidth to implement a DI scheme like the one you mention. I love DI so I'm not gonna say that's an invalid approach.

But if, like many codebases:

  • you still have a lot of Obj. C code
  • you use shared .xcconfig files to avoid having to re-specify the same build options everywhere
  • you don't like writing hacky build scripts to make your app submittable
  • you have any Xcode configurations with custom names
  • you use .xcconfig files
  • you still use CocoaPods or Carthage because that's what works with your CI pipeline's dependency cacheing solution and you don't want your workspace to take 30 minutes to open for anyone who hasn't opened it before
  • you want the flexibility to be able to override any build setting without referring to LLVM's source code repo
  • you don't have time to be a beta tester for buggy software that receives pretty low priority by its developer relative to their other concerns

... then I stand by my recommendation that people should not bother with using Swift Packages for local modules.

Xcode projects give you a lot more flexibility and control, they have a nice UI that lets you edit ALL settings, they support an unlimited number of customizable build configurations (not just "Debug" and "Release" like SPM) they and they just work.

If this comes off as "aggressive" that's your opinion, not mine. I'm not trying to attack anything or anyone. I'm just trying to do someone a favor by hopefully saving them the frustration that I personally went through.

If you're someone who works at Apple or works on SPM, I'm not saying SPM is bad. I just don't think SPM was designed for the local module use-case except for local development of a distributed package. SPM/Xcode integration is also not built around the use case of Swift packages as local modules. This aspect still needs a lot of work.

I hope Apple prioritizes it more. I would like to see them resolve to fully replace Xcode projects with SPM, but that doesn't seem to be where they are headed—Xcode 13 just intro'd a new Xcode project format. Why do that if you're going to get rid of them?

2 Likes
20

This is false. You can mix local and remote packages as much as you want. You are, however, limited in how you can mix versioned and unversioned packages. Here, "versioned" means that the package is a git repository with version tags. local/remote and versioned/unversioned are separate concepts. You could have any combination of these.

A local package can always be put under git source control. You don't necessarily need a remote. You can reference a local package using a file system path instead. Pay attention to the error message:

version Based Dependency Contains Unversioned Dependency