Swift PM, Bundles and Resources

This is a great read! Thank you for sharing!

I see that by the amount of open questions this will most likely not fit into the Swift 5 timeframe. That's fine I guess, but I think that involving the community rather earlier than later on these open questions would still have been valuable.

Also it would be great to have some sense of what the priorities of features is that the team is working on :slight_smile:

Thanks for sharing, it looks promising!

This year, as with the last few, all I want for WWDC Christmas is you, Swift Package Manager. To use you for iOS development. I'm sick of Cocoapods. It's now WWDC Advent and I'm guessing I'll be disappointed again this year.

Where's this proposal at?

I'd step up and help but I'm a clueless idiot.

For a version one, I don't care if it provides an API for type safe access to resources. I just want to be done with the invasiveness of Cocoapods and all the Ruby garbage I have to deal with.

8 Likes

One more idiot here, and I’d love to finally use SwiftPM for my AppKit projects.

2 Likes

Yes please, especially since having been bit by a nasty little surprise when 10.14.4 and Xcode 10.2 was released.

I had my frameworks integrated in to a macOS app by including the SPM project files. Suddenly though this app did crash on launch on my Mac after I upgraded. It couldn't find the Swift runtime… well, the correct one at least :wink:

Turns out that SPM hardcodes the Xcode-included Swift libraries' path into the @rpath of its frameworks, so when my app launched (it was still compiled with Xcode 10.1 and Swift 4.2 was therefore embedded) it first tried to find the one from Xcode on my system and that didn't end well for the linker :slight_smile:

So a previously working app suddenly crashed because I had the new version of Xcode installed.

Well to be honest: "I was holding it wrong"

When recompiling the app with Swift 5 the problem goes away all by itself, but just in case I added an .xcconfig file for SPM so it would add @loader_path/../Frameworks/. The praise for finding this problem on my end goes to the incredible @graskind. :bowing_man:

Reposting here just in case anyone else runs into this. After all, there's many SPM early adopters here :wink:

I wish that I wouldn't have to use unsupported hacks to use SPM. It is so well built (possibly because the team actually does prioritize the important stuff first) so I really want to SPM all the things. Kudos to @Aciid et al.! So grateful for the amazing work you do!

6 Likes

I was browsing GitHub trending and saw https://github.com/JamitLabs/Accio. Looks interesting!

1 Like

Hello everyone,

as @rudedogg already mentioned Accio, I'd like to point out that I just wrote an article about it which explains it's motivation, rationale and design in more detail. Basic usage instructions are given, too.

Please check it out:

I hope you find it useful!

2 Likes

After introducing Package support in Xcode, this feature is so much needed. What is a current plan with supporting bundle targets and resources in SwiftPM? Personally, I would be okay with any format, even if it’s not super-flexible or future-proof.

There's already a proposal draft by @abertelrud which was also mentioned in this WWDC session. They specifically asked for community feedback on the proposal and implied that Apple would do any work needed to update the Xcode integration once SwiftPM implemented support for resources.

So, it is now really time to review this properly:

6 Likes

SwiftPM is currently wrapping the Swift 5.1 release and there will likely be motion on the resources proposal after July.

6 Likes

I read the linked proposal draft, and not knowing where else to send my feedback, I will do it here.

Historically, macOS and iOS resources have been accessed primarily by name, using untyped Bundle APIs such as path(forResource:ofType:) that have assumed that each resource is stored as a separate file. [...] In the long run, we would like to do better. Because SwiftPM knows the names and types of resource files, it should be able provide type-safe access to those resources by, for example, generating the right declarations that the package code could then reference. [...] In the short term, however, we want to keep things simple and allow existing code to work without major modifications.

I am of the opinion that individual synthesized accessors should not be deferred. The typing isn’t the issue—we can defer that by making everything unspecific Data instances for now. I would rather never provide access to dynamic string identifiers or paths.

If an API like path(forResource: String, ofType: String) is provided, then the package manager and compiler can never have enough information to know which resources are actually being used, and that will prevent future optimization opportunities. The package could do this:

public func cardImage(suit: Suit, rank: Rank) -> Image {
    let name = suit.identifier + rank.identifier
    let file = Bundle.moduleResources.path(forResource: name)
    return Image(from: file)
}

With this ability, the package manager must naively bundle all available resources—not just the 52 card images, but also the cribbage board and peg images.

A client package might be making a solitaire game instead, but want to reuse the 52 card logic, so it links and calls cardImage(suit:rank:). But now they come bundled with board and peg images it does not want and will never use. The dead code contains dead resources that the optimizer has no means to identify.

I think it would be better never to provide dynamic identifiers like that, so that we keep the door open for better automatic optimization in the future.

An API where each resource is a direct property is much more future‐proof:

public func cardImage(suit: Suit, rank: Rank) -> Image {
    let data: Data
    switch suit {
    case .hearts:
        switch rank {
            case .ace:
                  data = Bundle.moduleResources.aceHearts
            case .two:
                  data = Bundle.moduleResources.twoHearts
            // ...
        }
        // ...
    }
    return Image(from: data)
}

Since every access is forced through the same property, when that property is optimized away, the optimizer knows it can remove the resource it represents too. The client only calls cardImage(suit:rank:), and never anything that uses moduleResources.cribbageBoard? Then the client needs AceHearts.png (and co.), but not Cribbage Board.png.

(I believe the vast majority of resource access likely directly loads individual and specific resources, and will not suffer the boilerplate bloat demonstrated by the above function.)

1 Like

Hi there!

Exactly like @SDGGiesbrecht, I read the mentioned proposal and wanted to leave some comments.

I wanted to highlight how we currently manage resources in our iOS project. We use Cocoapods to manage our dependencies and had to do some preparations so that we are able to statically link our dependencies which include local development pods (i.e. our feature modules). :smile:

As described in the initial post, copying all resources into the main bundle can lead to name collisions and run time crashes. For our local development pods, we therefore moved from using .resources in our podspecs to using resource_bundle. Cocoapods then creates a new target that creates a resource bundle that is then linked into the module's code bundle.

s.resource_bundles = { 'ModuleResources' => ['Module/**/*.xib', 'Module/**/*.xcassets', 'Module/**/*.json', 'Module/**/*.png'] }

Similar to the approach described in the proposal, we use an internal static property in an extension to handle resource bundle resolution per module.

static var resourceBundleName: String = "ModuleResources"

 static var resourceBundle: Bundle {
        let codeBundle = Bundle(for: Module.self)
        guard let subBundleUrl = codeBundle.url(forResource: Module.resourceBundleName, withExtension: "bundle"),
            let resourceBundle = Bundle(url: subBundleUrl) else {
                fatalError("misconfigured module")
        }
        return resourceBundle
    }

Loading images can then be achieved by using:

UIImage(named: name, in: Bundle.resourceBundle, compatibleWith: nil)

As described in the proposal, this approach has the downside of using Bundle(for: ...) and assumes that the resource bundles are inside the same bundle as the code. Additionally, it is not platform-agnostic.

I think @SDGGiesbrecht 's idea to expose resources as Data properties directly on the moduleResources bundle is great and would also eradicate bugs in which the resource name is not equal to the name used in code. Additionally, we could use this to strip out unused assets as part of the build process. It also reminds me of the way resources are accessed in Android.

The only question that then comes to my mind is how we would actually handle file name collisions inside one resource bundle. Image having a mock.json in two separate folders, both are valid files in their respective folders, but the lookup via path(forResource: name) would fail. Also, imagine having two files in different formats but using the same filename (i.e. mock.json and mock.png). Would we then include the file extension in the property name to resolve the name collision?

1 Like

Good observations!

I imagined the files would be automatically namespaced in the product layout since the generate access code is internal, there are no filename collisions between modules. Namespacing could be either with subdirectories or by automatically prefixing the file names themselves. Note that I understand implementation details like this have room for variation by platform or product type. All that needs to be stable is (a) where/how you put/declare the resources in the package repository and (b) the Swift interface generated for their retrieval.

Resources
↳ MyPackage
  ↳ MyModule
    ↳ en
      ↳ mock.json
    ↳ de
      ↳ mock.json
↳ DependencyPackage
  ↳ DependencyModule
    ↳ mock.json

This affects more than just file extensions. Swift identifiers have a character set significantly restricted compared to what filenames have at their disposal, so it is necessarily either a lossy conversion or the result is super ugly. This is the algorithm I currently use to Swift‐ify filenames, but it would not have to be that elaborate. An ultra‐simple solution would be to simply disallow filenames that are not already valid Swift identifiers so as not to have to clean them up at all. It think it is reasonable to simply throw a compiler error if a single module has multiple resources named in a way that ends up clashing. The module author can simply rename the files.

2 Likes

I agree that keeping the access code internal would solve these problems and that was my assumption. I actually meant resource name collisions in the same module. Namespacing would be an option which we could look into and maybe also define as part of the Package.swift file. Similar to Cocoapods that allows to define multiple resource bundles instead of just one, we could use this to namespace the resources.

let package = Package(
    name: "MyInfoPanel",
    targets: [
        .target(
            name: "MyInfoPanel",
            dependencies: ["IconWidget"],
            resources: [
                .bundle(name: "en", content: [
                    "en/*.json": .resource
                ]),
                .bundle(name: "en", content: [
                    "de/*.json": .resource
                ])
            ]
        ),
        .target(
            name: "IconWidget",
            exclude: ["Border.png"]
        ),
        .testTarget(
            name: "MyInfoPanelTests",
            dependencies: ["MyInfoPanel", "MyTextFixtures"]
        ),
        .testTarget(
            name: "MyTestFixtures",
        ),
    ]
)

This would resolve name collisions and leave the flexibility to the package owner to decide how they want to structure their package resources.

We could extend the idea above to include resolve dependencies into subnamespaces (i.e. image, resource, ...). In the roles, we could define a set of possible roles that a resource can take. Resources could then be accessed via this subnamespace. Potentially, we could also return the resource to the representing type (i.e. UIImage for iOS, etc.) based on the context it is used in.

// in Package.swift
.target(
    name: "MyInfoPanel",
    dependencies: ["IconWidget"],
    resources: [
        .bundle(name: "en", content: [
            "en/*.json": .resource
        ]),
        .bundle(name: "en", content: [
            "de/*.json": .resource
        ]),
        .include(content: [
            "*.png": .image
        ])
    ]
),
...
// in the package
let image = Bundle.moduleResources.images.aceHearts // -> UIImage, ...
let image = Bundle.moduleResources.images.aceHearts.raw // -> Data

Sure, I can see how those customization could come in useful at times. I do suspect that most projects wouldn’t be complicated enough to need the extra specificity. I would suggest treating it like a target’s path or sources—customizable, but for the most part just left to an implicit default.

(In case I was unclear, the en and de directories represented localizations of the same files. I would expect the same call to resources.mock to return the contents of one or the other depending on the localization active at runtime. Localization does not need to be actually supported in the first iteration, but it does need to be a consideration so that we don’t box ourselves out by our name‐spacing design. The only kind of name clash I was demonstrating there was across separate modules. I was leaving the problem of same‐module clashes for the following paragraph as a separate issue.)

Since the feedback process on the 2018 "Package Resources" proposal draft has already started here, let me mention two requirements I'd expect from resources support in SwiftPM which I both don't see covered by the draft:

  • allow type-safe access with resource availability checks by the compiler, preventing dynamic strings (removing the need for tools like SwiftGen entirely)
  • allow folder-based types for different contexts, e.g. to make initializing different UIImage resources from a .xcassets folder or a String from a directory containing .lproj folders possible

I've actually just pitched an alternative (incomplete) proposal draft which I plan to continuously work on:

I'd love to hear your feedback on that, too!

5 Likes

Any update on this?

I think the biggest underlying issue is that modern operating systems can have quite complex resource abstractions, and there is some reluctance to model that in SwiftPM (for good reason!)

The old idea of grabbing a file by a path is doable, of course, but is much, much less than ideal in today’s world. It basically is only good for raw data, like JSON files or canned databases. Not for UI stuff. Even then, they couldn’t take advantage of things like on-demand resources. It would be very, very basic. So I can see why it’s not a huge priority.

Darwin platforms support all kinds of UI-related resource variants - from device type, screen scale and dark mode to things like which version of Metal you’re running on. And then there are localised and the aforementioned on-demand resources. And what about Linux? And Windows?

https://developer.apple.com/library/archive/documentation/Xcode/Reference/xcode_ref-Asset_Catalog_Format/ImageSetType.html#//apple_ref/doc/uid/TP40015170-CH25-SW1

What if we start with simple resource packaging like CocoaPods provides and worry about integrating into platform and other APIs later? The lack of this feature is major drawback of SPM at the moment. It seems like providing the simplest version first would be better than waiting to nail down a single integrated abstraction for every platform.

3 Likes

Everything is still contained in a file/folder, referenceable by a filename, even if it is in an asset catalog. Unless you're thinking the package manifest has to directly address all those items you linked to in your post that are contained in the asset catalog Contents.json file.

1 Like