Swift PM, Bundles and Resources

I completely agree with that concern, and setting bad precedents is definitely a risk with the approach of adding resource support in a staged manner. But for the specific case of packages written for platforms that support asset archives, the idea that the author would still put all the resources in a single asset catalog which would be treated as a resource, compiled into an asset archive, and accessed using APIs that take a bundle as a parameter. The lookup API wouldn't refer to a path but would pass the bundle when instantiating the image.

A trickier topic would be to discuss whether and how to automatically compile loose resources in a package into an asset archive. That's where SwiftPM might need to gain some knowledge of the structure of asset catalogs.

I think it would be prejudicial for SPM to try to handle platform specific resources (a.k.a Darwin/iOS) natively: it should be as platform agnostic as possible.

Now, handling platform specific resources using SPM would be interesting and useful anyway. But other tools such as Gradle resolved this issue by providing a plugin system which would be the best solution IMHO.

As such I think we should only introduce simple and idiomatic resource handling in SPM core.

That makes sense. What that would mean in practice, then, would be that packages that already require one or more of the Darwin-based platforms (because of the APIs they use) would be able to contain .xcassets directories as resources, which SwiftPM would treat as just another resource but IDEs could have more specialized rules for. Packages that don't require a platform that knows about .xcassets directories would not be able to use them.

When you mention a plug-in system, do you mean for SwiftPM itself or for IDEs based on libSwiftPM? AFAIK that isn't really something that has been discussed a lot, and certainly the resource proposal couldn't assume it.

I like the idea of allowing the creation of custom, potentially platform-specific plugins in order to handle different types of resource files (e.g. xcassets). I think Google has a good model for this for their protobuf compiler:

https://www.expobrain.net/2015/09/13/create-a-plugin-for-google-protocol-buffer/

I could imagine the API would be something like:

swift package build . --plugin=asset-compilers XcodeAssetCompiler MyAssetCompiler

To write their own asset compiler, users would write a Swift program which could depend on modules like PackageModel.

We could add a "resources" value to TargetDescription which could take an array of glob strings alongside the "sources" e.g.:

TargetDescription(
    ...
   sources: ["**/*.swift"],
   assets: ["**/*.(png|jpg|xcassets),
   ...
)

Without any plugins, any assets specified would be ignored. When a plugin is provided, the plugin executable would be invoked, passing along the necessary parameters.
Asset compilers could operate on all of the files globbed from the assets specified in the target description, and multiple asset compilers could operate on the same asset file(s).
Xcode could bundle its own asset compiler, but for linux or other platforms engineers could write their own.
I could imagine people writing bespoke compilers for all types of assets, not just images or xcassets. Imagine the possibilities if there was an ecosystem of providers to handle all sorts of resources. Here are just a few:

  • An R.swift style compiler which generates type-safe code to access assets found in the target:
    • R.image.inbox.threads.unread
  • An Apollo GraphQL style compiler which compiles .graphql files into type safe queries:
    • Graph.Query.currentUserQuery.subscribe(onNext: { print($0.firstName) }.disposed(by: self.disposeBag)
  • A mock compiler which compiles .json files into API request stubs.
  • A protobuf compiler for tensorflow models (could invoke protoc under the hood).
  • So many more!

The first version could just be a simple API for creating new asset compilers, and a basic reference implementation. I wouldn't be surprised if within a week of releasing it there were a ton of open source compilers for most common use cases.

That sounds more like extensible build tools (interoperating with resources), which is a separate issue.

1 Like

I like this direction as well, but I agree that this level of extensibility is more in the scope of extensible build tools. There is also a concern that if there isn't at least a core set of types that is known to be handled on all platforms, it will become difficult to write platform-neutral (and IDE-neutral) package code that works everywhere. But it's not a bad idea, I just don't know that we should try to tackle it for the first version.

1 Like

What I did to temporarily workaround it is create a "Resources" folder and create bundles in there. Then I drag those bundles from SPM to my app project > Build Phases > Copy Bundle Resources. I didn't copy it, so it's pointing to it relatively which looks like it goes into DerivedData to get it from the Swift Package. Then my bundle reference looks like this:

public extension Bundle {
    private class TempClassForBundle {}

    static let mySPMBundle: Bundle = {
        // Bundle should be explicitly added to main project until SPM supports resources
        // https://bugs.swift.org/browse/SR-2866
        guard let url = Bundle.main.url(forResource: "MySPMBundle", withExtension: "bundle"),
            let bundle = Bundle(url: url) else {
                return Bundle(for: TempClassForBundle.self)
        }
        
        return bundle
    }()
}

Then I can use it like this: NSLocalizedString("duplicate.failure.error.message", bundle: . mySPMBundle). I'm still trying to get this to work with .xib's, but wanted to share in case any ideas or feedback from here regarding the planned SPM implementation.

Also I'm a bit confused on why .xcassets was suggested to store resources when bundles seems to be designed for bundling resources. Plus need to make sure embedded localizations work which I'm not sure how that would work with assets.

PS - I agree to keep it simple and postpone type-safe resources and extras to get embedded resources working with SPM in at least some capacity, even if there is some manual work involved by the consumer dev. Not being able to embed resources in Swift Packages is a show-stopper for a huge population of frameworks out there.

1 Like

Thanks for describing how you're currently working around the lack of resource support — this is roughly along the lines of what the draft proposal suggests automating (i.e. having the build system create a codeless bundle containing the resources of each module). So this makes a lot of sense.

The suggestion of using asset catalogs (.xcassets directories) on the input side was because they provide a highly structured way to provide multiple variants of resources, e.g. light vs dark images, 1x vs 2x resolution images, high-poly vs low-poly meshes, etc.

Asset catalogs do not themselves provide localization, so there would still need to be a separate asset catalog for each localization; they are orthogonal to localization.

Asset catalogs are also not replacements for bundles. Rather, asset catalogs are compiled to an asset archive that ends up as a file inside a bundle. In my view, bundles are good output artifacts, but I would hope that the way in which resources are specified in the package could be more platform-neutral and less tied to the output format. That would let the build system choose the right output format for each platform; which might be codeless bundles for all currently supported platforms, but might be something different for future platforms that define a different kind of runtime format for resources.

Asset catalogs do support localization in Xcode 11, but there is a known issue in the release notes:

You can now localize assets in asset catalogs. Localization is enabled in the attribute inspector. (12948139)

Localized assets in an asset catalog aren’t matched to the user-preferred languages and locales. (49565973)

For example, the Contents.json for an imageset:

{
  "images" : [
    {
      "idiom" : "universal",
      "filename" : "example.pdf"
    },
    {
      "idiom" : "universal",
      "filename" : "example-de.pdf",
      "locale" : "de"
    },
    {
      "idiom" : "universal",
      "filename" : "example-en.pdf",
      "locale" : "en"
    },
    {
      "idiom" : "universal",
      "filename" : "example-fr.pdf",
      "locale" : "fr"
    }
  ],
  "info" : {
    "version" : 1,
    "author" : "xcode"
  },
  "properties" : {
    "preserves-vector-representation" : true,
    "localizable" : true
  }
}
3 Likes

Thanks for the replies. I took a closer look at the proposal and think it's really powerful. Mapping resources via enum in the manifest is really flexible (btw an associated enum would be really nice here: roles: [.resource("*.dat")]).

Although I did have the same questions you brought up regarding multiple pattern matches conflicting. Would leveraging SwiftPM's existing conventions help simplify implementation while still leave options for the future? For example, how about a convention like this which plays off the existing convention:

MySwiftPackage
|-- Resources
    |-- MySwiftPackage
    |-- MySwiftPackageTests
|-- Sources
    |-- MySwiftPackage
|-- Tests
    |-- MySwiftPackageTests

It's true you can't mix code and resources with this, but think it's a fair trade to get the feature off the ground. It would still be a welcomed update later and an easy "migration" if anyone wishes to mix code and resources at that point (or wouldn't hurt if one decides to opt out).

Regarding the generated bundle name, would it make sense to generate it off the package name? What I mean is since the module name is known, how about import declarations or a typealias such as:

import MySwiftPackage
import bundle MySwiftPackage.bundle //or
typealias MySwiftPackageBundle = MySwiftPackage.bundle

I didn't realize resources fundamentally could not be embedded within static libraries. Is this still true even with the changes that came with SwiftPM? If there needs to be some kind of automation script to copy the bundle during the build process, there is a discussion and an experimental script that may help. In case you missed it here is the link and the script:

#!/usr/bin/swift

// FILE: swift-copy-testresources.sh
// verify swift path with "which -a swift"
// macOS: /usr/bin/swift 
// Ubuntu: /opt/swift/current/usr/bin/swift 
import Foundation

func copyTestResources() {
    let argv = ProcessInfo.processInfo.arguments
    // for i in 0..<argv.count {
    //     print("argv[\(i)] = \(argv[i])")
    // }
    let pwd = argv[argv.count-1]
    print("Executing swift-copy-testresources")
    print("  PWD=\(pwd)")

    let fm = FileManager.default

    let pwdUrl = URL(fileURLWithPath: pwd, isDirectory: true)
    let srcUrl = pwdUrl
        .appendingPathComponent("TestResources", isDirectory: true)
    let buildUrl = pwdUrl
        .appendingPathComponent(".build", isDirectory: true)
    let dstUrl = buildUrl
        .appendingPathComponent("Contents", isDirectory: true)
        .appendingPathComponent("Resources", isDirectory: true)

    do {
        let contents = try fm.contentsOfDirectory(at: srcUrl, includingPropertiesForKeys: [])
        do { try fm.removeItem(at: dstUrl) } catch { }
        try fm.createDirectory(at: dstUrl, withIntermediateDirectories: true)
        for fromUrl in contents {
            try fm.copyItem(
                at: fromUrl, 
                to: dstUrl.appendingPathComponent(fromUrl.lastPathComponent)
            )
        }
    } catch {
        print("  SKIP TestResources not copied. ")
        return
    }

    print("  SUCCESS TestResources copy completed.\n  FROM \(srcUrl)\n  TO \(dstUrl)")
}

copyTestResources()

There was a suggestion in that discussion about adding resources as local dependencies which seems like a clever approach as well.

Thanks again for the replies and proposal.

1 Like

Correct. It is not possible to use separate resource files. Since the package manager does not know about them, it cannot set them up for client packages. And since all the files will be in different places for client packages than they are for your own package, scripting something will be very unreliable.

However, I do know of two ways of getting around it:

  1. Resources that are only for tests. It is reasonable to expect the repository to be around whenever tests are running. Simply use #file to get the absolute path of the current source file, build a relative URL from that and load the related resource file. Problem solved.

  2. If binary size is not a concern. Embed resource files in a source file as base 64 encoded data in a string literal. (A tool of mine, Workspace, can automate this.) Then as long as the source file is checked in, the resources will be embedded in the executable and available no matter how the package is used. This allows you to treat the files as resources during development, but they are technically no longer separate resources after compiling. (The annoying binary bloat is why I watch this thread so closely, hoping for a better future.)

1 Like

As some prior art, Rust has two fantastic macros here, called include_str! and include_bytes! which make it trivial to pull data in from files at compile time.

5 Likes

Wow, I would love to see this kind of art on Swift.

1 Like

can this be used with *.storyboard file ?

If you can work out how to turn a Data instance into a properly loaded storyboard file, then yes.

A cursory look at this StackOverflow answer suggests that storyboards expect to be loaded through bundles. So I’m guessing your executable or library could:

  1. Create a bundle somewhere on disk (a temporary directory? Application Support?)
  2. Write the Data into the bundle as a storyboard file.
  3. Construct the corresponding Bundle instance from the URL to wherever you put it.
  4. Make the Bundle load the storyboard (à la StackOverflow link above).

However:

  • I don’t know how many of the other files need to be provided to the bundle (Info.plist?) so that the bundle will be able to load its storyboards properly.
  • It probably breaks App Store regulations in a dozen different ways.
  • I haven’t tried it, so I may be overlooking something.

I leave it up to you to decide whether the attempt is worth the effort.

But whether you try it or not, pray for a real solution to arrive soon.

To say, I’m curious about the draft proposal that generated extension code to Bundle, which is a hack for Apple’s Xcode Assets Catalog, but not something a package manager should do. And one question: What a package manager should do if I write the same extension to that Bundle ? A compile error ? No, it compiles when use swiftc, the code is generates by swiftPM.

What about just as simple as it can, to declare each Target a resource directory in logic, how to implement the resource access or query is what Xcode team should do. They bridge the logical directory to .bundle .xcassets. On Linux, these logical directory becomes actual file folder. Framework author can query them with different code per platform API.

This apply to the same for localization file lproj. It just a normal file folder on Linux. Only when running on Xcode it becomes the localization resources.

Try to bind the Swift Package Manager with the Foundation API of Bundle looks like not a good idea. Need a language-level package manager cares about a Framework level’s struct implementation ? Isn’t the logic should be inverted ?

Terms of Service

Privacy Policy

Cookie Policy