Draft Proposal: Package Resources

[sergiocampama]

The proposal mentions that the processed/copied resource bundles will be placed beside the final executable, and some comments here make it seem like the bundle name/path will be an implementation detail that only the generated SPMResources struct will know how to read from. Is that accurate?

My concern here is with Linux distributables. If you wanted to build, say, a Swift server which contains resources, how would a developer be able to package the executable and resource bundles into an archive that can be copied to the deployment machine, given that the name of the resource bundles are not known?

IIRC, executable files for a Swift package are placed within the .release/<platform>/release, meaning that if you had multiple executables with resource bundles, it would not be trivial to find which resource bundle maps to which executable (and depending on the naming mechanism, you might even have collisions).

[jechris (JC)]

Overall +1, espcially bout spmResources struct (although I would change its name as other suggested).

One note though:

@Aciid

SwiftPM will automatially detect the matching files
in a target and report them in the package model data structure. Similar to
source files, package authors will not need to explicitly declare these files in
the package manifest

IMHO it feels too magicaland might be unclear to developers when mandatory to declare their resources. I'd go for explicitly declaring resources no matter what and I don't see why it would require to harcode anything in spm?

[SDGGiesbrecht (Jeremy David Giesbrecht)]

Several people have mentioned a worry about the amount of extra code that would be necessary to support both SwiftPM and another package manager. Basically this is all it is:

#if SWIFT_PACKAGE // ← Or some custom condition passed from the manifest.
let thisTargetResourceBundle = SPMResources.bundle
#else
class BundleIdentifier {}
let thisTargetResourceBundle = Bundle(for: BundleIdentifier.self)
// ↑ The last two lines have always been necessary anyway.
#endif

And then it is used everywhere like this:

let image = UIImage(
    name: "MyIcon",
    in: thisTargetResourceBundle, // ← Voilà.
    compatibleWith: UITraitCollection(userInterfaceStyle: .dark))

Note that Bundle(for: BundleIdentifier.self) is not viable for every kind of product SwiftPM supports, so attempting to use the same strategy as other package managers just to save 4 lines of code per target would be incredibly restricting. (Right now I think it would be possible with exactly none of the existing product types; executable, dynamic library, and static library products are all not bundles. And of course a product can contain multiple targets...)

[SDGGiesbrecht (Jeremy David Giesbrecht)]

@sergiocampama

My concern here is with Linux distributables. If you wanted to build, say, a Swift server which contains resources, how would a developer be able to package the executable and resource bundles into an archive that can be copied to the deployment machine, given that the name of the resource bundles are not known?

Right now it is relatively easy to filter out the unwanted intermediate files, leaving only the actual products left in the product directory. Unless a resource bundle can somehow pass for one of those build intermediates, the same strategy should continue to work.

I would eventually like to see SwiftPM be capable of something like this:

build --all-products --configuration release --install-directory 'somewhere'

Where:

• --all-products is shorthand for building every product, but no internal targets.
• --install-directory is where SwiftPM should then copy all the actual product files, but none of the intermediates.

Then you could simply archive the directory passed to --install-directory and you would automatically have all the executables, dynamic libraries, resources, etc. collected neatly into one place for you.

If this were implemented, the problem of identifying the produced resource bundles would be moot.

(Such functionality is separate from this proposal though.)

[ahti (Lukas Stabe 🙃)]

@sergiocampama

My concern here is with Linux distributables. If you wanted to build, say, a Swift server which contains resources, how would a developer be able to package the executable and resource bundles into an archive that can be copied to the deployment machine, given that the name of the resource bundles are not known?

@SDGGiesbrecht

Then you could simply archive the directory passed to --install-directory and you would automatically have all the executables, dynamic libraries, resources, etc. collected neatly into one place for you.

Fwiw, to be good linux citizens, there would need to be be support for having executables, shared objects and resources all in different directories (/usr/bin, /usr/lib and a subfolder of /usr/share, in most cases).

[ahti (Lukas Stabe 🙃)]

@Aciid

Furthermore, the processing of some types of resources might result in the
generation of more source code, which needs to be compiled into the same module
as the code that needs the resource.

So would this feature, with custom rules, then also eventually basically replace the old extensible build tools proposal? Package Manager Extensible Build Tools

Small formatting nit: The proposal text is barely readable on mobile here in the forums due to the forced line breaks.

[SDGGiesbrecht (Jeremy David Giesbrecht)]

@ahti

Fwiw, to be good linux citizens, there would need to be be support for having executables, shared objects and resources all in different directories ( /usr/bin , /usr/lib and a subfolder of /usr/share , in most cases).

I would be interested to know if SwiftPM’s linker does things in a way that supports that. But we were mostly talking about the ability to collect all the products’ pieces in order to archive and distribute them, not about how to properly install that archive onto a platform.

(To be honest though, I’m not convinced anything but a symlink for the topmost product should actually go into /usr/bin. Two separate packages might have conflicting dependency graphs. Put them in separate directories and symlink them, no problems. But spill their dependencies into those global directories and the packages break each other.)

[ahti (Lukas Stabe 🙃)]

@SDGGiesbrecht

But we were mostly talking about the ability to collect all the products’ pieces in order to archive and distribute them, not about how to properly install that archive onto a platform.

Well yes, but what I'm saying is the mechanism that locates the resource bundle at run time should be configurable (at build time), so that the bundle can then be placed in /usr/share or wherever it is most appropriate by the person packaging the software, and correctly discovered at run time by the generated SPMResources struct.

[kiel (Kiel Gillard)]

I'm confused about the localisation support. As an iOS developer, will I be able to use NSLocalizedString and localized storyboards from an imported package?

[tonyarnold]

A big +1 from me, with the (already raised) exception of the name SPMResources (including the use of plurality here in the name). Resource reads nicer - also, we have basic module name spacing, right?

[gwendal.roue (Gwendal Roué)]

@kiel

I'm confused about the localisation support. As an iOS developer, will I be able to use NSLocalizedString and localized storyboards from an imported package?

Not NSLocalizedString(_:comment:) directly, because this function uses the main bundle. But NSLocalizedString(_:tableName:bundle:value:comment:) is what your want: just feed it with the package bundle provided by the proposal.

For localized storyboards, I let other knowledgable people answer your question.

[Karl]

So, firstly, I'm very glad this is coming up. It's a really important thing to tackle.

@Aciid

Add two new rules "copy" and "process" to existing list of rules. At this time,
there will be no additional built-in file type that use these two new rules.

I think it's important to point out any processing of resources outside of SwiftPM and the compiler toolchain will break cross-compilation. It's unfortunate, but unavoidable - target platforms may decide that resources need to be arbitrarily processed and there may not be tools to do that on every host (e.g. there is no public tool to create Darwin asset catalogues on Linux/Windows).

---

@Aciid

Vend an API in libSwiftPM to allow its clients to register additional file types
supported by those clients. SwiftPM will automatially detect the matching files
in a target and report them in the package model data structure. Similar to
source files, package authors will not need to explicitly declare these files in
the package manifest. This ability is useful for clients like Xcode to support
platform-specific file types (such as .metal ) without having to bake in
a hardcoded list in SwiftPM’s codebase.

This is an interesting design - by making this an API in libSwiftPM rather than using a 'plug-in' system, only builds which are driven by Xcode will know anything about how to handle these resources. I consider this to be a usability regression, especially for utility applications distributed via SwiftPM (which I think we consider to be an important/interesting use-case, since we have the swift run command). Such utility applications should be allowed to have GUIs and localised resources, IMO, even if you build & run them from the terminal.

So in general, I think that we should strive for the following commands to be equivalent:

• swift package generate-xcodeproj && xcodebuild
• swift build

(of course they're not, even today - Xcode uses different build paths and will ignore any dependencies already checked-out or built in the .build folder. I find that extremely annoying and would love it if Xcode would detect existing SPM build folders. I don't want Xcode getting up in my builds; building my package is SPM's job)

---

@Aciid

// Load an image that can be in an asset archive in a bundle. 
let image = UIImage(
  named: "MyIcon",
  in: SPMResources.bundle,
  compatibleWith: UITraitCollection(userInterfaceStyle: .dark)
)

(Reformatted to avoid horizontal scrolling)

What is missing from this document is any mention of how I specify that a particular image is intended for the "dark" user-interface style. I'm guessing we will use JSON-bundles like Xcode's xcassets uses:

- MyApp/
 |- Sources/
   |- MyButton.swift
   |- MyButton.imageset/
     |- Contents.json
     |- MyButton-2x.png
     |- MyButton-3x.png
     |- MyButton-dark-2x.png
     |- MyButton-dark-3x.png

... and Xcode will detect the .imageset extension (rather than the PNGs and JSON files inside), and process the manifest like it does for asset catalogues.

OK, so now let's imagine that Windows introduces it's own, similar thing - a bunch of images and a manifest which maps those images to environmental parameters. Let's call it a .win_imageset. Many of those images will be shared between both platforms, so will I need two copies of each? Or will I be able to store all images in one central location and share them among both platforms? Ideally, I would want to organise the resources something like this:

- MyApp/
 |- Sources/
   |- MyButton.swift
   |- MyButtonImages.swiftpm_ignored // <- All images in this folder, ignored by SPM.
     |- MyButton-2x.png
     |- MyButton-3x.png
     |- MyButton-dark-2x.png
     |- MyButton-dark-3x.png
     |- MyButton-someWindowsSpecificThing.png
   |- MyButtonImages.imageset
     |- Contents.json // <- maps Darwin environment params to images
   |- MyButtonImages.win_imageset
     |- Contents.xml // <- maps Windows environment params to images

Then, when a designer comes by and wants to change the image, they can see the entire set of images which are in use, across all platforms. Maintaining collections of symlinks would be annoying, and I think they're not entirely reliable on all systems and filesystems. I'm not sure if there's a better solution - maybe it depends on the processing tools (like Xcode's asset catalogue compiler) accepting relative paths outside of the .imageset folder.

---

I think that's all I've got for now.

[Aciid (Ankit Aggarwal)]

@thomasvl

The one downside would seem to be if you need to copy something, you also have to go to explicitly listing the process things also; but maybe that is the best as it is more clear.

I think we shouldn't require explicit declaration for the registered file types because you could have resources spread across your project and then you'll have to redeclare everything as soon as you need to explicitly copy something. So this would make the behavior a little different than sources. You would still be able to ignore files using exclude.

@thomasvl

Which then leads me to this; when explicitly using resource , you can specify paths for process and copy ; how exactly are they handled?

• For copy if the path is a file, then I'm guessing it is copied as is into a the bundle? But what if the path is a directory? Is that directory copied or are the items within it copied?
• For processed the same holds, what if it points to a file (or a package (.storyboard, etc.) vs. if it points to a directory?

i.e. - are these like sources where you should list the explicit files/bundles to be processed or are they sorta like path where you can point a directory of things to collect so it needs less updating as things are added/removed with time?

See the doc comments attached to the APIs in this section. The behavior for directories is also called out explicitly in the same section:

Packages will also be able to explicitly declare the resource files using the two new APIs in PackageDescription . These APIs have some interesting behavior when the given path is a directory:

• The .copy API allows copying directories as-is which is useful if a package author wants to retain the directory structure.
• The .process API allows applying the rule recursively to files inside a directory. This is useful for packages that have all of its resources contained in directories for organization.

@thomasvl

Are all resources types supported? Some subset?

SwiftPM will not have any builtin file types for resource rules so only copy rule will be supported right now.

[Aciid (Ankit Aggarwal)]

@beccadax

My other suggestion: There should be a built-in .ignore rule to go with .copy and .process , and dotfiles should probably be ignored by default.

There's an API to exclude files from a target. That would apply for resource files as well.

[thomasvl (Thomas Van Lenten)]

@Aciid

The one downside would seem to be if you need to copy something, you also have to go to explicitly listing the process things also; but maybe that is the best as it is more clear.

I think we shouldn't require explicit declaration for the registered file types because you could have resources spread across your project and then you'll have to redeclare everything as soon as you need to explicitly copy something. So this would make the behavior a little different than sources . You would still be able to ignore files using exclude .

So things auto found in sources would then have to get filtered to remove anything that was covered by an explicit resources entry?

@Aciid

Are all resources types supported? Some subset?

SwiftPM will not have any builtin file types for resource rules so only copy rule will be supported right now.

Does that mean a SwiftPM command line build on macOS can't do asset catalogs and that would only work if the package was imported into a Xcode project?

[Aciid (Ankit Aggarwal)]

@sergiocampama

The proposal mentions that the processed/copied resource bundles will be placed beside the final executable, and some comments here make it seem like the bundle name/path will be an implementation detail that only the generated SPMResources struct will know how to read from. Is that accurate?

Correct.

@sergiocampama

My concern here is with Linux distributables. If you wanted to build, say, a Swift server which contains resources, how would a developer be able to package the executable and resource bundles into an archive that can be copied to the deployment machine, given that the name of the resource bundles are not known?

IIRC, executable files for a Swift package are placed within the .release/<platform>/release , meaning that if you had multiple executables with resource bundles, it would not be trivial to find which resource bundle maps to which executable (and depending on the naming mechanism, you might even have collisions).

I think this will be covered by a "deployment feature" whenever we have it but in the meantime we can expose some command-line interface to return the bundle path for an executable from the bin directory.

[Aciid (Ankit Aggarwal)]

@ahti

the mechanism that locates the resource bundle at run time should be configurable (at build time), so that the bundle can then be placed in /usr/share or wherever it is most appropriate by the person packaging the software, and correctly discovered at run time by the generated SPMResources struct.

Maybe we can add some flag like --resources-prefix which embeds additional search paths. This would be similar to linker's rpath mechanism.

[Aciid (Ankit Aggarwal)]

The goal of this proposal is to allow Swift packages to declare and consume resource files that are supported in a cross-platform manner using Foundation APIs. What you're suggesting should be feasible once we have extensible build tools and APIs that allow you conditionally add files based on the platform.

[Aciid (Ankit Aggarwal)]

There's been a lot of feedback about the name of the generated accessor SPMResource. The proposal initially explored generating an extension on Bundle with the name moduleResources. This was changed to the struct because of some caveats this brings with @testable imports:

a) There is no way to disambiguate names in an extension. For example, if there are two modules that are imported into a test module using @testable, you have no way of disambiguating the two moduleResources.
b) If your test target only imports one module that has resources, you would be able to access its resources. This will immediately break as soon you import another target with resources.
c) If your test target has resources, the moduleResources of the test target will shadow the moduleResources of all of the targets you import.

I am now thinking that maybe this is OK. We can say that a module always need to explicitly expose the accessors using some API chosen by the package author (solves a and c). SwiftPM can always generate an extension for the test targets in order to shadow the moduleResources that may come from importing any target with resources (solves b).

[beccadax (Becca Royal-Gordon)]

@Aciid

a) There is no way to disambiguate names in an extension. For example, if there are two modules that are imported into a test module using @testable , you have no way of disambiguating the two moduleResources .
b) If your test target only imports one module that has resources, you would be able to access its resources. This will immediately break as soon you import another target with resources.
c) If your test target has resources, the moduleResources of the test target will shadow the moduleResources of all of the targets you import.

The design I’m currently considering for module-qualified names would alleviate these issues:

import UIKit
@testable import Foo

...
  let expected = UIImage(named: “bar”, in: .currentTarget)
  let actual = UIImage(named: “bar”, in: .Foo::currentTarget)
...

Granted, I need to actually implement the thing, but the plan is to do so.