Draft Proposal: Package Resources

In general big +1 to the proposal.

As a nitpick I am not a fan of the SPMResources naming, what about TargetResources or SwiftTargetResources?

10 Likes

+1 +1, we need this!

Agreed. I'd maybe prefer something like Resources, PackageResources, or PackagedResources

I'm very excited about the future direction of possibly using this namespace a place to code-gen type-safe accessors to individual resource files!

5 Likes

Have you considered a hybrid approach, whereby anything within a top-level Resources directory is included automatically, and additional resource files in other locations can be included manually as well?

Also, will there be a mechanism to specify that the contents of a particular directory should all be included, such as “This folder contains resources” perhaps with the ability to specify “…and they are all png files”?

2 Likes

Yep, see alternatives considered section. Oh nvm, you're talking about something else. Yes, I think this was brought up at a time but it feels too "magical". Such things have been a big source of confusion in the past.

Yep, pointing to a directory will be possible using the .process API. See this section. In the future, we can add support for glob patterns using fnmatch() semantics for restricting the file types.

3 Likes

This sounds fantastic!

I do share the sentiment though that Resources , PackageResources , or PackagedResources would be a better name for the struct.

6 Likes

You mentioned the use case of test fixtures. In my case I want to share my test fixtures across several test targets. Would I need to create an extra target to associate with the resources and use it as a dependency?

Correct. In such cases, you probably want to share more than just text fixtures anyway and there's already some sort of "TestSupport" target. This is also slightly hinted in this section:

This approach also allows creating codeless resource bundles. Any package target that really wants to just vend the bundle of resources could implement a single property to publicly expose the bundle to clients. It seems reasonable that the package authors have to explicitly vend them.

2 Likes

This sounds really great and I am really looking forward to having it available.
I also share the sentiment that SPMResource is a little clunky.
Maybe PackageResource or BundledResource could be an alternative.

Furthermore I wonder if there could be "pure" resource packages that may be a dependency in another package.
An example would be 3D mesh files (i.e. .obj or .dae files) that are usually quite large and therefore might be versioned outside of the source code repository, yet the runtime needs them to function.
If it might be possible to depend on them similar to a library dependency this might be nice to have.

Example:

Mesh package

// Package.swft
let package = Package(
    name: "MeshPackage",
    products: [.resourceBundle(name: "MeshResources", targets: ["MeshResources"])],
    targets: [.target(name: "MeshResources")]
)

Game package

// Package.swft
let package = Package(
    name: "Game",
    dependencies: [ .package(url: "https://example.com/meshPackage.git", from("1.0.0")) ],
    targets: [.target(name: "Game", dependencies: ["MeshPackage"])]
)
4 Likes

The use case you described is very similar to what Ben described here and will be possible with the design in this proposal. You can create a target with a single Swift file to expose the bundle path and then export it as a library product.

3 Likes

+1 (And despite my previous musing about various possible directions, my real preference is in agreement that the “standard” location for resources should be within the target’s source directory.)

I would like to know the semantics of the bundle property. If it goes untouched such that the compiler dead strips it, does SwiftPM reserve the right omit its associated resources? I know that functionality won’t likely be in the initial implementation. But I would still like the proposal to explicitly state that the SwiftPM is allowed to do that, and that the actual presence of the resources in the built product is only guaranteed by the actual presence of the bundle property in the built product. The foolishness of implementing some alternate access is hinted at in places...

For example, the source code cannot assume that
the resources will be in the same bundle as the compiled code (if bundles even
exist as separate entities on a given platform).

...but it only says resources may be hard to find. It is never stated that the they may not even exist if you try to access them some other way. I don’t want a world where the optimization becomes impossible because too many packages rely on alternate access to resources.

Otherwise, if you don’t think it is good future direction, please mention it in the alternatives section and explain why the idea was rejected.

5 Likes

Thanks for the feedback!

You raise a very good point. I want to see what others think but I tentatively agree that SwiftPM reserves the right to omit the associated resources if the bundle is not accessed using the provided accessor. The only supported way to access the bundle will be through the accessor provided by SwiftPM.

@NeoNacho also suggested that the build system implementations can use an unstable UUID for the bundle name to make it difficult to find it using some other mechanism.

3 Likes

Maybe a hash of some kind? You don't want something new on every build, it would be nice to get deterministic packages (yes, a bunch of the resource processing also fails at this, but we can wish).

2 Likes

I would proposed the syntax for SwiftPM resource bundle be

Bundle.swiftPackage.path(forResource: "DefaultSettings", ofType: "plist")

Accessing the generated bundle as an extension of Foundation.Bundle itself and not a struct used as a namespace, feels more natural to Swift's API naming guidelines.

Precendent:
Bundle.main Apple Developer Documentation

2 Likes

I'm not sure if I'm following how resources are found and handled, so first is this correct:

  • If resources is unset on the .target, then what exactly is done? The Rules for determining resource files section talks about the Sources and Tests directories, but doesn't the path actually come into play normally? i.e. - will the lack of a resources mean anything found in path with the supported extensions is used (included the defaulting is path isn't set)?
  • If resources is set on the .target, then no files are auto-found via extensions, just the resources path(s) are checked.

The one downside would seem to be if you need to copy something, you also have to go to explicitly listing the process things also; but maybe that is the best as it is more clear.

Which then leads me to this; when explicitly using resource, you can specify paths for process and copy; how exactly are they handled?

  • For copy if the path is a file, then I'm guessing it is copied as is into a the bundle? But what if the path is a directory? Is that directory copied or are the items within it copied?
  • For processed the same holds, what if it points to a file (or a package (.storyboard, etc.) vs. if it points to a directory?

i.e. - are these like sources where you should list the explicit files/bundles to be processed or are they sorta like path where you can point a directory of things to collect so it needs less updating as things are added/removed with time?

The proposal does say:

Are all resources types supported? Some subset?

One last comment/thought - SPMResources /SPM_MODULE_BUNDLE worries me a little because it means a package trying to support SwiftPM and other systems will have to resort to conditional code for resources as other systems tend to follow the existing mode of putting resources in the bundle the code goes into.

  1. Agree with that resource should bind to Target, not global via different targets.
  2. Worry about this extra hand to touch source code. Which means, to support Swift Package Manager , my exist code which use Xcode framework, should change code with that SPMResources , and with extra macro to support both non-SwiftPM user and SwiftPM user.
    What about just let Package Author, to give you the bundle name, so, I can use the same code, to support Xcode native project and SwiftPM ? Because actually, the bundle path is only related to a name, no need for a runtime magic struct something

Actually, for current Resource struct, it can have a extra similiar struct BundleResource. (The original bundle arg now receive protocol), It must have a name, other methods is the same as Resource. Each target, can have both raw resource (copy to sources files folder) and bundled resource (with the Bundle compatible folder format). This looks the same as CocoaPods DSL resources and resource_bundles, the same as Xcode's target build phase, which allows you to embemd resources, as well as any bundle resource (xxx.bundle) in Build Phase.

For raw resources files, it should be placed along side with compiled binary (dynamic/static). For bundled resources, it should be placed along side with compilbed binary, with a .bundle folder, which filename is the Package.swift DSL provided. Both of these two types have their use case, maybe. (Or, can we force all the resource must use bundle ? Because bundle can avoid same resource name conflict, especially using the Static Library Package, which have no .framework folder)

And the number for raw resources and bundle resources should not have a limit. Current that draft SPMResources have only one method bundle, which means, one target only one bundle? If I have both metal shaders for bundle1, image in bundles 2, must I mixed them, or create two Targets. Sounds frustrating and not flexible. Compared to existing CocoaPods resource bundles DSL and Xcode project itself.

This can make it possible to share resources by different Targets via user's own string based bundle name (They can provide a public API). Because your draft say that SPMBundle is internel level control, but different targets can not have same module acess control.

The external build toolchain like Xcode team, can register a plugin to both .png, .shader, and the bundle format (.bundle) when using process. For example, Xcode can try to build .xcassets to Assets.car, replace the original .xcassets file.

Excellent design.

I was thinking about this too, although my spelling was Bundle.current or Bundle.currentTarget.

My other suggestion: There should be a built-in .ignore rule to go with .copy and .process, and dotfiles should probably be ignored by default.

6 Likes

If the bundled resource and non-bundled resource have heavy differences, maybe provide two new args resources and bundledResources is better. Just my personal idea.

And, if we do not support non-bundled resources, can Package Author control The name of bundle ? Must it using a runtime magic struct to access this ? (Which will cause Package Author need dynamic or macro check, if they want to support native build tool like Xcode, as well as other Package Manager).

I like this approach, but we should differentiate the current target bundle from the current target associated resources bundle (which maybe or not be the same bundle): what about Bundle.currentTargetResources?

1 Like

The proposal mentions that the processed/copied resource bundles will be placed beside the final executable, and some comments here make it seem like the bundle name/path will be an implementation detail that only the generated SPMResources struct will know how to read from. Is that accurate?

My concern here is with Linux distributables. If you wanted to build, say, a Swift server which contains resources, how would a developer be able to package the executable and resource bundles into an archive that can be copied to the deployment machine, given that the name of the resource bundles are not known?

IIRC, executable files for a Swift package are placed within the .release/<platform>/release, meaning that if you had multiple executables with resource bundles, it would not be trivial to find which resource bundle maps to which executable (and depending on the naming mechanism, you might even have collisions).

3 Likes