Strategy for building libraries supporting multiple platforms or flavors


(Dan Thorpe) #1

Hi

First of all - congratulations to the Swift team for all their work, it’s really exciting times, and Swift Package Manager especially is a welcome surprise. I don’t think anyone saw this coming.

I have a question about cross platform packaging, as I’ve recently been struggling the best way to figure this out with current technologies, and I think it brings up some interesting things to consider when addressing the convention vs configuration balance. If this is the wrong list, or not something that you’re interested in, I apologize.

Are they any plans or strategies yet how cross platform libraries will be packaged? I think I’ve read through all the docs, but not seen anything on this, except referent to an target array in Package.swift.

The situation is that often a different set of source files is required when targeting say iOS, watchOS, tvOS, OS X, and I guess Linux. Because the available system frameworks on those platforms are not the same. Similarly, a library may offer a different flavor for the same platform, for example a library suitable for use in an application versus an extension API compatible version.

An example of such a framework is here: https://github.com/danthorpe/Operations/blob/development/Operations.podspec (linking to the podspec, and it’s the easiest way to see how much configuration is needed). In this particular case, it’s not possible at the moment to integrate the iOS extension compatible framework via Carthage because Xcode cannot produce different named products for different targets if the module name has been manually set. And in these situations I think the module name would need to be the same across all platforms.

So, for SPM and a file system based configuration - I’m wondering how this kind of thing would work. Xcode’s little checkboxes solve this problem quite neatly. I don’t really have any answers, just a few ideas.

1. A judicious use of files aliases so that the directory listing contains all the appropriate files? - Pro: avoids text based configuration a-la podspec, Con: this could be tricky to maintain, hard to visualize (but could be solved with tooling).

2. Files are named using a pattern which indicate which targets they should be included in? e.g. Sources/_Shared/iOS_Foo.swift, Sources/_Shared/iOS_watchOS_Foo.swift - Pro: can tell from looking at the file name what platform it would be included in. Con: Incredibly ugly, doesn’t scale, might not play nicely with Git which tracks content not files.

3. Some kind of source code annotation. i.e. annotate the source files in a comment block in such as way that their platform/target membership match up to what is defined in Package.swift - Pro: probably the most flexible, Changing targets/configuration can be tracked in SCM. - Con: still hard to visualize (but could be solved with tooling), introduces configuration which isn’t in Package.swift?

4. Just use a lowest common denominator approach to build trees of nested modules? - Pro: can keep to minimal configuration, no need to duplicate or alias files. Con: not sure how scalable this would be - at the worst case you might end up having to create many modules which are just single swift files.

Other thoughts would be how to include groupings of files which are common across multiple targets/platforms, could perhaps prefix the folder to indicate that SPM should not generate a build target using it?

Anyway, I’m really excited to see how SPM progresses, and would be interested in knowing the SPM teams' current thoughts on this.

Cheers,
Dan


(Max Howell) #2

Hi Dan,

I think I can express our approach here by saying: we want to build something where 90% of the work is easy and the remaining 10% is possible.

Also I want to say: we hope all packages (not apps) will be made to be cross-platform from the outset.

Currently we don’t support these kinds of configurable excludes/includes. But this is how I expect it will work in your Package.swift:

#if os(Linux)
targets["foo"].sources.remove("foo/bar.swift")
#endif

I have talked at length with other people here about potentially having directories for each platform, and ultimately we decided this leads to source layouts that are difficult to read and maintain. We feel that you are better served by writing modules that are cross-platform and then having apps that have a target per platform with some sources added and removed.

However, because our manifest is Swift you can easily cater to layouts that use a pattern of your choosing:

#if os(Linux)
targets["foo"].sources = targets["foo"].sources.filter {
   return $0.hasSuffix(".linux.swift")
}
#endif

I don’t think we should encourage excessive nesting of modules for platforms. It is just not practical. We aim for a sensible, pragmatic solution that caters to majority usage but is flexible enough to provide the features people need.

···

On Dec 7, 2015, at 1:09 PM, Dan Thorpe via swift-build-dev <swift-build-dev@swift.org> wrote:

Hi

First of all - congratulations to the Swift team for all their work, it’s really exciting times, and Swift Package Manager especially is a welcome surprise. I don’t think anyone saw this coming.

I have a question about cross platform packaging, as I’ve recently been struggling the best way to figure this out with current technologies, and I think it brings up some interesting things to consider when addressing the convention vs configuration balance. If this is the wrong list, or not something that you’re interested in, I apologize.

Are they any plans or strategies yet how cross platform libraries will be packaged? I think I’ve read through all the docs, but not seen anything on this, except referent to an target array in Package.swift.

The situation is that often a different set of source files is required when targeting say iOS, watchOS, tvOS, OS X, and I guess Linux. Because the available system frameworks on those platforms are not the same. Similarly, a library may offer a different flavor for the same platform, for example a library suitable for use in an application versus an extension API compatible version.

An example of such a framework is here: https://github.com/danthorpe/Operations/blob/development/Operations.podspec (linking to the podspec, and it’s the easiest way to see how much configuration is needed). In this particular case, it’s not possible at the moment to integrate the iOS extension compatible framework via Carthage because Xcode cannot produce different named products for different targets if the module name has been manually set. And in these situations I think the module name would need to be the same across all platforms.

So, for SPM and a file system based configuration - I’m wondering how this kind of thing would work. Xcode’s little checkboxes solve this problem quite neatly. I don’t really have any answers, just a few ideas.

1. A judicious use of files aliases so that the directory listing contains all the appropriate files? - Pro: avoids text based configuration a-la podspec, Con: this could be tricky to maintain, hard to visualize (but could be solved with tooling).

2. Files are named using a pattern which indicate which targets they should be included in? e.g. Sources/_Shared/iOS_Foo.swift, Sources/_Shared/iOS_watchOS_Foo.swift - Pro: can tell from looking at the file name what platform it would be included in. Con: Incredibly ugly, doesn’t scale, might not play nicely with Git which tracks content not files.

3. Some kind of source code annotation. i.e. annotate the source files in a comment block in such as way that their platform/target membership match up to what is defined in Package.swift - Pro: probably the most flexible, Changing targets/configuration can be tracked in SCM. - Con: still hard to visualize (but could be solved with tooling), introduces configuration which isn’t in Package.swift?

4. Just use a lowest common denominator approach to build trees of nested modules? - Pro: can keep to minimal configuration, no need to duplicate or alias files. Con: not sure how scalable this would be - at the worst case you might end up having to create many modules which are just single swift files.

Other thoughts would be how to include groupings of files which are common across multiple targets/platforms, could perhaps prefix the folder to indicate that SPM should not generate a build target using it?

Anyway, I’m really excited to see how SPM progresses, and would be interested in knowing the SPM teams' current thoughts on this.

Cheers,
Dan

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev