This pitch is part of the work to break up [Pitch] Extending Build Tool Plugins for Customizing Targets into consumable pieces. This is the first draft of the evolution proposal for extending build tool plugins to generate code for Clang family languages into mixed language targets provided by SE-0541.
Introduction
Building on SE-0541 which allows C/C++ and Swift source to mix in a single target, this feature removes restrictions and supports code generation from build tool plugins to do the same.
Motivation
Despite the Swift language's great support for interoperability with C and C++ libraries, Swift packages have lagged behind. SE-0541 brings packages a long way by allowing mixed language targets. However, today SwiftPM prevents build tool plugins from generating C and C++ code into Swift modules where interoperability could consume those files.
To simplify the usage of many C/C++ libraries from Swift, it has proven helpful to create additional Swift bindings for these libraries. This could include replacing opaque structure pointers which resolve to OpaquePointer in Swift with more type-safe interfaces. This could include a mix of C and Swift code to bridge these gaps. It could also include the generation of Clang API notes which can rename C functions so that clearer Swift functions can replace them but keep a familiar name.
Ideally, these bindings are generated either from metadata or by analyzing the C and Swift code of the bindings generated by the interoperability feature of the compiler. Today, these bindings are hand-coded, which makes it very easy for them to get out of date unless constantly maintained. When these bindings are generated, they can adapt as the APIs for the underlying
libraries evolve.
Today, the Swift package ecosystem is full of C shim targets to work around the inability to mix languages in a target. SE-0541 enables sources for a target to mix. The goal of this proposal is to extend that to the code generated for targets.
Proposed solution
The current restrictions in SwiftPM on the ability for plugins to produce C source, header files, and module maps are removed for C and Swift targets.
Support is added to the package manifest to allow header search paths to be made public to dependent targets. Support is also added to allow plugins to contribute header search path settings to the target for the header files it generates.
This feature relies on the capabilities of Swift Build where targets are naturally mixed language. It is not made available when using the deprecated Native build system of SwiftPM which does not.
An example package in SwiftPM's repo is provided to illustrate the use of most of these features. It is in Examples/swift-sdl. It includes a code generator that parses the expected Swift interop interface to SDL using swift-syntax and parses the SDL header files using clang to produce an AST in JSON and generate a Swift binding that eliminates occurrences of OpaquePointer and replaces them with wrapper structs that replicate the opaque structure pointer types found in the C interface. It includes an API notes file to rename the current interface functions so that the Swift binding can reuse those names and call into those interface functions wrapping and unwrapping the OpaquePointers.
The example assumes that the SDL library source is accessible to the swift-sdl package using a git submodule. The SDL library is built using CMake using an external process into a build-* directory for each platform swift-sdl supports. These libraries are made accessible with a new library search path setting on the SwiftSDL target.
This example shows how valuable these capabilities are to making the API for these valuable libraries more Swift friendly improving the developer experience.
Detailed design
Removing restrictions
For Swift 6.4, an experimental version of this feature was introduced that allowed for the generation of non-Swift source, header files, and module maps for targets recognized as C modules. This was facilitated with a special tag that was placed in the swift-tools-version in the package manifest.
With SE-0541, Swift modules may now have C source so the check for C modules is removed, allowing these files to be generated for Swift modules as well. Also, the check for the tag is removed and replaced by the check for the supported Swift tools version.
Header search path extensions
One of the reasons why generated C header files were not supported in the past is due to the complexity of integrating them into the build. Currently, Swift packages do allow for the specification of a headerSearchPath C and C++ setting; however, that is made private to the target and cannot be used by dependent targets. This is especially cumbersome with the Swift binding use case, which may want to re-export header files for the library through a generated or hand-provided module map.
It is an error if both the target and a plugin produce a module map. This includes the generated module map when the public headers path is declared or implied by the existence of a directory named include. It is assumed that if the user provides a module map, it should be used unaltered by files generated by plugins. Package authors may, however, leverage the plugin header search path feature below to expose generated header files to be included by the target-provided header and module map.
Header search path usage
First, the header search path setting is expanded to include a "usage" property. This defines how the path may be used by the target itself and by dependent targets that compile against the target. It is inspired by the usage properties from CMake.
public enum Usage: String, Codable, Hashable, Sendable {
case `public`
case `private`
case interface
}
public struct CSetting {
public static func headerSearchPath(
_ path: String,
usage: BuildSettingUsage,
_ condition: BuildSettingCondition? = nil
) -> CSetting {
...
}
}
public struct CXXSetting {
public static func headerSearchPath(
_ path: String,
usage: BuildSettingUsage,
_ condition: BuildSettingCondition? = nil
) -> CXXSetting {
...
}
}
publicspecifies that the header search path is used by the compile of the associated target and is imparted on targets that compile against the public interface provided by that target.privatespecifies the header search path is used only for the compile of the associated target. It implies the APIs in those header files are private to that target.interfacespecifies the header search path is only imparted to dependents. This implies the APIs in these header files are public versions of the APIs for the target but are not needed for the build of that target.
Header search path added by plugins
If a target provides a hand-written module map, or uses a generated one for its publicHeadersPath, and wants to reference header files generated by a plugin, the path to those header files needs to be added to the headerSearchPath for the target.
To facilitate this, a new case is added to the Command enum to add build settings to the target the plugin is adding commands for. It purposely looks like build settings for targets in the package manifest. It has most of the features those settings offer including the usage feature added for header search paths for targets, as well as platform and build configuration conditionals.
public enum BuildSettingUsage: Sendable {
case `private`
case `public`
case interface
}
public struct Platform: Equatable, Sendable {
public static func custom(\_ platformName: String) -> Platform {
...
}
public static let macOS: Platform = ...
public static let macCatalyst: Platform = ...
public static let iOS: Platform = ...
public static let tvOS: Platform = ...
public static let watchOS: Platform = ...
public static let visionOS: Platform = ...
public static let driverKit: Platform = ...
public static let linux: Platform = ...
public static let windows: Platform = ...
public static let android: Platform = ...
public static let wasi: Platform = ...
public static let openbsd: Platform = ...
public static let freebsd: Platform = ...
public static let anyAppleOS: Platform = ...
}
public struct BuildConfiguration: Sendable {
public static let debug: BuildConfiguration = ...
public static let release: BuildConfiguration = ...
}
public struct BuildSettingCondition: Sendable {
public static func when(
platforms: \[Platform\]? = nil,
configuration: BuildConfiguration? = nil
) -> BuildSettingCondition {
...
}
}
public struct CSetting: Sendable {
public static func headerSearchPath(
_ path: String,
usage: BuildSettingUsage,
_ condition: BuildSettingCondition? = nil
) -> CSetting {
...
}
}
public struct CXXSetting: Sendable {
public static func headerSearchPath(
_ path: String,
usage: BuildSettingUsage,
_ condition: BuildSettingCondition? = nil
) -> CXXSetting {
...
}
}
public enum Command {
case buildSettings(
cSettings: [CSetting] = [],
cxxSettings: [CXXSetting] = []
)
}
Note: Trait conditionals are not available since plugins are executed after the traits have been evaluated when constructing the package graph, and so would be unable to affect the graph.
Library search path
To assist in the Swift binding use case, it should be possible to incorporate the library files for the external library into the build of the package.
To assist with this, a new linker setting is added to the package manifest to provide the library search path. A similar setting is provided for the framework search path. As with the header search path, the paths must be relative and point to a directory under the package root directory.
public struct LinkerSetting: Sendable {
public static func librarySearchPath(
_ path: String,
usage: BuildSettingUsage = .public,
_ condition: BuildSettingCondition? = nil
) -> LinkerSetting {
...
}
public static func frameworkSearchPath(
_ path: String,
usage: BuildSettingUsage = .public,
_ condition: BuildSettingCondition? = nil
) -> LinkerSetting {
...
}
}
Security
This does not affect the security profile of Swift packages. It does allow plugin tools to generate additional types of source. However, similar capabilities exist with Swift packages today.
Impact on existing packages
This feature is enabled with Swift tools version 6.5. Existing packages will be unaffected.
Alternatives considered
Consideration was given to whether plugins needed to be able to add header search paths. Given that module maps may only have one umbrella entry, handling multiple header paths would not be possible. Using the plugin-provided header search paths, module maps may be written to allow header files to be included from the target as well as the plugin-generated files.
Future directions
Package targets still require there to be at least one source file in the target so it can identify the target as pure C/C++ versus Swift. One often sees empty.c and empty.swift files to satisfy that requirement when the source is solely from generated files.
This is purely an internal requirement still leftover from the Native build system, where the decision about what type of module a target is must be made before the plugins run. SE-0541 allowed Swift modules to gain C module properties. These concepts should be further blurred into a single "source" module and let the build system determine what type it is after the plugins run.
That will require changes to the Plugin API which makes assumptions about the type of module at plugin invocation time where plugins can query certain build settings from the model.
Currently those settings are filtered by the build setting conditions so that only those settings for the current build request are made available. However, with the Swift Build architecture, the build description produced by SwiftPM needs to be independent of the build environment. The same package model needs to service multiple build requests.
To maintain that independence, the plugin build settings API added in this proposal will be expanded to give plugins access to all the build settings for a target, as well as a new category for plugin settings.
Acknowledgements
Thanks to Owen Voorhees for his work on SE-0541 and his advice on this proposal.
Also thanks to all the package writers who've pointed out the need for this feature and the example repos that have helped shape it. Our work continues to make their work easier and to eliminate the need for Makefiles and command plugins and external processes that would be better served being managed by the SwiftPM build system.