Great proposal, I'm excited to delete some checked-in generated code!
I took a stab at designing a hypothetical SwiftProtobuf PackageExtension using the APIs in this proposal with the real SwiftProtobuf package and came up with a few questions.
For background, the SwiftProtobuf project contains a plugin executable protoc-gen-swift
, implemented in pure Swift and exported as an executable product by Swift Package Manager. Adding a new packageExtension product to that package implementing should be relatively straightforward. However, that package is actually not enough to generate Swift source code from a .proto file, it needs the Protobuf Compiler tool (protoc
) as well.
To illustrate, a typical invocation of protoc
to generate Swift source files looks like this (with variables filled in by the build system):
${ProtocToolPath} \
--plugin=protoc-gen-swift=${ProtocGenSwiftToolPath} \
--swift_out=${TargetGeneratedSourcesDir} \
--swift_opt=ProtoPathModuleMappings=${ModuleMappingsFilePath} \
--swift_opt=Visibility=Public \
-I ${TargetSourcesDir}/Proto \
-I ${TargetDependencyIncludePath} \
-I ${PackageDependencyIncludePath} \
-I ${SystemIncludePath} \
${TargetSourcesDir}/Proto/example.proto
# Produces ${TargetGeneratedSourcesDir}/example.pb.swift
Looking over the proposed API I'm not sure that all of these variables can be filled in.
ProtocToolPath
is the path to the protoc
tool executable. The protoc
tool is typically installed on the system somewhere in PATH
or downloaded into a project build dir and run from there [1]. If the tool is on the system PATH
we need a way to express this (and escape the sandbox for it). If the tool is to be downloaded into the build dir we need a way to express that so that the package extension can find the tool.
[1] The Protobuf Gradle Plugin allows this to be configured.
ProtocGenSwiftToolPath
is the path to the protoc-gen-swift
tool, which is an executable product from the SwiftProtobuf package. This tool be obtained with TargetBuildContext.lookup(tool: "protoc-gen-swift")
, but a new API will be needed to the Tool
protocol to get its path.
TargetGeneratedSourcesDir
is a directory for generated sources that will be compiled into the current target. This can be obtained with TargetBuildContext.buildDirectory.appending("ProtobufGeneratedSources").string
.
ModuleMappingsFilePath
is the path to a generated file containing metadata for the Swift Protobuf plugin. It contains mappings of .proto
file names to their corresponding Swift module names so that generated code contains the correct import statements. It will need to be generated prior to the above protoc
invocation and take metadata for the current target and its transitive dependencies as arguments. It seems possible to create another Tool to generate this file, and extend the TargetBuildContext protocol to have a new property var dependencies: [TargetBuildContext] { get }
. This tool would have an empty set of inputs in the current target.
TargetSourceDir
is the root of the sources directory for the current target, for example Sources/ExampleAPI
. It's required to allow protos to write their imports without regard to which target they're in (similar to allowing chevron-includes in C projects). It's unclear if this is easily attainable from TargetBuildContext.inputs
.
TargetDependencyIncludePath
is a path to a directory in a target dependency containing .proto
files that can be imported similar to include
in a C target. It will require an API to walk a target's dependencies and query metadata about attached Build Rules and Package Extensions.
PackageDependencyIncludePath
is the same as a TargetDependencyIncludePath
, except that it points to a directory in a package dependency's checkout.
SystemIncludePath
is a path to a directory containing "well-known" protos, similar to /usr/include
. If protoc
is installed on the system this will probably be a nearby system directory and it will thus need to be allowed from the sandbox. If protoc
was downloaded into the build directory this will be an adjacent resource directory and the tool lookup API will need to be able to find it.
Summarizing my main questions:
- How do you imagine supporting "system" tools and their associated resources?
- How do you imagine supporting "downloaded" tools and resources (that SwiftPM can't build)?
- How do you feel about extending the API to allow querying transitive dependencies?
- How do we express llbuild-level dependencies on inputs in transitive dependencies?