SPM build tool plugins access to package directory

So I started to play with automating some things using the new build tool plugins for one of our projects where we have a command line tool that will generate a flatbuffers schema specification from Swift source files (which is the 'true' definition) and then in turn run flatc on that schema to generate flatbuffers accessors - was very easy to get it working (with some help from GitHub CodeSearch) and I got everything generated inside the .build/xxxx/yyy path.

In reality, I'd want to have the client depending on this build tool to be allowed to generate code to the myPackage/Sources/DataModel directory (and to use normal SPM dependency management for those, I want them to be part of the default target), but the build tools are sandboxed so that fails - is there any appropriate way of getting things from the sandboxed output directory to the main package one?

The other obvious approach would be to do the generation of this in a separate SPM and just pull that one in for the client, but it'd be interesting to solve the above to automate the generation of that original repo too, so back to square one in that case.

1 Like

Ok, I just found the wwdc session and it’s clear this is intentional - but I must admit I really don’t get it including the example used by @abertelrud - generating the source code in the build directory - how should one make it part of the actual build then? Side note is that I might want them in the repository too, but that’s a different discussion point.

So, if I use a build tool for source generation that ends up in .build, how do I make use of it from my spm project?

As source gen is repeatedly used as the example I must be missing something fundamental here.

The generated source code automatically becomes part of the target you're applying the plugin to, there shouldn't be any extra steps required.

1 Like

Thanks! But are the symbols in that generated code available for e.g. completion in Xcode then? Browsable? Just because it's generated code doesn't mean it's only for building? I'll play around now (but it'd be great to mention that somewhere, or did I miss it?)

Thanks, it seems to work - but no completion nor browseability of generated source from Xcode and I get a warning for the flatbuffers schema also - don't know if it is suppressible:

no rule to process file '/Users/hassila/Library/Developer/Xcode/DerivedData/api-core-fkhvsgdluetzutagazrzwuafefhz/SourcePackages/api-core/Core/DataModelBuildTool/datamodel.fbs' of type 'file' for architecture 'arm64'

I think the lack of visibility of the generated source is a bit unfortunate from a usability standpoint (it'd be more ergonomic currently with a command tool that generates source into the package, so one can get completion, but then it's not automated anymore...), otherwise it's really nice that it's so little work to get integrated build tools, nice start!

So hope the lack of source visibility (and completion) is something we might see in the future, not sure how that would be integrated though.

1 Like

You should get completion once the source has been generated (I believe that should happen during the index build as well, definitely as part of indexing-while-building, but it's not immediate indeed). I believe "Jump To Definition" also works.

For the warning, I am assuming that scheme file is an output of your plugin but shouldn't actually be part of the built product? In that case, I think the only way to suppress the warning would be not declaring it as an output file. Currently the API contract is basically that anything that is an output of a plugin will become part of the built product, except for some known unsupported types (e.g. C family language source code gets a specific warning in tools-version 5.7).

2 Likes

@NeoNacho thanks very much for the explanation, it did indeed start working after I had built including jump to definition. Awesome! I removed the .fbs as the output and that warning was squashed (it is actually the input to the next command so I thought I should declare it, but seems to work ok now!).

Very nice, it was just a bit 'too magical' and worked out of the box, maybe I missed it, but otherwise a note somewhere that any source generated by a plugin for a target is automatically added to the target would have been great :slight_smile:

Thanks!

1 Like

Hopefully final follow-up in Package built with dependency on another package that uses a build tool plugin fails to find the build tool product

I'm on a similar path however my .prebuildCommand is running a shell script to fetch a number of otf fonts which I wish to be included in my package Bundle.

The script executes fine, but I'm struggling to know how my fonts get from the sandboxed output directory to the main package one.

I have a similar output:

no rule to process file 'xxxxx/Assets' of type 'file' for architecture 'arm64'

Resolved! It was simply due to needing to use swift tools 5.7 rather than 5.6
Painful because my build plugin was getting executed and sandbox was getting populated with resource.

1 Like

Unfortunately, this is intentional. Plugins in 5.6 didn't correctly handle unsupported cases like resources, but we didn't want to retroactively change the behavior for older packages in later versions of SwiftPM.

1 Like