I've been building a small UI application on Linux using the SwiftGtk package. This package uses a custom build script which generates wrappers and sets a bunch of flags for SwiftPM to build the project. And it works.
One of the biggest problems I'm having is the lack of sourcekit-lsp support while editing. GTK is a huge framework, and not being able to see what properties and methods a type has can be really frustrating. Sourcekit-lsp apparently can't resolve importing the SwiftGtk framework, so it can't get any of this information.
However - SwiftPM can build the project. Why isn't sourcekit-lsp able to use those flags when resolving dependencies?
You can pass the same -Xswiftc and -Xcc arguments that you pass to swift-build to sourcekit-lsp on the command line. How you pass arguments to sourcekit-lsp will depend on the editor you're using, for example there is an argument array in the VSCode sourcekit-lsp settings, which you can provide either in your user settings, or workspace settings.
Sure, but is there an option to say "use whatever flags SwiftPM is using for each file"? Those flags come out of a bunch of shell scripts and I don't think I should have to dig and discover what their values are (basically, un-scripting them in to hardcoded values in my workspace) when SwiftPM could just record the flags it saw .
IMO, an integrated toolchain should strive so that anything the package manager can build, and anything the compiler can understand, should be available to all the tools (even if your build system is complex - the package manager understands which additional information the compiler needs, especially if it was already successfully built). There should be a way to share this information with minimal friction for the user.
This would be great to have for the reasons you mention. Unfortunately swiftpm does not store that information in a way we can access it robustly. If swiftpm exposed it, we would be happy to make use of it.
Note: the issue you are describing also effects swiftpm itself. If you have a package that cannot build without -Xswiftc/-Xcc/etc., then every time you invoke swift-build you have to provide the same necessary arguments. They are outside the pure build model that is exposed by Package.swift.
From what I can see, it appears that sourcekit-lsp does quite a lot of guesswork to try and reconstruct the SwiftPM understanding of things, starting from the manifest itself (which is where this problem emerges: there is no way to reconstruct information that come from outside the manifest). That's kind of sad; IMO SwiftPM should be doing a better job to help sourcekit-lsp just construct a correct compiler invocation for each file.
Trying to mirror SwiftPM's understanding of the manifest and which compiler flags it adds seems like a game where the only winning move is not to play. It's probably more robust to have SwiftPM serialise its state to a JSON file (at the highest level where it is possible to entirely reconstruct the build plan).
This is a known issue with the way llbuild + swiftpm interact making it hard for other tools to get compiler arguments. I think there is general agreement to fix this so swiftpm can compute the arguments directly, but no one has had time so far. It hasn't been a priority for us since this logic is very stable and hasn't been a source of bugs for us in practice.
Can the arguments always be computed though? For example when cross-compiling, as far as I understand, the --triple argument always has to be passed explicitly to swift build. How is SwiftPM supposed to know which platform I'm cross-compiling to beforehand without me passing any flags to it?
To clarify, as I'm primarily working on cross-compiled code these days outside of Xcode, this is a very big pain point. SourceKit-LSP always assumes I'm compiling for the host, there's no way known to me to make it aware of cross-compilation, so none of the #if os(...) compiler directives are handled correctly by it.
Matching the set of -Xswiftc -Xcc etc. arguments that are passed to swift-build. Today the only way for this to happen is to pass them to sourcekit-lsp yourself, since there is no standard place to store them.
How sourcekit-lsp internally interacts with swiftpm to compute compiler arguments.
The comment you're replying to is about (2), but I think the problem you're talking about for choosing the target is (1). As you say, sourcekit-lsp cannot compute it, but then neither can swiftpm - that's why you had to provide it to swift-build as well.
Thanks for clarifying, but unfortunately, I'm still not sure how SourceKit-LSP can be used when cross-compiling. I won't be able to pass the triple argument to sourcekit-lsp with -Xswiftc, since this flag is used by swift-build itself to resolve the targets with conditional target dependencies based on the triple. If I start sourcekit-lsp with the -Xswiftc --triple -Xswiftc wasm32-unknown-wasi flag, the build fails because swift-build (i.e. SwiftPM) would attempt to build targets it's not supposed to touch based on conditions given in Package.swift.
Oh, I didn't realize this was specifically a swiftpm option not a compiler option that was being forwarded. Yeah, we need to thread it through sourcekit-lsp as well like we do for -Xswiftc etc. Should be straightforward, I can put up a PR in a bit.
Famous last words. It turns out there are a handful of related options that all interact with each other to determine the destination/toolchain. @Max_Desiatov would you mind filing a bug (bugs.swift.org) and mention which options you are using to help prioritize this (e.g. --triple, --sdk, --destination, --toolchain).
Yes - what I'm suggesting is that rather than try to independently reproduce SwiftPM's understanding by importing its logic and applying it to the same manifest, we should just straight save/restore the state of SwiftPM before it does any actual building, when it knows exactly what the next stage of the pipeline (the compiler) needs. That way sourcekit-lsp doesn't need to guess, and will always match whatever SwiftPM did last time you ran it.
I was gonna create a PR to propagate the --destination argument from sourcekit-lsp to swift build, but discovered that sourcekit-lsp still doesn't use the new argument parser, I've created a PR for that in the meantime. The new --destination argument implementation depends on this PR now.
.