I've started building a simple prototype to test/validate workflows for building shared libraries with evolution turned on (as we have a need for that for our product) - starting with macOS / .xcframework but with the long term goal to get it working on Linux also as discussed elsewhere.
I just found one interesting peculiarity - it seems that SwiftPM will copy the shared library from a .xcframework when using it as a binary dependency - this seems a bit counterintuitive, as it makes a local non-shared copy then of the shared library.
Wouldn't it make sense to (at least optionally) allow for making the reference to the shared library provided by the .xcframework to reside in place, so that when building with evolution support turned on I can share the single shared library image across multiple clients consuming it - just as typical Apple system level frameworks?
I think it's just done this way to avoid adding additional rpaths to the binary, especially in release mode. That said, I hadn't actually questioned this before, we were just following the approach Xcode takes (which has a ProcessXCFramework task that copies the libraries into place at the beginning of the build).
Aha. I think it’d make sense to consider differentiate between an xcframework that contains a static library (which is fine to copy) vs one that contains a dynamic one perhaps? Especially when using a file system reference - for a remote URL it’s natural to copy. It is desirable to be able to build a shared library with evolution turned on and really share it amongst consumers..
It feels a bit marginal to me to do something that would only really be useful in debug mode and if someone happens to use the same local package for multiple executables at the same time.
BTW, you should be able to do this already by just passing the necessary -rpath flags to swift build (that said I'm not 100% firm on how the precedence of rpaths works, it might be necessary to also get rid of the default one via something like install_name_tool).
Oh, I’m all about release mode really - but thanks for the idea about adding specific r-paths - I’ll test that tomorrow and see how the precedence works out.
Ok, I can work around it with LD_LIBRARY_PATH / DYLD_LIBRARY_PATH at least, didn't find any ways to set it for the actual swift build (maybe I'm missing something)?
Still, I think the default for .dynamic libraries (especially those with evolution turned on...) should be to link to the dependency location if in a filesystem rather than copying it - imagine that you acquire a third party library and install it on you machine - it'd be nice for all consumers to share that instance - and to take advantage of subsequent upgrades of the shared library (which is using evolution...).
I'm a bit confused about what you mean here since you seem to be mixing development time and deployment. I agree that there are cases where you'd want to share identical libraries during deployment, but that's not something SwiftPM covers today.
If instead of copying a library during building, we'd insert an rpath to it, that would in the vast majority of cases not be portable to deployment (e.g. we'd reference something like /home/users/package/.build/checkouts/other/x.xcframework/....). Setting DYLD_LIBRARY_PATH or such would also not change anything for deployment.
So I agree that for a deployment story, it would be interesting to offer control over rpaths of individual binaries (with sensible defaults), but whether or not we copy the library into the build directory during development is orthogonal to that.
Let me try to clarify - let's imagine a concrete use case:
The customer installs our binary distributed framework which they want to dynamically link / use the same copy at runtime - this is installed to e.g. /MyGreatLibrary/MyLibrary.xcframework - this is then installed on both the development and deployment machine at the customer site.
They add this as a dependency to a set of products - at that point, you just want to reference that point in the file system - the assumption here is that the development and deployment location would be identical.
Well, that's not where the customer would reference the xcframework in my example, see above - it's expected to be installed on development/production machines (as part of our product).
Of course, having control over rpaths at the build time with sensible defaults as you suggest would be fine too (and then I agree it's an orthogonal issue, apart from perhaps being a bit unnecessary work performed during the build, but that's another issue too). patchelf and install_name_tool may do the trick in the meantime (as there's no way to manipulate rpaths embedded using SwiftPM?)
Even though our xcframework is a pure swift library built with library evolution? (And xcframework seems like the right vehicle) It seems systemtargets are more about interfacing with c libraries and similar? Or I’m missing something.
I wouldn't say so, the distinction between the two is more about versioning. From the perspective of SwiftPM, a system library is inherently unversioned vs. a binary dependency is part of the regular package resolution process. So any binary coming from a binary dependency is inherently thought of being more "local".
Again, this is only thinking about development time, though. For deployment, you may want to distribute what was a system-library with your program or use a pre-installed version of a binary dependency.