Is it possible to dllimport things that are dllexported from the same library? (Is that what you were referencing with "binary size and runtime overheads, and unnecessary warnings"?) If so, I think -static can be an "expert" flag, because everything will work "correctly" for static or dynamic linking if you omit it from a library, but if you include it you must link statically. If not…can we fake that somehow?
I agree that the mechanism for this would be to store in the swiftmodule a "guaranteed to be linked statically" flag, derived from whether -static is present when compiling.
Technically, no. However, the Microsoft linker will realize that is what happened and do so, emitting either warning 4217, 4826, or 4049 in the process. This will increase the binary size due to the indirect reference, the additional rebase entry in the IAT, the additional rebase itself, and the indirect call.
I really would rather this not be an expert mode flag. This is already needed to generate a static library, and its already plumbed through. This is not a new flag that I am proposing. But, yes, this was part of the reason that my original work focused entirely on getting the shared linking model working. The rest becomes much easier once that path works. Furthermore, with Swift Package Manager, this information is readily available to the build and we can pass down the information to the driver as appropriate.
I dont know if this will help at all, and i know this is a little out of topic, but right in the beginning of my mixed C++ and Swift codebase i have settled for use GN as a meta builder for Ninja avoiding using the swift builder, because i've just wanted to define the build rules for C++ and Swift in one place.
So i've just built a custom builder generator for swift on GN and now looking at this im kind of glad to have taken that route.
GN is Bazel inspired, but with a smaller and right to the point C++ codebase.
With it you have to define shared_library("library") or static_library("library") right on the module or you can have a variable to replace it for you according to some circunstances..
Like:
if (some_condition)
module = shared_library
else
module = static_library
than you can just wrap it with:
module("library")
Also its also easier to change the compiler to build, so it would be easy to try clang-cl + lld on Windows instead of the vanilla cl and link (you define this in a compile tool module that is parsed from the build manifest and used for everything, making it variable and with some flags that can be evaluated in runtime).
So back to the matter here, i wonder if part of those troubles would be solved if the IDL language used on Swift pm were more explicit about some things and powerful enough to build complex C++ codebases that happened to be dependencies of a swift project.
What I mean by "expert mode" is that you don't have to know about it to use Swift on Windows. (Maybe I should call it "intermediate mode".) The only people who have to know about it will be those building libraries (1) that aren't using SwiftPM (2), and because that's a subset of a subset I feel like it's okay if it's a bit subtle.
I don't love the idea of having to get it right, but the more I think about it the less the concrete use cases for trying to work around it hold up:
Reuse of object files between static and dynamic library targets. People probably never do this on Windows because of these issues, but it does come up sometimes on ELF and Mach-O systems. Even there, though, it has drawbacks: the very things that -static would help with. We're just living with those drawbacks today whenever we build static libraries.
Statically linking several modules' object files into a dynamic library, while still exporting their API. In that case we wouldn't want the other effects of -static anyway, so it's not actually a valid objection.
So maybe it's fine to just go with -static as a driver option, stored as a flag in swiftmodules (and probably turned into a per-item flag at the SIL level), and give up on the idea that the default "non-static" objects can still be linked statically if you want.
Yes, these flags matter purely to those that are either building Swift Package Manager itself or are building without Swift Package Manager at which point the assumption should be that they are aware of what it takes to build Swift code manually.
Right, this is not done on Windows in general. In the case that someone does that, we should treat the objects as if they were built for dynamic linking, and you live with the drawbacks. More importantly, we have given you the tools to handle this properly, and I would say that it is a safe assumption that Swift Package Manager will handle that correctly (once we have what it is nailed down) if you used Swift Package Manager. So this devolves into, you are not using the correct tools (which is the same as someone invoking the linker - do not do that, use the linker driver), use swift package manager to build.
This however is and interesting point. I didn't consider this aspect at all. How often do you expect this to occur? To educate me, when is this useful? That is, when do you want to build a library as static but re-export its interfaces? I suppose that this problem can be solved (via a DEFs file or an explicit set of /EXPORT: to the linker), so it isn't preventative, just inconvenient.
I failed to make this clear in the original post, so I apologize. This is not a new option. This is an existing option being used to inform code generation.
People keep asking about doing this on Apple platforms to break up their top-level framework into multiple modules, but still be able to distribute a single .framework bundle. (Most likely in that scenario you'd have a top-level module that re-exports the others.) In that case you're only grouping them for your client's convenience, which makes more sense for framework bundles anyway. If you're already distributing a DLL and a .lib and a .swiftmodule or .swiftinterface, distributing several of them doesn't seem like a terrible inconvenience.
And my response wasn't very clear either :-) I meant "given that we don't have a reason to change the model, we don't need to change the option, and if we're having the option, writing it into the swiftmodule seems fine".
In case it’s useful: some details for the implementation of this were discussed a while back in this thread (although no progress was made beyond that as far as I know): SPM Windows: Link issues during build - #24 by jrose
Sorry to necromance this thread, but, I think that there was one last piece here that wasn’t explicitly discussed and I believe should be as it turns out to change the behavior slightly.
I think that the key insights needed here are that there exists a module that exports the sub-modules. This would be the module which imports the sub-modules. We always have the syntax that communicates the semantics: @_exported. The final piece is that this requires compiler support to orchestrate.
When an exported static module is identified (the swift module has the static bit already), the compiler would enumerate the public interfaces and mark them for export. This would add the symbols to the import library (tbd) and the dll (dylib). The additional swift interface/module files could be either distributed or can be merged. I’m not sure which of the two is preferable. However, this would fully allow the use of the static libraries to create a shared library or even export the symbols from an executable for something like -rdynamic.
Thank you for the discussion here, I think that the final result of this is going to be a really amazing experience on windows (and the results should benefit other platforms as well). Please let me know if I’m overlooking anything as I hope to try to finish this off soon. This is one of the few last pieces that I know that needs work.
Apologies for reviving this thread. Is there a work item that tracks the static linking for windows? I took a look at the GitHub issues under Apple/swift but could not find it.
If you are looking for building and using static Swift libraries, not a static standard library, and if you are using CMake or bazel you can use static libraries.
I have been reading through the examples for quite a while.
What I never figured out is how to statically link to a Swift static library without having the source of that library.
I am building a game engine in Swift and users will need to import the engine library so they can use it. I don't want to share the sources for multiple reasons, so I need a way to just grab the module and lib files and link them.
That involves a bit more effort - namely you need to explicitly pass along the path information to the swift invocation (e.g. swift build -Xlinker -I %SourceRoot%\third_party\library\include -L%SourceRoot%\third_party\library\lib). The long term desire here is to enable this through nuget integration so that a library can be fetched from a nuget repository and integrated into the build directly.
Sure, swift-format, swift-driver, swift-package-manager, sourcekit-lsp are all great examples for this IMO. IIRC, it is best documented in swift-driver though (at least that is where I believe I had written some documentation with explanations, but maybe it was sourcekit-lsp?).
Edit: Didn’t really find what I need. I’d love to see some project that actually imports a Swift static lib without having the source. Maybe vis .swiftmodule ?
For Swift static libraries, in addition to the .lib you need the .swiftmodule directory to be redistributed. The latter is passed via -I during the build.
That I am less certain about. I don't think that the LSP uses the swiftmodule to provide completion but rather relies on the source indexing to provide that. I might be completely wrong on that part, but without teaching SK-LSP about deserialising the swiftmodule, I don't think that it would provide completion.
Yeah, I thought so. That defeats the whole purpose of static libs.
Is there any other way to link to my engine code without having to see the sources ?
Currently, I am building a dynamic lib and declare all public engine APIs with _cdecl which isn't really optimal.
You do not need to see the sources to link to the code. But you will need that for the completion through SourceKit-LSP. You could generate documentation and ship that along with the libraries to enable the use of the library.