SPM and dynamic linking

Hi,

Probably this is a misunderstanding on my part, but I'd like to convince SPM to do more dynamic library generation/linking, and am finding it hard to do so.

I know I can explicit request a dynamic library in my own Package.swift using "type: .dynamic,", and that works fine and produces the .so as I'd expect (its a .so as I'm on linux for this).

However, if I clone an external package that doesn't specify either static or dynamic, only the static .swiftmodule is generated - is there any way to request a dynamic library without changing the Package.swift?

Similarly though libraries can request dynamic linking, an executable pulling in a package (which again doesn't specify dynamic or static, leaving it to its 'client' AIUI), also gets statically linked, and I can't see an option to request dynamic linking.

Any help much appreciated,
Thanks,
Jon

1 Like

I started looking into the SPM source, and it turned out to be not too hard to get at least a minimum of what I was looking for working - I have it both producing a dynlib for a external package, and also dynamic linking a sample executable. Is there likely to be any interest in accepting these changes if I looked to tidy them up further?

Thanks,
Jon

I'd definitely be interested in seeing that merged - static linking is still not properly available on Windows (mostly from the SPM side). So, getting this support would be great.

I've pushed what I have so far to my fork.

It needs more work - at the least, more comments, documentation and testing.

However if anyone has any comments at this stage, or wants to give it a try, that'd be great. In particular, whether the approach of keeping things as 'automatic' libraries, but modifying the build plan to create products for them, is correct?

Thanks,
Jon

I definitely would like to take a closer look and possibly help out if it looks promising, but I do not have time at this very moment. I am posting this to let you know so that if I havenā€™t remembered to get back to you in a week or so, you can ping me with a reminder.

Okay, so I see you have added --prefer-dynamic, and you have enabled processing of .automatic in several places. (I think it would be better to resolve early whether to divert it into .static or .dynamic mode and then let it simply flow through all the same code branches as the explicit options, but we can get to that later.)

It is not yet clear to me what the precise intended meaning of --prefer-dynamic is. What is supposed to be dynamic?

  • Are the toolchain libraries dynamic?
  • Are dependency products dynamic? All of them, or just conjunction nodes where it would be useful?
  • Should it split .automatic libraries into targets for more fineā€grained heuristics?
  • What about libraries in the same package?

Some other important questions:

  • Should it (or a companion flag) override explicit .static?
  • What happens when an explicit .static ends up in two .dynamic libraries linked into the same executable?
  • How does this interplay with interā€module optimization?
  • How much of this would be better as heuristics improvements for the default instead of an explicit option?
  • Are we in over our heads and all we really wanted is to produce a .so or .a for a topā€level .automatic library that we then expect to use outside of SwiftPM?

The answers to these questions affect whether it will need to go through the full evolution process and in turn, how much work would be ahead of you.

Thanks for taking the time to look at this, these are all great questions. Let me have a first swing at answering them:

(Apologies if answering inline ends up confusing, but it seemed the easiest way to try and make sure I'd looked at all the questions.)

Yes, that is what I meant. They originate from the Swift toolchain and not from the dependency graph. I ask because there has been recent discussion about changing the defaults for these, which might have some overlap with what we are doing here.

Right now a manifest cannot declare a dependency from a target on a product in the same package. It is already confusing to some that a .dynamic product creates a .so, but a neighbouring executable statically links the targets instead. (It is less confusing when the target and product names donā€™t match, but in many cases they do.) The flag controls you propose would make this quirk more visible and possibly more painful. It might need to be fixed/rethought first.

It appears to happen far more often than necessary. SwiftSyntax itself is an example. I suspect this is due to autocomplete presenting the field for type, and users reaching for .static without knowing about the .automatic enum case. (They are not even the same; .static creates a .a in the products directory, automatic does not.)

That is only part of what I mean. Already static linking can do some dead stripping, specialization, etc. and it is getting better with time. Theoretically the same optimizations ought to be possible for dynamicā€linked libraries built together in the dependency graph, but I do not think any of this has been implemented yet. So is the worse size and speed acceptable in the meantime, and does it need to be communicated to users somehow? Also, if a package is an executableā€dynamic library pair, where one uses the other, should the dynamic library be stripped down to only what the executable actually uses or not? Do you know of any other ramifications I have not thought of yet?

Terms of Service

Privacy Policy

Cookie Policy