Swift Package Manager: Linker Flags


(Ilija Tovilo) #1

Happy new year everyone!

I’m writing a wrapper around the LLVM-C API for Swift and thought it’d be fun to use the Swift Package Manager.
So I created a repository for the module.modulemap that includes the relevant .h files (as instructed in Documentation/SystemModules.md in the GitHub repository).

The package itself compiles fine and building the project that includes it works too, except that it doesn’t link.
The problem is that you have to pass some LLVM linker flags and I have no idea how to do that with the Swift Package Manager.

I’ve searched the tutorials, documentation and the source code but couldn’t find a solution.
Is there a way to add linker flags / compile flags to your Package.swift file?

It would be helpful to pass those flags manually, at least until the package manager is mature enough to handle those things on its own.

Thanks for the help!


(Daniel Dunbar) #2

You can't do this via the package manager, but you can include "link" declarations in the module map itself which specify additional linker arguments to plumb through when that module is used. See:
  http://clang.llvm.org/docs/Modules.html#link-declaration

Here is a concrete example, which is how Swift knows to automatically link libpthread and libdl when Glibc is used:
  https://github.com/apple/swift/blob/master/stdlib/public/Glibc/module.map.in

- Daniel

···

On Jan 1, 2016, at 4:48 PM, Ilija Tovilo via swift-users <swift-users@swift.org> wrote:

Happy new year everyone!

I’m writing a wrapper around the LLVM-C API for Swift and thought it’d be fun to use the Swift Package Manager.
So I created a repository for the module.modulemap that includes the relevant .h files (as instructed in Documentation/SystemModules.md in the GitHub repository).

The package itself compiles fine and building the project that includes it works too, except that it doesn’t link.
The problem is that you have to pass some LLVM linker flags and I have no idea how to do that with the Swift Package Manager.

I’ve searched the tutorials, documentation and the source code but couldn’t find a solution.
Is there a way to add linker flags / compile flags to your Package.swift file?

It would be helpful to pass those flags manually, at least until the package manager is mature enough to handle those things on its own.

Thanks for the help!

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


#3

Hi all,

I've also been having trouble getting a Swift interface to LLVM's C API. The basic gist is here:

https://gist.github.com/stephencelis/5de13eeb9743e7a3aed3

I've:

- Installed LLVM via homebrew, so it lives in "/usr/local/opt/llvm" (I've also built LLVM myself and have the same ).
- Passed "-I" and "-L" to send includes/lib paths to the "-Xcc" and "-Xlinker" flags. (Can a module map or package be configured directly with these paths? Or do all dependent projects need to use these flags, as well?
- Added many more LLVM headers/links to the module map and continued to have the same issue.

The linker's still having trouble. I'm probably missing something very basic.

Stephen

···

On Jan 4, 2016, at 1:08 PM, Daniel Dunbar via swift-users <swift-users@swift.org> wrote:

You can't do this via the package manager, but you can include "link" declarations in the module map itself which specify additional linker arguments to plumb through when that module is used. See:
  http://clang.llvm.org/docs/Modules.html#link-declaration

Here is a concrete example, which is how Swift knows to automatically link libpthread and libdl when Glibc is used:
  https://github.com/apple/swift/blob/master/stdlib/public/Glibc/module.map.in

- Daniel

On Jan 1, 2016, at 4:48 PM, Ilija Tovilo via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

Happy new year everyone!

I’m writing a wrapper around the LLVM-C API for Swift and thought it’d be fun to use the Swift Package Manager.
So I created a repository for the module.modulemap that includes the relevant .h files (as instructed in Documentation/SystemModules.md in the GitHub repository).

The package itself compiles fine and building the project that includes it works too, except that it doesn’t link.
The problem is that you have to pass some LLVM linker flags and I have no idea how to do that with the Swift Package Manager.

I’ve searched the tutorials, documentation and the source code but couldn’t find a solution.
Is there a way to add linker flags / compile flags to your Package.swift file?

It would be helpful to pass those flags manually, at least until the package manager is mature enough to handle those things on its own.

Thanks for the help!

_______________________________________________
swift-users mailing list
swift-users@swift.org <mailto:swift-users@swift.org>
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Max Howell) #4

Possibly LLVMGetGlobalContext is not a real symbol?

If it is a #define then it will not always get mapped by swiftc to the same symbol in Swift.

···

Hi all,

I've also been having trouble getting a Swift interface to LLVM's C API. The basic gist is here:

https://gist.github.com/stephencelis/5de13eeb9743e7a3aed3

I've:

- Installed LLVM via homebrew, so it lives in "/usr/local/opt/llvm" (I've also built LLVM myself and have the same ).
- Passed "-I" and "-L" to send includes/lib paths to the "-Xcc" and "-Xlinker" flags. (Can a module map or package be configured directly with these paths? Or do all dependent projects need to use these flags, as well?
- Added many more LLVM headers/links to the module map and continued to have the same issue.

The linker's still having trouble. I'm probably missing something very basic.

Stephen

On Jan 4, 2016, at 1:08 PM, Daniel Dunbar via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

You can't do this via the package manager, but you can include "link" declarations in the module map itself which specify additional linker arguments to plumb through when that module is used. See:
  http://clang.llvm.org/docs/Modules.html#link-declaration

Here is a concrete example, which is how Swift knows to automatically link libpthread and libdl when Glibc is used:
  https://github.com/apple/swift/blob/master/stdlib/public/Glibc/module.map.in

- Daniel

On Jan 1, 2016, at 4:48 PM, Ilija Tovilo via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

Happy new year everyone!

I’m writing a wrapper around the LLVM-C API for Swift and thought it’d be fun to use the Swift Package Manager.
So I created a repository for the module.modulemap that includes the relevant .h files (as instructed in Documentation/SystemModules.md in the GitHub repository).

The package itself compiles fine and building the project that includes it works too, except that it doesn’t link.
The problem is that you have to pass some LLVM linker flags and I have no idea how to do that with the Swift Package Manager.

I’ve searched the tutorials, documentation and the source code but couldn’t find a solution.
Is there a way to add linker flags / compile flags to your Package.swift file?

It would be helpful to pass those flags manually, at least until the package manager is mature enough to handle those things on its own.

Thanks for the help!

_______________________________________________
swift-users mailing list
swift-users@swift.org <mailto:swift-users@swift.org>
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org <mailto:swift-users@swift.org>
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


#5

Hm, it definitely appears to be there:

  ~ $ nm -g /usr/local/opt/llvm/lib/libLLVMCore.a | grep LLVMGetGlobalContext
  0000000000000081 T _LLVMGetGlobalContext

And the matching header definition:

  ~ $ grep LLVMGetGlobalContext /usr/local/opt/llvm/include/llvm-c/Core.h
  LLVMContextRef LLVMGetGlobalContext(void);

Stephen

···

On Feb 22, 2016, at 1:06 PM, Max Howell <max.howell@apple.com> wrote:

Possibly LLVMGetGlobalContext is not a real symbol?

If it is a #define then it will not always get mapped by swiftc to the same symbol in Swift.


(Greg Parker) #6

Did you forget to actually link to libLLVMCore? I see your -L flags in that build log, but I don't see anything like `-Xlinker -lLLVMCore`.

···

On Feb 22, 2016, at 7:44 AM, Stephen Celis via swift-users <swift-users@swift.org> wrote:

Hi all,

I've also been having trouble getting a Swift interface to LLVM's C API. The basic gist is here:

https://gist.github.com/stephencelis/5de13eeb9743e7a3aed3

I've:

- Installed LLVM via homebrew, so it lives in "/usr/local/opt/llvm" (I've also built LLVM myself and have the same ).
- Passed "-I" and "-L" to send includes/lib paths to the "-Xcc" and "-Xlinker" flags. (Can a module map or package be configured directly with these paths? Or do all dependent projects need to use these flags, as well?
- Added many more LLVM headers/links to the module map and continued to have the same issue.

The linker's still having trouble. I'm probably missing something very basic.

--
Greg Parker gparker@apple.com Runtime Wrangler


#7

Looking at the definition in the header file, it looks that it doesnt
declare any export definition for the functions like:

__attribute__((__visibility__("default")))

So maybe it would only work via static linking?

Im working in something here where theres a binding for a dinamic
library, and given is a dynamic linking
theres o way the swift runtime will "see" the symbols without it.

[snip]

  ~ $ nm -g /usr/local/opt/llvm/lib/libLLVMCore.a | grep LLVMGetGlobalContext
  0000000000000081 T _LLVMGetGlobalContext

Thanks for the reply! The `-g` flag on `nm` should only print out external symbols, so I unfortunately don't think that's it :confused:

I have a feeling there's a very basic linker issue I'm overlooking.

Stephen

···

On Feb 23, 2016, at 4:35 PM, Fabio Kaminski <fabiokaminski@gmail.com> wrote:


#8

Did you forget to actually link to libLLVMCore? I see your -L flags in that build log, but I don't see anything like `-Xlinker -lLLVMCore`.

Huh. I was under the impression that the module map "link" declaration was applied. Is that not the case? Should it be? Should I file a bug?

I seem to be making headway, though this means my Swift target becomes even more problematic for dependent projects to use (with a rapidly growing Makefile).

Use `nm -m` to get a readable description of symbol attributes like visibility. In this case "T" is an exported symbol in the __TEXT segment so visibility should not be a problem. If the symbol were not exported then it would be labeled "t".

Ah, much nicer. Thanks for the tip!

Stephen

···

On Feb 23, 2016, at 6:43 PM, Greg Parker <gparker@apple.com> wrote:


(Greg Parker) #9

Use `nm -m` to get a readable description of symbol attributes like visibility. In this case "T" is an exported symbol in the __TEXT segment so visibility should not be a problem. If the symbol were not exported then it would be labeled "t".

···

On Feb 23, 2016, at 1:51 PM, Stephen Celis via swift-users <swift-users@swift.org> wrote:

On Feb 23, 2016, at 4:35 PM, Fabio Kaminski <fabiokaminski@gmail.com> wrote:

Looking at the definition in the header file, it looks that it doesnt
declare any export definition for the functions like:

__attribute__((__visibility__("default")))

So maybe it would only work via static linking?

Im working in something here where theres a binding for a dinamic
library, and given is a dynamic linking
theres o way the swift runtime will "see" the symbols without it.

[snip]

~ $ nm -g /usr/local/opt/llvm/lib/libLLVMCore.a | grep LLVMGetGlobalContext
0000000000000081 T _LLVMGetGlobalContext

Thanks for the reply! The `-g` flag on `nm` should only print out external symbols, so I unfortunately don't think that's it :confused:

--
Greg Parker gparker@apple.com Runtime Wrangler


#10

Thanks for following up :slight_smile:

1 - For me at least the "link" directive works only for shared libs,
and given you are looking into the symbols of the .a file and not the
.dylib or .so, that its probably what the modulemap will try to link
against?

Ah, interesting! Is there a reason for this limitation? Could a bug be filed for a swiftpm enhancement that automatically adds `-l` flags for whatever is included as a `link` in the module map?

I think the [extern_c] directive maybe is the magic sauce that will
try to link the symbols using the correct mangling technique "_" ?

Unfortunately [extern_c] doesn't seem to have any effect on linking, though it appears to be the correct attribute according to http://clang.llvm.org/docs/Modules.html#module-declaration. I was able to fix the problem, though, with this Makefile in my dependent project:

https://gist.github.com/stephencelis/7f76f8d26a1df64ee3ff#file-llvm_makefile-mak

It's an unfortunate limitation. Requiring dependent projects to set certain `-Xcc` and `-Xlinker` flags makes the parent CLLVM dependency a lot less easy to use. I'm not sure if there have been evolution, bug, or internal discussions around this, though. At least I'm unblocked in the meantime. Thanks again!

···

On Feb 24, 2016, at 8:46 PM, Fabio Kaminski <fabiokaminski@gmail.com> wrote:

--
Stephen


(Max Howell) #11

Did you forget to actually link to libLLVMCore? I see your -L flags in that build log, but I don't see anything like `-Xlinker -lLLVMCore`.

Huh. I was under the impression that the module map "link" declaration was applied. Is that not the case? Should it be? Should I file a bug?

Provided that the module map file specifies link “LLVMCore” then yes, you should not need to do this. If you do, then it’s a bug.

Try running `swift build -vv` to get full instantiation information.


(Max Howell) #12

Thanks for following up :slight_smile:

1 - For me at least the "link" directive works only for shared libs,
and given you are looking into the symbols of the .a file and not the
.dylib or .so, that its probably what the modulemap will try to link
against?

Ah, interesting! Is there a reason for this limitation? Could a bug be filed for a swiftpm enhancement that automatically adds `-l` flags for whatever is included as a `link` in the module map?

This is a limitation of module maps, so any improvement would have to go into clang.

I think the [extern_c] directive maybe is the magic sauce that will
try to link the symbols using the correct mangling technique "_" ?

Unfortunately [extern_c] doesn't seem to have any effect on linking, though it appears to be the correct attribute according to http://clang.llvm.org/docs/Modules.html#module-declaration. I was able to fix the problem, though, with this Makefile in my dependent project:

https://gist.github.com/stephencelis/7f76f8d26a1df64ee3ff#file-llvm_makefile-mak

It's an unfortunate limitation. Requiring dependent projects to set certain `-Xcc` and `-Xlinker` flags makes the parent CLLVM dependency a lot less easy to use. I'm not sure if there have been evolution, bug, or internal discussions around this, though. At least I'm unblocked in the meantime. Thanks again!

-Xcc/-Xlinker are strictly work-ardoun methods so that people can make progress with swift build until we have real solutions.

Watch for my proposal to improve module map support coming to swift-evolution at some point.

···

On Feb 24, 2016, at 6:03 PM, Stephen Celis via swift-users <swift-users@swift.org> wrote:

On Feb 24, 2016, at 8:46 PM, Fabio Kaminski <fabiokaminski@gmail.com <mailto:fabiokaminski@gmail.com>> wrote: