Compiling C-usable objects/shared objects


(Daniel Farina) #1

I've been playing with recent versions of Swift on Linux and think the
whole thing is rather neat. I'm most curious about how swift programs might
be embeddable in C projects.

I found a number of documentation artifacts about calling C functions and
providing Swift callbacks to C, but none about how to disable mangling of
symbols or doing whatever else was necessary to allow C programs to link a
Swift .o/.so.

Also ambiguous to me are runtime requirements, particularly in terms of
background threads and memory management. Many of the target C programs I
have their in mind have their own memory and concurrency management
strategies, e.g. extensions for Python, Ruby, Postgres...

Is this something that works? Or could be made to work? Has a lot of subtle
problems besides throwing "convention(c)" on some functions and turning off
mangling?

Thanks for considering my questions.


(Joe Groff) #2

Swift uses its own calling convention for Swift-to-Swift calls, so it's not a simple case of disabling mangling. We don't have a supported solution yet for exporting symbols with C linkage and calling conventions, but in master there is a prototype of an attribute, @_cdecl("foo"), which can be used on a function to export it as a C-callable function named foo. For instance:

  @_cdecl("module_foo")
  func foo(x: Int) -> Int { return x + 1 }

would be usable from C or ObjC as:

  int module_foo(int x);

Note that you still can't define C-compatible struct types from within Swift; they must be defined in C and imported into Swift. Regarding runtime requirements, Swift requires its runtime to function, and there's a requirement that any Swift code that interacts must share the same runtime within a process. Since Swift is not yet ABI-stable, if your primary intent is to provide a shared library for use from C, you may want to statically link the Swift standard library into your .so, and only export C symbols from it for external use. That should keep the "Swiftiness" of the shared library as an internal implementation detail.

-Joe

···

On Apr 11, 2016, at 11:36 AM, Daniel Farina via swift-users <swift-users@swift.org> wrote:

I've been playing with recent versions of Swift on Linux and think the whole thing is rather neat. I'm most curious about how swift programs might be embeddable in C projects.

I found a number of documentation artifacts about calling C functions and providing Swift callbacks to C, but none about how to disable mangling of symbols or doing whatever else was necessary to allow C programs to link a Swift .o/.so.

Also ambiguous to me are runtime requirements, particularly in terms of background threads and memory management. Many of the target C programs I have their in mind have their own memory and concurrency management strategies, e.g. extensions for Python, Ruby, Postgres...

Is this something that works? Or could be made to work? Has a lot of subtle problems besides throwing "convention(c)" on some functions and turning off mangling?

Thanks for considering my questions.


(Daniel Farina) #3

>
> I've been playing with recent versions of Swift on Linux and think the
whole thing is rather neat. I'm most curious about how swift programs might
be embeddable in C projects.
>
> I found a number of documentation artifacts about calling C functions
and providing Swift callbacks to C, but none about how to disable mangling
of symbols or doing whatever else was necessary to allow C programs to link
a Swift .o/.so.
>
> Also ambiguous to me are runtime requirements, particularly in terms of
background threads and memory management. Many of the target C programs I
have their in mind have their own memory and concurrency management
strategies, e.g. extensions for Python, Ruby, Postgres...
>
> Is this something that works? Or could be made to work? Has a lot of
subtle problems besides throwing "convention(c)" on some functions and
turning off mangling?
>
> Thanks for considering my questions.

Swift uses its own calling convention for Swift-to-Swift calls, so it's
not a simple case of disabling mangling. We don't have a supported solution
yet for exporting symbols with C linkage and calling conventions, but in
master there is a prototype of an attribute, @_cdecl("foo"), which can be
used on a function to export it as a C-callable function named foo. For
instance:

        @_cdecl("module_foo")
        func foo(x: Int) -> Int { return x + 1 }

would be usable from C or ObjC as:

        int module_foo(int x);

Very cool. The reference to "_cdecl" will help me out with searching the
source, too.

Note that you still can't define C-compatible struct types from within
Swift; they must be defined in C and imported into Swift. Regarding runtime
requirements, Swift requires its runtime to function, and there's a
requirement that any Swift code that interacts must share the same runtime
within a process. Since Swift is not yet ABI-stable, if your primary intent
is to provide a shared library for use from C, you may want to statically
link the Swift standard library into your .so, and only export C symbols
from it for external use. That should keep the "Swiftiness" of the shared
library as an internal implementation detail.

Yes, that's exactly the goal. One follow-up question: I'm not so much
concerned about there existing some runtime, but rather what is in it. In
particular, threads, memory management that is hard to integrate with other
programs, or something else that you might think an impediment to embedding
in other programs that have concurrency or memory models.

Thanks for your help.

···

On Mon, Apr 11, 2016 at 11:47 AM Joe Groff <jgroff@apple.com> wrote:

> On Apr 11, 2016, at 11:36 AM, Daniel Farina via swift-users < > swift-users@swift.org> wrote:


(Joe Groff) #4

Swift's runtime consists mostly of the runtime database for generic type metadata and associated casting/querying logic, and support for reference counting and allocation of Swift classes and other heap objects. The type database takes up memory, but is all lazily initialized and concurrency-safe so should be mostly invisible to external code. Interfacing Swift reference counting with other memory management systems has many of the same challenges as would other reference counting schemes, such as ObjC, COM, or GObject, but should be straightforward compared to, say, a GC system where pinning objects is necessary while they're accessible from C.

-Joe

···

On Apr 11, 2016, at 11:57 AM, Daniel Farina <daniel@fdr.io> wrote:

On Mon, Apr 11, 2016 at 11:47 AM Joe Groff <jgroff@apple.com> wrote:

> On Apr 11, 2016, at 11:36 AM, Daniel Farina via swift-users <swift-users@swift.org> wrote:
>
> I've been playing with recent versions of Swift on Linux and think the whole thing is rather neat. I'm most curious about how swift programs might be embeddable in C projects.
>
> I found a number of documentation artifacts about calling C functions and providing Swift callbacks to C, but none about how to disable mangling of symbols or doing whatever else was necessary to allow C programs to link a Swift .o/.so.
>
> Also ambiguous to me are runtime requirements, particularly in terms of background threads and memory management. Many of the target C programs I have their in mind have their own memory and concurrency management strategies, e.g. extensions for Python, Ruby, Postgres...
>
> Is this something that works? Or could be made to work? Has a lot of subtle problems besides throwing "convention(c)" on some functions and turning off mangling?
>
> Thanks for considering my questions.

Swift uses its own calling convention for Swift-to-Swift calls, so it's not a simple case of disabling mangling. We don't have a supported solution yet for exporting symbols with C linkage and calling conventions, but in master there is a prototype of an attribute, @_cdecl("foo"), which can be used on a function to export it as a C-callable function named foo. For instance:

        @_cdecl("module_foo")
        func foo(x: Int) -> Int { return x + 1 }

would be usable from C or ObjC as:

        int module_foo(int x);

Very cool. The reference to "_cdecl" will help me out with searching the source, too.

Note that you still can't define C-compatible struct types from within Swift; they must be defined in C and imported into Swift. Regarding runtime requirements, Swift requires its runtime to function, and there's a requirement that any Swift code that interacts must share the same runtime within a process. Since Swift is not yet ABI-stable, if your primary intent is to provide a shared library for use from C, you may want to statically link the Swift standard library into your .so, and only export C symbols from it for external use. That should keep the "Swiftiness" of the shared library as an internal implementation detail.

Yes, that's exactly the goal. One follow-up question: I'm not so much concerned about there existing some runtime, but rather what is in it. In particular, threads, memory management that is hard to integrate with other programs, or something else that you might think an impediment to embedding in other programs that have concurrency or memory models.


(Daniel Farina) #5

That's wonderful. Okay, I'll play with this. Thanks.

···

On Mon, Apr 11, 2016 at 12:04 PM Joe Groff <jgroff@apple.com> wrote:

> Yes, that's exactly the goal. One follow-up question: I'm not so much
concerned about there existing some runtime, but rather what is in it. In
particular, threads, memory management that is hard to integrate with other
programs, or something else that you might think an impediment to embedding
in other programs that have concurrency or memory models.

Swift's runtime consists mostly of the runtime database for generic type
metadata and associated casting/querying logic, and support for reference
counting and allocation of Swift classes and other heap objects. The type
database takes up memory, but is all lazily initialized and
concurrency-safe so should be mostly invisible to external code.
Interfacing Swift reference counting with other memory management systems
has many of the same challenges as would other reference counting schemes,
such as ObjC, COM, or GObject, but should be straightforward compared to,
say, a GC system where pinning objects is necessary while they're
accessible from C.