`@convention(thin)` function pointers

Swift closures normally consist of two pointer-sized fields, one for the function entry point, and one for a reference to the captures. There are many occasions where developers want to work abstractly with functions that are known not to have any captures, and they would like to statically enforce that no capture context is allocated, and get the size and performance benefits of passing around a single, non-reference-counted function pointer. Officially, the only way to do this in Swift today is with @convention(c) function types, but these types are limited by C interop, and can only have parameters and returns of C-compatible types. Unofficially, there's @convention(thin), but it was never intended for human consumption, and attempting to use it will cause compiler crashes and miscompiles for anyone not intimately aware of Swift's implementation details. Let's talk about fixing that so that @convention(thin) can become an officially supported feature.

Making calling convention details explicit

Swift function types by default abstract away a number of calling convention details, and Swift will automatically convert between different calling conventions for the same high-level function type. However, it does this by capturing the old function value in order to create the new function value, and we can't generally do that with a function representation that doesn't support captures. The relevant details include:

  • what ownership convention is used to manage the parameter's lifetime
  • whether a parameter is passed or returned indirectly (as a pointer to a memory location containing the value) or directly (as one or more registers directly containing the value)

SE-0377 already puts the ownership convention under developer control with the borrowing and consuming modifiers, and SE-0390 made these modifiers a requirement for parameters of ~Copyable type, since the convention cannot generally be abstracted away without the ability to implicitly copy values. For similar reasons, I think we could say that a @convention(thin) function type must make its parameter and return ownership conventions explicit, as well as the indirectness of those parameters and returns. In addition to the existing consuming/borrowing/inout modifiers for specifying parameter convention, we could also introduce direct and indirect modifiers to explicitly state the indirectness of parameters and returns, and so you can state a thin function type with full specificity as:

func foo<T>(_ a: Int, _ b: T) -> T {
  return b
}

func getFoo<T>(of type: T.Type) -> @convention(thin) (direct borrowing Int, indirect borrowing T) -> indirect T {
  return foo
}

In order to maintain source compatibility with the existing feature, we can start warning if a @convention(thin) function doesn't explicitly state the directness and ownership convention of any parameters or returns.

That said, having to specify explicit directness and ownership for every parameter is very verbose. We could conceivably still default to borrowing ownership for copyable parameters. We could also decide to make one of direct or indirect implicit, and have only the other need to be stated explicitly. Only certain types support being passed or returned directly, and indirect arguments are more amenable to type punning and other shenanigans for the iron-hearted, so indirect might be a better default if we decide to go that way.

Manually setting the symbol name for Swift functions

Another unofficial feature we've had for a long time, and told people not to use if there are any alternatives for as long, is the @_silgen_name attribute, which sets the symbol name to use for the Swift entry point of a function. We ward people away from this feature for similar reasons to @convention(thin), that it isn't possible to use correctly without deep knowledge of the Swift implementation. However, with full manual control of the calling convention, and proper support for thin function pointers, a lot of that reason goes away, since you could then conceivably write in one binary:

@symbolName("foo")
public func foo(x: direct borrowing Int, y: indirect borrowing Any) { ... }

and in another binary do:

let foo = unsafeBitCast(
  dlsym(sohandle, "foo")!,
  to: (@convention(thin) (direct borrowing Int, indirect borrowing Any) -> ()).self)
foo(17, "hello")

to dynamically look up the symbol, which you can't do today.

22 Likes

Some time ago I came up with an idea of using a single pointer sized values for swift closures. Could that fly?

We cannot change the representation of closures at this point.

So this is sneakily a proposal for direct/indirect. :-) I do have some questions/concerns:

  • Does directness + ownership fully capture the calling convention for all types, and are we sure that will be true in the future?

  • Will the compiler have to diagnose types that can't be passed directly? Up until now I think that hasn't been a Sema-level (roughly, "type-checking time") concept, and maybe it's still something that can be deferred to SIL diagnostics (roughly, "flow-sensitive diagnostics that run later in compilation than type-checking"). But I do worry about people trying to use direct NonFrozenStruct outside the original module and getting weird errors from the compiler, and I myself don't know whether direct Any or indirect NSObject are valid concepts or not.

  • Having @symbolName without @cdecl worries me: I think people will start exposing Swift functions to C [even more than they already do], and have things mostly work but then cause extremely weird errors in the corner cases. (But while @cdecl isn't that much more work, it's a lot more than this…)

  • The dlsym code is only correct if both sides use the same ABI, which could be a cause of weird issues on non-stable-ABI platforms if people don't think about it. But there is an unsafeBitCast there, so maybe it's ""fine"".

11 Likes

I will have to read again this when I am less tired but I'm glad to see this feature.

This is strictly about function pointers, correct? Are there any implications for methods accessed/passed as closures/function pointers?

I will start by saying I don’t have full enough context to understand which problem these particular direct/indirect keywords are solving, but with all these additional keywords popping up in the last couple of years, Swift is starting to feel like C++.

My gut feeling on this proposal is therefore a -1. I feel like Swift should be Swift. If we want to use C++ features, we should just use C++ (there is active work on that, which I welcome and support). I don’t really buy the argument of “just don’t use it” in the case of additional keywords.

2 Likes

So there are a couple of calling convention variations I can think of that aren't entirely addressed by this:

  • Tuples may or may not be exploded into individual parameters or returns depending on the abstraction level. We could however say that, if you mark a tuple with explicit directness, then it's always treated as a single aggregate, and to match a convention that destructures the tuple, you can specify it as individual arguments.
  • Methods and closure invocation functions use the @convention(method) calling convention in SIL, which puts the self parameter in the special context register and preserves it across calls. It could be useful to expose @convention(method) as a surface-language-level convention too, perhaps, in order to allow for things like dynamically constructing protocol conformances, splitting a closure into separate context reference and invocation function values, and so on.

Yeah, we'd have to formalize the rules for what subset of types are allowed to be passed direct. That would at least include:

  • a struct or enum that's either @frozen or in the same module, and whose elements are all either indirect or can themselves be passed directly
  • an object reference of class type
  • a generic type or existential with an AnyObject constraint (either directly or implied)

At the risk of tying the feature to a boat anchor, maybe symbolName could take on the responsibliity for manually specifying both C and Swift symbol names, since a lot of the additional modifiers such a feature might need, like visibility, section, usedness, and so on, could apply independently to the Swift and C entry points. So you could end up writing something like

@symbolName(swift: "swift_foo", visibility: internal)
@symbolName(c: "c_foo", visibility: public, section: "__TEXT,__fooplugn", used)
func foo(...)

if you wanted to export a Swift entry point internally for some other Swift code to indirectly link against, while exporting the C entry point to be consumed by some plugin or other interface that needs it.

Maybe we could have a way to still semi-mangle the name to include the expected function type of the entry point, so that if you try to look it up with a different type, the lookup still fails.

1 Like

We also currently have @_expose(Cxx, "external_symbol_name") for C++ interop, so we should really aim to unify all of these under a single umbrella.

8 Likes

Note that @_expose doesn't specify the symbol name, it specifies the name of the C++ declaration that represents the original Swift declaration in C++, similar to the NS_SWIFT_NAME macro for Objective-C interoperability. We also don't require it anymore for exposing things to C++, so it might not make sense to unify it with symbolName.

3 Likes

Yeah, all of the existing attributes for exporting symbols to various FFI are different in very subtle ways, which I believe are the following:

  1. func f() without attributes:

    • Symbols emitted into binary: One—a normal Swift function named with the mangling of f.
    • Decls in generated header: N/A without C++ interop. With C++ interop, an inline trampoline function named module_name::f is printed that calls the Swift f.
    • Swift visibility for which decls are generated in the header: public
  2. @_silgen_name("g") func f()

    • Symbols emitted into binary: One—a normal Swift function, but named g instead of the mangling of f.
    • Decls in generated header: Same as #1.
    • Swift visibility for which decls are generated in the header: Same as #1.
  3. @_cdecl("g") func f()

    • Symbols emitted into binary: Two—a normal Swift function named with the mangling of f, and a trampoline C function g that calls f.
    • Decls in generated header: An extern "C" function declaration for the trampoline is printed. If C++ interop is enabled, a C++ inline trampoline is also printed, but it looks like it invokes the C trampoline instead of the Swift symbol? (generated header in output window)
    • Swift visibility for which decls are generated in the header: public, internal
  4. @_expose(Cxx, "g") func f()

    • Symbols emitted into binary: One—a normal Swift function named with the mangling of f.
    • Decls in generated header: N/A without C++ interop. With C++ interop, an inline trampoline function named module_name::g is printed that calls the Swift f.
    • Swift visibility for which decls are generated in the header: public
  5. @objc(g) func f() (where f is a method in a class):

    • Symbols emitted into binary: Two—a normal Swift function named with the mangling of f, and a trampoline Obj-C method named g that calls f.
    • Decls in generated header: An Objective-C method decl for the trampoline g is printed.
    • Swift visibility for which decls are generated in the header: public, internal

So I agree that there's a bit of a spectrum here; @_silgen_name affects the name of the original symbol, @_cdecl and @objc leave the original symbol alone and introduce a new one for external clients, and @_expose(Cxx, ...) only affects the generated header.

I don't think that's a reason to avoid unifying these concepts altogether (although the ship has already sailed for @objc, which is both officially supported and very widely used). If we claim that @symbolName (or whatever it's called) "defines the name that foreign languages interact with a symbol", then I think we could argue that @_cdecl and @_expose(Cxx, ...) fit that description even if C++ interop doesn't require it, and I'd feel comfortable saying that @symbolName(swift, ...) is defining the name that foreign languages (or a foreign process via dladdr) which aren't using Swift's C++ interop use to interact with a function that still uses a Swift calling convention, since those may be the only times you'd want to rename the Swift symbol. (I suppose maintaining ABI compatibility while renaming an API is another one.)

10 Likes

Being generic, function foo also accepts a hidden parameter for metatype of T.

We could transform it’s signature to make it explicit:

let f: @convention(thin) (String.Type, direct borrowing Int, indirect borrowing String) -> indirect String = getFoo(of: String.self)

But that won’t work if generic signature involves protocol conformances:

func bar<T: Hashable>(_ x: T) {}
let b: @convention(thin) (String.Type, {{{Conformance of String to Hashable?}}}, indirect borrowing String) = bar
1 Like