Formalizing @cdecl

[uliwitness (Not a Kitteh)]

The need to distinguish between ObjC and C mentioned leads me to the conclusion that the more generic attribute name would be better. @export(c,"foo") and @export(objc,"foo") would side-step the issue elegantly, and allow for growth.

For the case where there's no un-ambiguous translation from Swift to the target language for a type (like Bool), I'd say add a qualifier to the argument that has the issue. Otherwise you can either royally shoot yourself in the foot if someone inserts a parameter, and you end up off-by-one. That would also make error reporting easier.

Maybe:

@export(objc) func foo(_ : @export(objc,int) Bool)

To leave the option of annotating one function for several languages if needed? Similarly, this might be used to control whether something becomes a block or a function in C.

Though personally, I would say if you want to export to multiple languages, create several functions. Readability with annotations for several languages would just decrease too much. So maybe we could leave away the objc in the second case:

@export(objc) func foo(_ : @export(int) Bool)

-emit-objc-header - what about -emit-cfamily-header ?

[Geordie_J (ephemer)]

Thanks @jrose first of all for pushing on this topic. I think it’s highly valuable for Swift 6's stated goal of better cross-platform compatibility.

My team and I have been using @_cdecl and @_silgen_name since Swift was open sourced, and couldn’t do without them for our purposes (interop with Java via the JNI, interop with Python before PythonKit was a thing, etc.)

My initial reaction was to agree with some voices here moving to separate setting the calling convention and changing the symbol name. I’m not sure of its current status but we have had to use @_silgen_name instead of @_cdecl in the past due to the latter's lack of availability on Linux. When we did, we had to make use of the lucky coincidence that the Swift calling convention was compatible with the C one for (off the top of my head) small numbers of arguments / small-sized structs. So, at first glance, it seems obvious that we need both.

Thinking about it more though, all we need to care about IMHO is that our function can be called from another language that can call C. The ability to set a custom symbol name, rather than letting the Swift compiler pick a reasonable default, is functionality that IMO can be left to another proposal. For now it seems to me that we should aim to implement a combined attribute that sets the calling convention and mangles the name in a way reasonable for that calling convention. For the sake of argument I would name these @callableBy(callingConventionName) and the optional, currently out of scope, functionality @renameSymbol(newSymbolName).

Even though for me it'd be enough to only support C via @callableBy(c) for now, I do think it’d be worth allowing expansion to other calling conventions and name mangling schemes in the future. For example @callableBy(cpp) func myFunc(x: Int) -> Int should be enough to set the calling convention and mangle the name in a way expected by C++, so that C++-based callers could just declare that a function called myFunc exists with the argument/return types defined in Swift - without the developer on either side of that exchange needing to know how C++ mangles its names. Given my lack of C++ knowledge, I will stop speculating about that language specifically here- I’m only picking on it because I’m aware it uses another name mangling scheme. Clearly, the discussion of how to convert types here is relevant and necessary too, but I'm not versed in it well enough to add a lot to that side of things.

Of course, the thoughts I gathered here all fall down if calling convention is orthogonal to the name mangling scheme. So I'm interested to hear what others have to say about this, in particular @compnerd, because he brought up various calling conventions used under Windows. If we were to set any one of those calling conventions, would it be clear how to mangle the name? Or would the possibility exist that it could be a C++ or a C function for symbol naming purposes, for example?

[compnerd (Saleem Abdulrasool)]

All the cases that I brought up were primarily focused on C. For C++ the situation is far more murky.

The calling convention and name decoration requires a third piece of information which is missing. The current piece which seems best to me: @extern(c++) @convention(thiscall) doesn’t tell me everything I need. Yes, this calls a function that has an implicit this parameter, that is it is a class bound method, with a C++ interop, but it fails to inform me of the ABI. Did you mean MSVC? Itanium? GNU? (Yes, GCC had a custom ABI before itanium).

Would it be possible to augment it? Yes, you could spell this @extern(c++) @convention(thiscall) @abi(msvc), but I suspect that people may balk at that. We could default the ABI based on the target triple and allow it to be overridden to reduce the complexity some.

This extends really well though: from the basic @extern(python) through @extern(c) @convention(vectorcall) to @extern(c++) @convention(thiscall) @abi(msvc).

One argument that I can see the flip side for: making the linkage cover the ABI aspect as well as @extern(c++-msvc). My desire for the split with the additional attribute is that we don’t want users to understand the ABI unless it is material (i.e. Darwin doesn’t gain anything from c++-itanium over c++) and it simplifies processing the string in the compiler.

[compnerd (Saleem Abdulrasool)]

Follow up notes:

• Windows supports MSVC and itanium ABIs for C++ (windows-msvc and windows-itanium)
• Cygwin and MinGW use a variant of the itanium ABI
• ARM Linux uses a variant of the itanium ABI
• Darwin uses a variant of the itanium ABI
• There are alternative ABIs like HP which have not been enumerated here for brevity
• thiscall annotation for class bound functions is important for differentiation from freestanding functions
• annotation for global variables is required for some platforms. e.g. MSVC ABI decorates global variables in C++ which itanium does not:

int j;

will be decorated as j on itanium but will be decorated as ?j@@3HA on MSVC.

namespace n {
void function(int, int);
void function(int, void *);
struct S {
  static void function(int);
  void function(int, int);
};
}

specifies 4 functions with different decorations and calling conventions.

I am citing prior art for the preference for the @extern spelling from C++:

extern "C" { void function(void); }
extern "C++" { void function(int); }
extern "Java" { void function(int, int); }

C++ 9.11 [dcl.link] permits extension for other languages, going so far as to recommend non-capitalized languages. The provided Java external linkage example is valid with GCC (which enabled interop with gcj).

[Geordie_J (ephemer)]

Forgive my ignorance but would would @extern(python) actually do? By that I mean, what would we expect the compiler (etc.) to do differently when it sees this attribute. Maybe I shouldn’t pick on this single example too closely, but I think it would help the discussion as to the scope of the proposed feature at hand. Using CTypes in Python allows you to call C(-compatible) functions, but I don’t think the Swift compiler should be expected to implement routines to make a Swift function callable by Python. (Incidentally I recently made a pull request on PythonKit to do just that, but as a library feature).

On the same note, what do we expect @extern(c) to actually do? Is it just about demangling the symbol name? Would it be reasonable to also give it the default of setting a C calling convention, maybe with an overridable default: @extern(c, convention: MSVC)?

I notice you’ve picked extern here, which obviously parallels C/C++. To me that makes it not a bad choice, but is it a really good one? Phrased differently, is it a choice that we would make for Swift if C had never existed (ignoring the paradox inherent in the question for a second).

Your post has brought attention to the additional question of ABI, but I can't think of a use case for signifying a single function to be compatible with one ABI where the rest of the binary output is compatible with another. I would believe you if you said that one exists, but it seems to be a feature with a very limited target audience.

[jayton (Jens Ayton)]

@Geordie_J

To me that makes it not a bad choice, but is it a really good one?

I don’t like truncated names (like func), but I think a more important consideration is how extern works syntactically in C: extern is attached to a declaration, and then controls either exporting or importing of a symbol depending on whether the symbol is defined. In today’s Swift, @_silgen_name kind of parallels this, since it can be used on a bodyless declaration or a full function definition, while @_cname can only be used on a full definition.

Since prototypes are Not A Thing in Swift, I don’t think the C design (or the @_silgen_name design) are particularly appropriate, and separate attributes for importing and exporting symbols would be desirable.

[compnerd (Saleem Abdulrasool)]

@Geordie_J

Forgive my ignorance but would would @extern(python) actually do ?

Nothing, it's more a point of flexibility that can be enabled. Consider PythonKit in the compiler. Im not arguing that we should do that, merely that we do not design something which precludes it.

@Geordie_J

On the same note, what do we expect @extern(c) to actually do? Is it just about demangling the symbol name? Would it be reasonable to also give it the default of setting a C calling convention, maybe with an overridable default: @extern(c, convention: MSVC) ?

@extern(c) would indicate that the function should be setup for C linkage - it covers name decoration and calling convention. As to the calling convention for C, I don't think that we need that to be something which can be overridden. I think that the target triple is sufficient for that purpose as breaking that breaks nearly everything implicitly.

@Geordie_J

I notice you’ve picked extern here, which obviously parallels C/C++. To me that makes it not a bad choice, but is it a really good one?

IMO, yes. The reason is that there is precedence and associated meaning with this for those working with the ABI level considerations.

@Geordie_J

Your post has brought attention to the additional question of ABI, but I can't think of a use case for signifying a single function to be compatible with one ABI where the rest of the binary output is compatible with another.

When you are working with dynamically linked libraries that are loaded at runtime, you can isolate the address space sufficiently to enable multiple ABIs for C++ to coexist (dangerous, but possible). Yes, I am aware of the complications of the C++ standard wrt doing this.

All that said, I am simply pointing out the various things which come up when dealing with interoperability. I think that starting with @extern(c++) @convention(thiscall) (member functions and pointer to member functions) and @extern(c++) (global variables and free functions) is a great starting point. Extending the support for @abi is something which can be done as a follow up.

[Torust (Thomas Roughton)]

@compnerd

@Geordie_J

I notice you’ve picked extern here, which obviously parallels C/C++. To me that makes it not a bad choice, but is it a really good one?

IMO, yes. The reason is that there is precedence and associated meaning with this for those working with the ABI level considerations.

To give another option here, picking up on how you've been referring to it: @linkage rather than @extern seems a more descriptive attribute.

[jrose (Jordan Rose)]

Sorry for dropping the post and then disappearing for a week! I forgot how busy a first week of work can be. ;-) But of course what I've come back to (and snooped on during the week) is a bunch of really good points. I'll break up my responses based on topic; this one's about spelling out what we need to support on the C side:

@allevato

Since you mention the upcoming C++ interop briefly a couple times, it would be good to consider how that fits into a C-family interop story. If we merely remove the underscore from @_cdecl , then we'll have two distinct attributes for C and Obj-C interop. If we later want to add a way to export some subset of Swift types/functions as C++ types/functions, would we then have to add another attribute, like @cppdecl ?

This is a really good point, and I think I agree it means that it'd be better to have something parameterized. After all, there are only three numbers in computer science, and "C and C++" is already more than one even if @objc remains special.

@allevato

What does header generation look like if we're generating a header for code that uses @objc, something equivalent to @_cdecl, and something for C++ all in the same module? Does the flag decide what to generate, or is there always a single header with appropriate #if guards around each section?

For now, I think the rule is "everything you can see gets generated", since @objc is still illegal without -enable-objc-interop. I don't know how that scales to a module that wants to support two-way or three-way interop, but we don't have to support that right out of the gate.

@compnerd

Id like to throw a bit of paint into the colour of the bikeshed here. I'm not too fond of @cdecl as the attribute. The reason is that it is confusing precisely where it is used - interoperability with C. The spelling of @cdecl is too near the spelling of __cdecl and __attribute__((__cdecl__)) which specify the calling convention.

This is also a good point, but I'm not sure how much it matters if the goal is "expose a Swift decl so it can be called from C". We don't need to support all the Windows calling conventions; we just need to pick one that everyone else supports calling. Still, it's an another con for a plain @cdecl.

@compnerd

The calling convention and name decoration requires a third piece of information which is missing. The current piece which seems best to me: @extern(c++) @convention(thiscall) doesn’t tell me everything I need. Yes, this calls a function that has an implicit this parameter, that is it is a class bound method, with a C++ interop, but it fails to inform me of the ABI. Did you mean MSVC? Itanium? GNU? (Yes, GCC had a custom ABI before itanium).

Ditto: I don't think we need this level of control. We should pick the default ABI for your target triple and make sure the generated header has it annotated.

The counterargument to this is of course the "plug-in entry point" use case, but until we see one I think we might be able to get away without it.

@Joe_Groff

There are other reasons we may need to be able to encode arbitrary C function types in @convention(c) Swift function types, something like @convention(c, typeName: "BOOL (BOOL)") in Swift.

I don't the idea of doing this at the type level, but it does seem like something we could allow on declarations (in some form). I don't think we need to plan it out now, though, while bool vs. BOOL is immediately relevant.

@uliwitness

Maybe:

@export(objc) func foo(_ : @export(objc,int) Bool)

To leave the option of annotating one function for several languages if needed? Similarly, this might be used to control whether something becomes a block or a function in C.

Ditto (even in simplified form).

[jrose (Jordan Rose)]

@allevato

@jrose

• Plug-in-style APIs that expect your library to define a certain unmangled symbol name.

Is this referring to plug-ins implemented in Swift and consumed by a Swift host program? This seems like an anti-pattern since, as you mention, it requires the plug-in and the host to marshal the plug-in instances through either unsafe pointers or at a minimum lose type information using a hypothetical SwiftOpaqueRef type that you described.

This is about C-style plug-ins that dlopen a library and dlsym its entry point. In this case, the entry point's signature is defined in terms of the host's API; an example is libFuzzer:

The first step in using libFuzzer on a library is to implement a fuzz target – a function that accepts an array of bytes and does something interesting with these bytes using the API under test. Like this:

// fuzz_target.cc
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
  DoSomethingInterestingWithMyAPI(Data, Size);
  return 0;  // Non-zero return values are reserved for future use.
}

By default, libFuzzer expects your library to export a symbol with exactly the name 'LLVMFuzzerTestOneInput'. People have already been using libFuzzer from Swift via @_cdecl (and @_silgen_name, but that's technically incorrect). There's no custom types involved, but if there were they'd be structs defined in libFuzzer's headers, not on the plug-in side.

[jrose (Jordan Rose)]

@jayton

It seems to me that @_cdecl actually does two distinct things:

• Constrain the types and calling conventions of a function such that it can be called from C (or other languages with C interop). This overlaps with @convention(c) , but I don’t really know how closely.
• Define the linkage name of a function, which is used for static and dynamic linking and also shows up in backtraces. This overlaps with @_silgen_name .

@_silgen_name is the elephant in the room. Part of that is right there in the name of the attribute: "SILGen name". @_silgen_name doesn't just change the symbol name of a function; it changes how the compiler itself sees the function, well before it's emitted.* It also does not change the calling convention, but the Swift calling convention is mostly compatible with C's on most platforms and those "mosts" should freak you out.

The other thing @_silgen_name does is let you declare functions that exist without importing a module or header file. That's inherently unsafe, but it's also how Rust calls C functions**, so I don't want to say it's forever off the table. For now, though, it'd be a big shift in how what Swift allows, so I want to set this aside. I really want to set this aside.

@Geordie_J

I’m not sure of its current status but we have had to use @_silgen_name instead of @_cdecl in the past due to the latter's lack of availability on Linux.

I don't think this was ever a thing; please go see if you can remove @_silgen_name!

* For what it's worth, I believe @_cdecl is currently implemented the same way, and I'm not sure we should keep that. I'd rather preserve the Swift name until we get down to LLVM, which is where we've resolved all names and no longer need to look in imported modules.

** Though often these extern blocks are generated by a tool like bindgen rather than written by hand, precisely because it's easy to get it wrong.

---

Okay, so what about the other aspect: the separation of name control from calling convention? The main thing I'm thinking about here is the goal: call a Swift-defined function from C.

@jayton

Language orthogonality aside, it’s not entirely obvious that exporting an API to C should default to changing the linkage name. Instead of generating C prototypes for exported Swift functions, an exporter could generate stubs like this:

Jens's stub here is clever, and mostly works because we've kept our mangling scheme based on the C identifier character set. I had originally dismissed the idea because I didn't think of a way to do it without language extensions. I would still like a way to control the symbol name too, but maybe that really is separable.

Anyway, the conclusion I'm coming to is that we should have an attribute like @available where there's one required parameter—the language—and then several keyword arguments. That leaves room for expansion in the future.

@extern(c)
@extern(c, name: fookit_setup)
@extern(c++, abi: itanium)

If we follow the convention of @objc, then @extern(c) would generate a C function but use the Swift symbol name, and @extern(c, name: fookit_setup) would actually change the symbol name. That's fairly subtle, though, so maybe we'd want to do something else? (As has been pointed out above, you can always write a wrapper function that has the right C name if you also care about the name in Swift.)

Does the name really need a parameter label, since it's the most common thing? It does if we want to support Objective-C selectors without quotes, and I think quotes are bad here because the compiler does need to validate it a little. Okay, technically you can distinguish them because selectors with arguments always end in a colon, but still.

I'm not the first in the thread to come up with the argument-based attribute, of course; I'm just throwing my weight behind it. Y'all have convinced me. :-)

One thing that's missing here is the @objc @cdecl thing. How do you distinguish between a C function that allows ObjC types from a C function that doesn't? Maybe something like @extern(c, allow: objc)? That does scale to @extern(c, allow: c++), which is a thing that happens sometimes… But if anyone has better suggestions, I'm all ears.

*** You can't do this with Objective-C methods because you need to be able to implement a protocol method in both Objective-C and Swift and have it work from the other language. And it wouldn't work at all with classes.

---

Finally, the name. I don't love the name extern because I feel like it's more strongly associated with it's "declare but not define" meaning than its "share with other files/libraries/languages" meaning, but it does have massive precedent. My thoughts on the other suggestions in the thread:

• @linkage isn't a term most people would know, I think. (It means more than just C vs. C++ in C-land, too.)

• @decl feels very generic (though I appreciate that Swift is just another choice there)

• @export really sounds like it's related to imports to me.

• @callableBy feels wordy and doesn't work on things other than functions.

• @convention(c) doesn't feel like it'll scale to all our uses, and @extern @convention(c) seems needlessly verbose even if @extern also supports a symbol name.

---

I was hoping I'd come to conclusions at the end of these update posts, but there aren't too many to be had. I'm glad we're having the discussion, though!

P.S. I don't like anyone's names for the alternate -emit-objc-header either. Bleah!

[michelf (Michel Fortin)]

@jrose

How do you distinguish between a C function that allows ObjC types from a C function that doesn't? Maybe something like @extern(c, allow: objc) ?

If it starts with @extern(c, ...) I'd expect the function to be visible in a C header. If the thing is only exported to an Objective-C header, I'd rather see it start with @extern(objc, ...). Then add something to say it takes the form of a C function: @extern(objc, format: c)

@jrose

Finally, the name.

@exposed(c): starts with "ex" like extern, but without suggesting it was defined externally.

[jrose (Jordan Rose)]

@expose[d] does match the language in documentation about “exposing methods to Objective-C”. It feels a little long to me but it may be the best trade-off we get.

I don’t love @expose(objc) making a C function, but I guess we could say (in the future) “methods on structs and enums become functions, methods on a class become methods”, which would be 99% of the use cases anyway.

[stuchlej (Mikoláš Stuchlík)]

I would like to express my excitement for this feature. The ability to create not only functions, but types (including function pointers) exposable to C would be great to have.

Would it be possible to create a better support for C macro system? Fox example the ability to wrap C macro with arguments as a functions.

[Geordie_J (ephemer)]

@jrose

One thing that's missing here is the @objc @cdecl thing. How do you distinguish between a C function that allows ObjC types from a C function that doesn't? Maybe something like @extern(c, allow: objc) ? That does scale to @extern(c, allow: c++) , which is a thing that happens sometimes… But if anyone has better suggestions, I'm all ears.

I'm not very familiar with ObjC interop in Swift: to me it has always just been a confusing addition that is kind if implicitly there – as you mentioned in a post above, @_cdecl on Darwin platforms appears to always imply @objc anyway.

In my opinion we should remove that confusion and have @extern(c) only do one consistent thing across all platforms (i.e. only set linkage and calling convention). I think the way you wrote this originally (leaving the two attributes separate) is the right thing to do. @objc @extern(c) seems like the right mix of keeping things as they were while keeping attributes orthogonal.

For the record I am fine with the idea of not demangling symbol names by default in @extern(c) so long as we still can define them ourselves (via @extern(c, name: "...") would be fine).

One thing this discussion is missing is forward declaration of external symbols. That is possible via @_silgen_name but AFAIK there's no way to specify a forward declaration made in that way as needing, for example, the C calling convention – which is something we need very often.

[hishnash (Matthaus Woolard)]

@jrose

• Struct declarations : I think @cdecl structs make sense but I don't want to roll them into this proposal; keep defining your structs in C for now.

Having the ability to expose structs and static instances of C structs would enable Swift libs to expose CPython modules directly.

[jrose (Jordan Rose)]

@Geordie_J

One thing this discussion is missing is forward declaration of external symbols. That is possible via @_silgen_name but AFAIK there's no way to specify a forward declaration made in that way as needing, for example, the C calling convention – which is something we need very often.

As mentioned that’s very deliberate: besides being something that’s easy to separate from the “defining” side, it’s something that I expect to be a lot more controversial.

[jrose (Jordan Rose)]

@stuchlej

Would it be possible to create a better support for C macro system? Fox example the ability to wrap C macro with arguments as a functions.

That’s about the other direction, C-into-Swift, and it’s a hard problem because most macros give no indication at all what types it expects. I don’t know if there’s ever going to be a better answer than “write a wrapper function in C”.

[Geordie_J (ephemer)]

Fair enough. For now we will continue using @_silgen_name and hoping the calling convention stays "compatible enough".

I am also aware we could create and import a C module that exposes these, so it's not really a blocker for us. I guess the real issue I have here is that creating and importing a C module currently is not as straightforward as it could be. If that was easier at a tooling level I personally could drop the requirement for forward declarations of C functions altogether.

[Joe_Groff]

You should never use @_silgen_name. The calling convention is already fundamentally incompatible on some targets (wasm, and likely arm64e on Apple platforms when it eventually stabilizes). You can forward declare C entry points by importing a header that contains the C declaration.