[Pitch] Controlling function definition visibility in clients

I guess I don't understand the use case for having an @exported(interface) attribute if it is not to offer a mechanism to achieve encapsulation in a wmo-all-the-things-by-default mode. In other modes, the behavior of @exported(interface) is already the default, implicit behavior for public, package, and @usableFromInline internal declarations, as long as they don't have one of the existing attributes that implies the effect of @exported(implementation). We're not proposing to make that implicit behavior explicit by requiring library authors to add redundant @exported(interface) on all of their public declarations, right?

What are the reasons someone would use the proposed @exported(interface) attribute?

Having not used Embedded Swift, this may be a naive question but: is this to say that if we expose (ha) this feature, then there will be a language-level difference between Embedded Swift and vanilla Swift? would this not violate our original overarching declaration that Embedded Swift is to be a subset and not a dialect?

While I suppose you could use this feature to re-impose separate binary compilation in a wmo-all-the-things environment, I agree it seems like a poor and error-prone tool to do so.
The use case for embedded Swift, as I understand it at least, is to let a build product explicitly provide the necessary linkable symbols for whatever the build product is being linked into.

In my head, I can see this as a generalization of @main—in order to build a useful executable binary, you need to provide an int main(int argc, char **argv, ...) symbol that the C runtime libraries can link to the process entry point. But if you want to build a Windows GUI executable, you need a __stdcall WinMain instead; if you want to build a firmware image, you might need to provide a bunch of symbols that link against the vendor's library; if you want to build a plugin dynamic library, you have to provide a bunch of symbols according to the host program's plugin interface, and so on. That seems to me like a generally useful use case (and as a bonus, maybe it lets us demote @main to a macro in the fullness of time), irrespective of whether a platform is considered embedded or not. It's also a use case where manual intervention is probably inevitable, since it is unlikely in the full universe of weird link environments that we'd ever be able to get all of the correct behavior to just fall out of sensible build defaults.

1 Like

Maybe a @kubamracek question: is there a world where Embedded Swift does not WMO all the things all the time, or is it load-bearing for acceptable results?

Yes, that's one reasonable way to view it: we're identifying the symbols that need to be available for other tools to link against.

I have needed @export(interface) for two reasons:

  1. Ensure that there is a public symbol for some external system to see
  2. Prevent the implementation from being exported to clients because it exposes dependencies (basically, making internal import fully hide the dependency)

My proposal document does say that. Specifically, it says:

@export that includes the implementation argument inherits all of the restrictions as @inlinable that are outlined in SE-0193, for example, the definition itself can only reference public entities or those that are themselves @usableFromInline.

Embedded Swift is a convenient way to describe the "expose all implementations everywhere" extra, at the other end from Library Evolution. You are absolutely right that this is an optimization model that can be applied to non-Embedded Swift.

It is load-bearing. As a simple example: a generic function being @export(interface) in Embedded Swift conflicts with the requirement to monomorphize all generics.

It's effectively a different default. Non-Embedded Swift normally defaults public symbols to @export(interface). Cross-module optimization (in non-embedded) infers @export(interface,implementation) for some public symbols. Embedded Swift defaults basically everything to @export(implementation). As Joe notes above, we could enable similar optimizations in non-Embedded Swift that defaults everything to @export(implementation).

Doug

3 Likes

If a module doesn't have any public declarations that are generic or otherwise unsupportable, though, then it should still be possible to support a separate-binary compilation mode. If you look at the WMO-vs-binary compilation mode as setting the default @export behavior, then it seems like it would fall out that building in @export(interface)-by-default mode would work for an embedded Swift target, but only when all the default declarations would allowed to be @export(interface) to begin with. (And you could still use explicit @export(implementation) if you have a layer of generic interface you want to expose to clients without baking into the binary.)

1 Like

Is @_alwaysEmitIntoClient really equivalent to @inline(only)? It allows for outlining still, no?

Spelling aside, we should absolutely formalize the feature.

Hey all,

I've tried to answer a bunch of the questions that have come up here in a new revision. In it, I spell out:

  • The existing language features that affect symbol availability and when definitions are made available to clients
  • How different compiler optimizations affect symbol availability and when definitions are made available to clients
  • What implementation hiding is and why it is import
  • The relationship between the @export attribute and access specifiers like public and internal
  • Necessary limitations involving Embedded Swift

EDIT: One more addition based on discussions (and because the implementation already does part of it): this attribute can also apply to stored properties and types.

Thanks for the great discussion so far!

Doug

5 Likes

Thanks Doug.

Module B cannot call the function secret under any circumstance. However, with aggressive CMO or Embedded Swift, the compiler will still make the definition available when compiling B, which can be used to (for example) inline both f() and secret into the body of g.

If this behavior is not desired, the secret function could be marked as @export(interface) to ensure that it is compiled to a symbol that it usable from outside of module A. It is still private, meaning that it still cannot be referenced by source code outside of that file.

This could imply @export(interface) over private func secret() prevents secret() from being inlined into f() (or else, whatever we don’t want to leak from secret() can still be inlined into f(), which can then be inlined into g(), without the export status of secret() ever coming into play). Could you clarify this? Your table explaining the relationships between @export and @inline doesn’t have a column for the case that the function has no @inline attribute.

My guess for the fix would have been to add @exported(interface) to f instead of secret.

You're right; you would also need @inline(never) to make sure that secret doesn't get inlined into f. The easier solution would be, as you suggest, to put @export(interface) on the public function.

Great point. I added a third column in this commit, and clarified the discussion referenced in your point above.

Doug

1 Like

Having worked on the proposal document more, I'm starting to dislike the @export(interface) and @export(implementation) thing. I feel like the always/never terminology we have in the existing underscored attributes and that we had with my first @inlinable-based version is more clear. Here's a new suggestion for painting this bikeshed:

  • emitIntoClient(always): same as @_alwaysEmitIntoClient or the currently-proposed @export(implementation).
  • emitIntoClient(never): same as @_neverEmitIntoClient or the currently-proposed @export(interface).

This attribute doesn't subsume or replace @inlinable, unless we were to introduce a third state. I think it's fine not to subsume @inlinable.

Thoughts on this particular color?

Doug

1 Like

First time I ever have a feeling that I finally understood how inlining works in Swift was after it was described using @export notation. Before it I read documentation about underscored attributes and some posts about the topic.

4 Likes

Is the "never" option meaningfully different from the standard behaviour of a public symbol? I feel like I'm missing something.

The standard behavior is not the behavior in Embedded Swift, and may not be what we want long term if we want more cross-module optimization by default. There's a lot of (understandable!) confusion here because different optimizations do different things, so I added a section to the proposal on how different compiler optimizations mess with the presence of symbols and availability of definitions in the callers.

Doug

1 Like

The thing I liked about the export notation was that it was clear about what was being exported.

I don’t care between export and emit for the verb but I did like the lack of ambiguity.

It’s not style-guidey though, that’s true.

So we would have:

  • @emitIntoClient(always) → @export(implementation)
  • @inlinable → @export(interface, implementation)
  • @emitIntoClient(never) → @export(interface)

I don’t think there’s a great way to expand this syntax towards symbol visibility, which is one of the future directions, but there are probably other ways to solve that particular problem anyways (for instance, don’t make entry points for public symbols from static libraries that were imported with internal import).

To be honest, I did like subsuming @inlinable because my experience is that people misunderstand it to nudge the inliner, like inline does in C, and I found it clearer that @export didn’t.

I also thought that attaching this to the access modifier optionally wasn’t such a bad idea.

1 Like

I feel like @emitIntoClient(always) does not really clarify that it also has the side effect of removing the exported symbol from the compiled object file. That’s one of the things I like about the @exported syntax (perhaps expanded to @export(interface: false, implementation: true)) — it makes it very obvious that you’re removing the ABI symbol. And @export(interface: true, implementation: false) internal could be another spelling for @usableFromInline internal.

1 Like

I like this aspect too. @inlinable is definitely a source of confusion.

I would like to take this opportunity to ask that once @_alwaysEmitIntoClient is formalized there is an official recommendation for contributors on when to use @backDeployed or @export(implementation). I say this because although @backDeployed has been around for over 2 years, @_alwaysEmitIntoClient continues to be used for new back deployed APIs.

1 Like

I’m still trying to wrap my head around this proposal and the @inline one. How do things work when @export(...) isn’t specified, and how does it interact with @inlinable?