Pitch: Cross-module inlining and specialization

[Chris_Lattner]

1) Why not another level of access control? There is a reasonable argument that what you’re doing is making something “more public than public” or that you’re making the “body also public”. I’m not strongly in favor of this design approach, but if you agree, the doc should explain why you’re not in favor of it.

Oh, I forgot to address this point. I’m against this because even though this proposal says inlinable requires public, it is really orthogonal to public, because eventually we will have something like the current @_versioned attribute, where internal (or even private, as some have suggested) functions can be “public ABI”. For example,

I’m also against it, because it should theoretically be orthogonal to open/public. While we could define “fragile” as being “more permissive than open and public”, that is weird and access control is already too complicated. :-)

-Chris

···

On Oct 2, 2017, at 11:27 PM, Slava Pestov <spestov@apple.com> wrote:

On Oct 2, 2017, at 10:57 PM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:

@_versioned func myInternalDetails() {
  // …
}

@inlinable
public func myPublicFunction() {
  myInternalDetails() // I can reference this from an inlinable function, but users can’t call it directly
}

Slava

[Chris_Lattner]

I disagree. The semantics being proposed perfectly overlap with the transitional plan for overlays (which matters for the next few years), but they are the wrong default for anything other than overlays and the wrong thing for long term API evolution over the next 20 years.

-Chris

···

On Oct 3, 2017, at 9:50 AM, Joe Groff <jgroff@apple.com> wrote:

On Oct 2, 2017, at 10:58 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

We have discussed adding a "versioned @inlinable" variant that preserves the public entry point for older clients, while making the declaration inlinable for newer clients. This will likely be a separate proposal and discussion.

5) It eliminates this complexity.

It wouldn't avoid the complexity, because we want the "non-ABI, always-emit-into-client" behavior for the standard library. For the soon-to-be-ABI-stable libraries where @inlinable even matters, such as the standard library and Apple SDK overlays, there's pretty much perfect overlap between things we want to inline and things we don't want to take up binary space and ABI surface in binaries, so the behavior Slava proposes seems like the right default.

[Chris_Lattner]

As I said above, I pretty strongly disagree with this approach. It encourages a “one attribute/declmodifier/keyword per decl kind” approach, which is user hostile. It doesn’t make sense for “exhaustive" to apply to enums, "@inlinable” to apply to function-like things, something else to apply to structs, something else to apply to classes, etc.

I’d rather that we define one “thing” which can be parameterized. I’d propose that that be named “fragile” because it appropriately carries negative connotations. If some declaration has multiple levels of fragility that we want to support, we can allow it to be parameterized, e.g.:

/// C style fragility, all things about the decl are fragile.
fragile struct Foo { … }

/// Just layout is fragile, but not other things (whatever they would be?)
fragile(layout) struct Foo { … }

I pretty strongly believe that we should start with the default of having a unifying principle that ties together all resilience concerns under a single umbrella, and fission those concerns when we’re faced with something that forces it. Starting with them split and hoping that we’ll come around to merging them someday is the wrong side of optimism IMO.

We’ve been talking about this approach for years, which is why I’m somewhat surprised to see the new direction these things are taking.

-Chris

···

On Oct 3, 2017, at 10:04 AM, Joe Groff <jgroff@apple.com> wrote:

On Oct 2, 2017, at 10:58 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The major question I have is “why yet another attribute”. The thread about exhaustive/extensible enums is similarly proposing introducing another one-off way to be enums fragile, and this is directly related just for function-like things.

I’d love to see rationale in the proposal for why you’re not taking this in one of these directions:

1) Why not another level of access control? There is a reasonable argument that what you’re doing is making something “more public than public” or that you’re making the “body also public”. I’m not strongly in favor of this design approach, but if you agree, the doc should explain why you’re not in favor of it.

2) Why can’t we have a single Swift-wide concept that unifies all of the resilience ideas under a single umbrella like “fragile” - which indicates that the body of a declaration is knowable to clients? There is a very reasonable holistic design where “fragile public func” makes its body inlinable, and “fragile enum” similarly makes the cases knowable to the client (thus making exhaustive switching a possibility). I am strongly in favor of this approach.

I'm of two minds on this. So far, it has seemingly been the case that, for every kind of declaration, there's been exactly one resilience-related attribute, and with blurry enough vision you could say they're all really doing the same thing, "expose this declaration body directly to clients", which only makes sense as a concept for public API. If these properties hold, then sure, it might make sense to consider these all as one "extra-public" concept. On the other hand, we don't know yet whether these properties will hold in the long term as we're actively designing the facilities we need for ABI stability and resilience. From an incremental, iterative design perspective, I think it's wiser to keep them all separate attributes, even if it's a bit messy in the short term. Once the design stabilizes, we can look at whether it makes sense to fold related concepts together. It's easier to fold different things together after the fact then to try to separate things after they've been mixed together.

[Chris_Lattner]

In any case, even if you’re opposed to these approaches, I’d love for the “alternatives considered” section to indicate what the objection is. I am really very concerned that you’re causing a keyword/attribute explosion and conceptual complexity by adding too many small things to individual parts of the language. We would ideally have a simple and holistic solution to resilience.

I agree with that keyword/attribute explosion is a concern. We also plan on submitting a proposal to add a @fixedContents attribute for structs (currently implemented as @_fixed_layout) which enables more efficient access patterns in resilient code, for example direct access of stored properties, at the cost of preventing new stored properties from being added in a binary-compatible manner. So we would have ‘nonexhaustive’ enums, @fixedContents structs, and @inlinable functions/properties/initializers.

Yes, and then we’ll need something else for classes as well (*head explodes*).

I really prefer having a single “fragile” concept, and allowing it to be parameterized for declarations that can be fragile in different ways: "fragile(storage) class Foo” or whatever.

Perhaps it makes sense to have a single ‘fragile’ keyword replace @fixedContents and @inlinable. For enums, ‘nonexhaustive’ does “feel” a bit different, so perhaps it makes sense for it to be its own thing.

The core team discussed enums about a year ago and my recollection was that we came to the decision that there was “c style fragility” and “non-fragility” that were the only interesting points on the spectrum. Intermediate points like saying that "the list of members can be fixed, but their enum values were not” was deemed to be only of academic curiosity. If we ever had a compelling use-case for such a thing, we could parameterize fragile to support it.

From an implementation perspective it doesn’t really matter if we have multiple attributes or one, so of course I’d prefer to go with the approach that makes the most sense to people language design-wise.

Yes, I’m only concerned with the user visible design here. Internally implementing these things as a matrix of capabilities is entirely sensible, but not the purview of swift-evolution.

This semantic doesn’t make sense to me, and I think we need to change it. I think we are better served with the semantics of “the body may be inlined, but doesn’t have to.”

That is the effect it has today. The decision to inline or not is made by the optimizer, and @inlinable doesn’t change anything here; it makes the body available if the optimizer chooses to do so.

If I read the proposal correctly, it is saying that the body has to be inlined into the modules that use it, because there is no symbol published from the module that defines it. That’s the concept of inlining that I’m objecting to.

-Chris

···

On Oct 2, 2017, at 11:11 PM, Slava Pestov <spestov@apple.com> wrote:

[taylorswift]

I think we should try to separate visibility from access control. In other
words, the compiler should be able to see more than the user. I want to be
able to write private and internal code that cannot be called explicitly in
source, but can still be inlined by the compiler. Right now people are
doing this with underscored methods and variable names but I don’t think
that’s a good convention to use. We should have something at the language
level that enforces that something shouldn’t be referenced by name outside
of its scope, but is public for all compilation and ABI purposes. Maybe an
attribute like @visible or a new keyword or something.

···

On Mon, Oct 2, 2017 at 4:45 PM, Xiaodi Wu via swift-evolution < swift-evolution@swift.org> wrote:

This is unduly restrictive; @_versioned (despite being the wrong spelling)
is what we want here. To be callable from an inlinable function, internal
things need only be visible in terms of public ABI, not necessarily
inlinable, just as public things need only be public and not necessarily
inlinable.
On Mon, Oct 2, 2017 at 16:37 Nevin Brackett-Rozinsky via swift-evolution < > swift-evolution@swift.org> wrote:

On Mon, Oct 2, 2017 at 5:21 PM, Slava Pestov <spestov@apple.com> wrote:

Thanks for taking a look!

> On Oct 2, 2017, at 2:19 PM, Nevin Brackett-Rozinsky < >>> nevin.brackettrozinsky@gmail.com> wrote:
> 3. Even though @inlinable will have no effect on declarations which
are not public, we should still allow it to be placed there. That way when
the access level is later changed to be public, the attribute is already
where it should be. This is similar to why we permit, eg., members of an
internal type to be declared public, which was discussed and decided
previously on Swift Evolution.

This is an interesting point. Do you think the attribute should be
completely ignored, or should the restrictions on references to non-public
things, etc still be enforced?

Hmm, good question!

I rather like the idea Greg Parker put forth, where non-public @inlinable
items can be used by public @inlinable ones, which implies that the
restrictions should indeed still apply—something @inlinable can only
reference public or @inlinable things.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[xwu (Xiaodi Wu)]

I think we should try to separate visibility from access control. In other
words, the compiler should be able to see more than the user. I want to be
able to write private and internal code that cannot be called explicitly in
source, but can still be inlined by the compiler. Right now people are
doing this with underscored methods and variable names but I don’t think
that’s a good convention to use. We should have something at the language
level that enforces that something shouldn’t be referenced by name outside
of its scope, but is public for all compilation and ABI purposes. Maybe an
attribute like @visible or a new keyword or something.

Right, that’s @_versioned, essentially.

···

On Mon, Oct 2, 2017 at 17:41 Taylor Swift <kelvin13ma@gmail.com> wrote:

On Mon, Oct 2, 2017 at 4:45 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

This is unduly restrictive; @_versioned (despite being the wrong
spelling) is what we want here. To be callable from an inlinable function,
internal things need only be visible in terms of public ABI, not
necessarily inlinable, just as public things need only be public and not
necessarily inlinable.
On Mon, Oct 2, 2017 at 16:37 Nevin Brackett-Rozinsky via swift-evolution < >> swift-evolution@swift.org> wrote:

On Mon, Oct 2, 2017 at 5:21 PM, Slava Pestov <spestov@apple.com> wrote:

Thanks for taking a look!

> On Oct 2, 2017, at 2:19 PM, Nevin Brackett-Rozinsky < >>>> nevin.brackettrozinsky@gmail.com> wrote:
> 3. Even though @inlinable will have no effect on declarations which
are not public, we should still allow it to be placed there. That way when
the access level is later changed to be public, the attribute is already
where it should be. This is similar to why we permit, eg., members of an
internal type to be declared public, which was discussed and decided
previously on Swift Evolution.

This is an interesting point. Do you think the attribute should be
completely ignored, or should the restrictions on references to non-public
things, etc still be enforced?

Hmm, good question!

I rather like the idea Greg Parker put forth, where non-public
@inlinable items can be used by public @inlinable ones, which implies that
the restrictions should indeed still apply—something @inlinable can only
reference public or @inlinable things.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Andrew_Trick]

FWIW, the @inlinable name has always confused me. Methods not marked @inlinable are still internally inlinable. "Inlining" is already a term of art with specific semantics in other languages, and even in Swift is it's own thing to be controlled independently from resilience. The real issue I have with the name is that it says nothing about resilience. I’ll never forget that fragility is the opposite of resilience. I can't see how a @fragile attribute would ever be misconstrued.

As for the various shades of fragility of data types, I don't see why that can't be handled as qualifiers or additional optional attributes for expert developers. It’s just a matter of picking a reasonable default.

-Andy

···

On Oct 2, 2017, at 11:20 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org> wrote:

On Oct 2, 2017, at 11:11 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

This semantic doesn’t make sense to me, and I think we need to change it. I think we are better served with the semantics of “the body may be inlined, but doesn’t have to.”

That is the effect it has today. The decision to inline or not is made by the optimizer, and @inlinable doesn’t change anything here; it makes the body available if the optimizer chooses to do so.

Also remember we have the @inline(never) attribute. It’s not underscored so I’m assuming it’s an “official” part of the language. And "@inline(never) @inlinable" is a perfectly valid combination — it serializes the SIL for the function body, and while inlining it is prohibited, it is still subject to specialization, function signature optimizations, etc.

Slava

[Chris_Lattner]

Despite the underscore, I don’t think that this was officially ever “design reviewed”, and I don’t think it is widely used. If we need to change or restrict or add requirements to @inline(never), we can do so IMO. We can design adequate upgrade paths and keep the old behavior around for compatibility if necessary. Lets focus on what the right long term model is.

-Chris

···

On Oct 2, 2017, at 11:20 PM, Slava Pestov <spestov@apple.com> wrote:

On Oct 2, 2017, at 11:11 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

This semantic doesn’t make sense to me, and I think we need to change it. I think we are better served with the semantics of “the body may be inlined, but doesn’t have to.”

That is the effect it has today. The decision to inline or not is made by the optimizer, and @inlinable doesn’t change anything here; it makes the body available if the optimizer chooses to do so.

Also remember we have the @inline(never) attribute. It’s not underscored so I’m assuming it’s an “official” part of the language. And "@inline(never) @inlinable" is a perfectly valid combination — it serializes the SIL for the function body, and while inlining it is prohibited, it is still subject to specialization, function signature optimizations, etc.

[Slava_Pestov]

We have discussed adding a "versioned @inlinable" variant that preserves the public entry point for older clients, while making the declaration inlinable for newer clients. This will likely be a separate proposal and discussion.

5) It eliminates this complexity.

It wouldn't avoid the complexity, because we want the "non-ABI, always-emit-into-client" behavior for the standard library. For the soon-to-be-ABI-stable libraries where @inlinable even matters, such as the standard library and Apple SDK overlays, there's pretty much perfect overlap between things we want to inline and things we don't want to take up binary space and ABI surface in binaries, so the behavior Slava proposes seems like the right default.

I disagree. The semantics being proposed perfectly overlap with the transitional plan for overlays (which matters for the next few years), but they are the wrong default for anything other than overlays and the wrong thing for long term API evolution over the next 20 years.

Can you elaborate on this? If inlinable functions have public entry points, the version in the framework may or may not be called… because of SIL serialization and inlining. Since the existence of the public entry point doesn’t offer much of a guarantee, it seems desirable to not have the public entry point. For example if the inlinable function is not used elsewhere in the framework, we wouldn’t have to emit it at all. This might make the standard library smaller for instance.

However I’m still waiting for Dave or Jordan to chime in with the original justification for the ‘always emit into client’ behavior. IIRC there was a resilience-related argument too, but I don’t remember what it is now.

Slava

···

On Oct 3, 2017, at 9:56 PM, Chris Lattner <clattner@nondot.org> wrote:

On Oct 3, 2017, at 9:50 AM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Oct 2, 2017, at 10:58 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

-Chris

[Slava_Pestov]

FWIW, I was hoping we wouldn’t need to expose any such attribute for classes (or protocols) at all, because classes are already “slow” and anything we do to make them resilient doesn’t make things much “slower”. But that could change, of course.

Slava

···

On Oct 3, 2017, at 10:14 PM, Chris Lattner <clattner@nondot.org> wrote:

On Oct 2, 2017, at 11:11 PM, Slava Pestov <spestov@apple.com> wrote:

In any case, even if you’re opposed to these approaches, I’d love for the “alternatives considered” section to indicate what the objection is. I am really very concerned that you’re causing a keyword/attribute explosion and conceptual complexity by adding too many small things to individual parts of the language. We would ideally have a simple and holistic solution to resilience.

I agree with that keyword/attribute explosion is a concern. We also plan on submitting a proposal to add a @fixedContents attribute for structs (currently implemented as @_fixed_layout) which enables more efficient access patterns in resilient code, for example direct access of stored properties, at the cost of preventing new stored properties from being added in a binary-compatible manner. So we would have ‘nonexhaustive’ enums, @fixedContents structs, and @inlinable functions/properties/initializers.

Yes, and then we’ll need something else for classes as well (*head explodes*).

[Joe_Groff]

We have discussed adding a "versioned @inlinable" variant that preserves the public entry point for older clients, while making the declaration inlinable for newer clients. This will likely be a separate proposal and discussion.

5) It eliminates this complexity.

It wouldn't avoid the complexity, because we want the "non-ABI, always-emit-into-client" behavior for the standard library. For the soon-to-be-ABI-stable libraries where @inlinable even matters, such as the standard library and Apple SDK overlays, there's pretty much perfect overlap between things we want to inline and things we don't want to take up binary space and ABI surface in binaries, so the behavior Slava proposes seems like the right default.

I disagree. The semantics being proposed perfectly overlap with the transitional plan for overlays (which matters for the next few years), but they are the wrong default for anything other than overlays and the wrong thing for long term API evolution over the next 20 years.

I disagree with this. 'inline' functions in C and C++ have to be backed by a symbol in the binary in order to guarantee function pointer identity, but we don't have that constraint. Without that constraint, there's almost no way that having a fallback definition in the binary is better:

- It becomes an ABI compatibility liability that has to be preserved forever.
- It increases binary size for a function that's rarely used, and which is often much larger as an outlined generic function than the simple operation that can be inlined into client code. Inlining makes the most sense when the inlined operation is smaller than a function call, so in many cases the net dylib + executable size would increase.
- It increases the uncertainty of the behavior client code sees. If an inlinable function must always be emitted in the client, then client code *always* gets the current definition. If an inlinable function calls into the dylib when the compiler chooses not to inline it, then you may get the current definition, or you may get an older definition from any published version of the dylib. Ideally these all behave the same if the function is inlinable, but quirks are going to be inevitable.

-Joe

···

On Oct 3, 2017, at 9:56 PM, Chris Lattner <clattner@nondot.org> wrote:

On Oct 3, 2017, at 9:50 AM, Joe Groff <jgroff@apple.com> wrote:

On Oct 2, 2017, at 10:58 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

-Chris

[Joe_Groff]

The major question I have is “why yet another attribute”. The thread about exhaustive/extensible enums is similarly proposing introducing another one-off way to be enums fragile, and this is directly related just for function-like things.

I’d love to see rationale in the proposal for why you’re not taking this in one of these directions:

1) Why not another level of access control? There is a reasonable argument that what you’re doing is making something “more public than public” or that you’re making the “body also public”. I’m not strongly in favor of this design approach, but if you agree, the doc should explain why you’re not in favor of it.

2) Why can’t we have a single Swift-wide concept that unifies all of the resilience ideas under a single umbrella like “fragile” - which indicates that the body of a declaration is knowable to clients? There is a very reasonable holistic design where “fragile public func” makes its body inlinable, and “fragile enum” similarly makes the cases knowable to the client (thus making exhaustive switching a possibility). I am strongly in favor of this approach.

I'm of two minds on this. So far, it has seemingly been the case that, for every kind of declaration, there's been exactly one resilience-related attribute, and with blurry enough vision you could say they're all really doing the same thing, "expose this declaration body directly to clients", which only makes sense as a concept for public API. If these properties hold, then sure, it might make sense to consider these all as one "extra-public" concept. On the other hand, we don't know yet whether these properties will hold in the long term as we're actively designing the facilities we need for ABI stability and resilience. From an incremental, iterative design perspective, I think it's wiser to keep them all separate attributes, even if it's a bit messy in the short term. Once the design stabilizes, we can look at whether it makes sense to fold related concepts together. It's easier to fold different things together after the fact then to try to separate things after they've been mixed together.

As I said above, I pretty strongly disagree with this approach. It encourages a “one attribute/declmodifier/keyword per decl kind” approach, which is user hostile. It doesn’t make sense for “exhaustive" to apply to enums, "@inlinable” to apply to function-like things, something else to apply to structs, something else to apply to classes, etc.

Premature abstraction is user-hostile too. These things all have subtly different impacts on language semantics, some of which we're still discovering as we develop their designs.

-Joe

···

On Oct 3, 2017, at 10:03 PM, Chris Lattner <clattner@nondot.org> wrote:

On Oct 3, 2017, at 10:04 AM, Joe Groff <jgroff@apple.com> wrote:

On Oct 2, 2017, at 10:58 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

I’d rather that we define one “thing” which can be parameterized. I’d propose that that be named “fragile” because it appropriately carries negative connotations. If some declaration has multiple levels of fragility that we want to support, we can allow it to be parameterized, e.g.:

/// C style fragility, all things about the decl are fragile.
fragile struct Foo { … }

/// Just layout is fragile, but not other things (whatever they would be?)
fragile(layout) struct Foo { … }

I pretty strongly believe that we should start with the default of having a unifying principle that ties together all resilience concerns under a single umbrella, and fission those concerns when we’re faced with something that forces it. Starting with them split and hoping that we’ll come around to merging them someday is the wrong side of optimism IMO.

We’ve been talking about this approach for years, which is why I’m somewhat surprised to see the new direction these things are taking.

-Chris

[Nevin]

This is unduly restrictive; @_versioned (despite being the wrong spelling)
is what we want here. To be callable from an inlinable function, internal
things need only be visible in terms of public ABI, not necessarily
inlinable, just as public things need only be public and not necessarily
inlinable.

Unduly restrictive is fine for now, we don’t need to make @_versioned ready
for primetime just to roll out @inlinable. We can begin with the semantics
that @inlinable functions can only call things which are themselves either
@inlinable or public, and then subsequently in the future we can expand it
to “…or whatever-we-end-up-calling-@_versioned”.

That way the scope of this proposal can stay narrow and self-contained.

Nevin

···

On Mon, Oct 2, 2017 at 5:45 PM, Xiaodi Wu via swift-evolution < swift-evolution@swift.org> wrote:

On Mon, Oct 2, 2017 at 16:37 Nevin Brackett-Rozinsky via swift-evolution < > swift-evolution@swift.org> wrote:

On Mon, Oct 2, 2017 at 5:21 PM, Slava Pestov <spestov@apple.com> wrote:

Thanks for taking a look!

> On Oct 2, 2017, at 2:19 PM, Nevin Brackett-Rozinsky < >>> nevin.brackettrozinsky@gmail.com> wrote:
> 3. Even though @inlinable will have no effect on declarations which
are not public, we should still allow it to be placed there. That way when
the access level is later changed to be public, the attribute is already
where it should be. This is similar to why we permit, eg., members of an
internal type to be declared public, which was discussed and decided
previously on Swift Evolution.

This is an interesting point. Do you think the attribute should be
completely ignored, or should the restrictions on references to non-public
things, etc still be enforced?

Hmm, good question!

I rather like the idea Greg Parker put forth, where non-public @inlinable
items can be used by public @inlinable ones, which implies that the
restrictions should indeed still apply—something @inlinable can only
reference public or @inlinable things.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[taylorswift]

Right now @_versioned is only for internal declarations. We should have
something similar for private and fileprivate declarations. I think most
people use those modifiers for code organization, not binary resilience, so
we would do well to make the two intents separate and explicit.

···

On Mon, Oct 2, 2017 at 6:42 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Mon, Oct 2, 2017 at 17:41 Taylor Swift <kelvin13ma@gmail.com> wrote:

I think we should try to separate visibility from access control. In
other words, the compiler should be able to see more than the user. I want
to be able to write private and internal code that cannot be called
explicitly in source, but can still be inlined by the compiler. Right now
people are doing this with underscored methods and variable names but I
don’t think that’s a good convention to use. We should have something at
the language level that enforces that something shouldn’t be referenced by
name outside of its scope, but is public for all compilation and ABI
purposes. Maybe an attribute like @visible or a new keyword or something.

Right, that’s @_versioned, essentially.

On Mon, Oct 2, 2017 at 4:45 PM, Xiaodi Wu via swift-evolution < >> swift-evolution@swift.org> wrote:

This is unduly restrictive; @_versioned (despite being the wrong
spelling) is what we want here. To be callable from an inlinable function,
internal things need only be visible in terms of public ABI, not
necessarily inlinable, just as public things need only be public and not
necessarily inlinable.
On Mon, Oct 2, 2017 at 16:37 Nevin Brackett-Rozinsky via swift-evolution >>> <swift-evolution@swift.org> wrote:

On Mon, Oct 2, 2017 at 5:21 PM, Slava Pestov <spestov@apple.com> wrote:

Thanks for taking a look!

> On Oct 2, 2017, at 2:19 PM, Nevin Brackett-Rozinsky < >>>>> nevin.brackettrozinsky@gmail.com> wrote:
> 3. Even though @inlinable will have no effect on declarations which
are not public, we should still allow it to be placed there. That way when
the access level is later changed to be public, the attribute is already
where it should be. This is similar to why we permit, eg., members of an
internal type to be declared public, which was discussed and decided
previously on Swift Evolution.

This is an interesting point. Do you think the attribute should be
completely ignored, or should the restrictions on references to non-public
things, etc still be enforced?

Hmm, good question!

I rather like the idea Greg Parker put forth, where non-public
@inlinable items can be used by public @inlinable ones, which implies that
the restrictions should indeed still apply—something @inlinable can only
reference public or @inlinable things.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Slava_Pestov]

It would be a trivial change to allow @_versioned on private and fileprivate declarations, but there are two pitfalls to keep in mind:

- Private symbols are mangled with a ‘discriminator’ which is basically a hash of the file name. So now it would be part of the ABI, which seems fragile — you can’t move the private function to another source file, or rename the source file.

- Similarly, right now a @_versioned function becoming public is an ABI compatible change. This would no longer work if you could have private @_versioned functions, because the symbol name would change if it became public.

For these reasons we decided against “private versioned” as a concept. I feel like internal is enough here.

Slava

···

On Oct 2, 2017, at 4:54 PM, Taylor Swift <kelvin13ma@gmail.com> wrote:

Right now @_versioned is only for internal declarations. We should have something similar for private and fileprivate declarations. I think most people use those modifiers for code organization, not binary resilience, so we would do well to make the two intents separate and explicit.

On Mon, Oct 2, 2017 at 6:42 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Mon, Oct 2, 2017 at 17:41 Taylor Swift <kelvin13ma@gmail.com <mailto:kelvin13ma@gmail.com>> wrote:
I think we should try to separate visibility from access control. In other words, the compiler should be able to see more than the user. I want to be able to write private and internal code that cannot be called explicitly in source, but can still be inlined by the compiler. Right now people are doing this with underscored methods and variable names but I don’t think that’s a good convention to use. We should have something at the language level that enforces that something shouldn’t be referenced by name outside of its scope, but is public for all compilation and ABI purposes. Maybe an attribute like @visible or a new keyword or something.

Right, that’s @_versioned, essentially.

On Mon, Oct 2, 2017 at 4:45 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
This is unduly restrictive; @_versioned (despite being the wrong spelling) is what we want here. To be callable from an inlinable function, internal things need only be visible in terms of public ABI, not necessarily inlinable, just as public things need only be public and not necessarily inlinable.
On Mon, Oct 2, 2017 at 16:37 Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Mon, Oct 2, 2017 at 5:21 PM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:
Thanks for taking a look!

> On Oct 2, 2017, at 2:19 PM, Nevin Brackett-Rozinsky <nevin.brackettrozinsky@gmail.com <mailto:nevin.brackettrozinsky@gmail.com>> wrote:
> 3. Even though @inlinable will have no effect on declarations which are not public, we should still allow it to be placed there. That way when the access level is later changed to be public, the attribute is already where it should be. This is similar to why we permit, eg., members of an internal type to be declared public, which was discussed and decided previously on Swift Evolution.

This is an interesting point. Do you think the attribute should be completely ignored, or should the restrictions on references to non-public things, etc still be enforced?

Hmm, good question!

I rather like the idea Greg Parker put forth, where non-public @inlinable items can be used by public @inlinable ones, which implies that the restrictions should indeed still apply—something @inlinable can only reference public or @inlinable things.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[Slava_Pestov]

Is this only a problem with fileprivate or does it extend to private members too? I feel like this would be a very valuable feature to support.

Private members too. Consider this example,

struct S {
  private func f() {}
}

The member S.f mangles as _T06struct1SV1f33_AB643CAAAE0894CD0BC8584D7CA3AD23LLyyF. In this case, I suppose we won’t need the private discriminator because there can only be one S.f that’s directly a member of S, and not an extension. However imagine if two different source files both defined extensions of S, with a private member f. You would need to disambiguate them somehow.

Slava

···

On Oct 2, 2017, at 7:52 PM, Kelvin Ma <kelvin13ma@gmail.com> wrote:

On Mon, Oct 2, 2017 at 9:43 PM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:
It would be a trivial change to allow @_versioned on private and fileprivate declarations, but there are two pitfalls to keep in mind:

- Private symbols are mangled with a ‘discriminator’ which is basically a hash of the file name. So now it would be part of the ABI, which seems fragile — you can’t move the private function to another source file, or rename the source file.

- Similarly, right now a @_versioned function becoming public is an ABI compatible change. This would no longer work if you could have private @_versioned functions, because the symbol name would change if it became public.

For these reasons we decided against “private versioned” as a concept. I feel like internal is enough here.

Slava

On Oct 2, 2017, at 4:54 PM, Taylor Swift <kelvin13ma@gmail.com <mailto:kelvin13ma@gmail.com>> wrote:

Right now @_versioned is only for internal declarations. We should have something similar for private and fileprivate declarations. I think most people use those modifiers for code organization, not binary resilience, so we would do well to make the two intents separate and explicit.

On Mon, Oct 2, 2017 at 6:42 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Mon, Oct 2, 2017 at 17:41 Taylor Swift <kelvin13ma@gmail.com <mailto:kelvin13ma@gmail.com>> wrote:
I think we should try to separate visibility from access control. In other words, the compiler should be able to see more than the user. I want to be able to write private and internal code that cannot be called explicitly in source, but can still be inlined by the compiler. Right now people are doing this with underscored methods and variable names but I don’t think that’s a good convention to use. We should have something at the language level that enforces that something shouldn’t be referenced by name outside of its scope, but is public for all compilation and ABI purposes. Maybe an attribute like @visible or a new keyword or something.

Right, that’s @_versioned, essentially.

On Mon, Oct 2, 2017 at 4:45 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
This is unduly restrictive; @_versioned (despite being the wrong spelling) is what we want here. To be callable from an inlinable function, internal things need only be visible in terms of public ABI, not necessarily inlinable, just as public things need only be public and not necessarily inlinable.
On Mon, Oct 2, 2017 at 16:37 Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Mon, Oct 2, 2017 at 5:21 PM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:
Thanks for taking a look!

> On Oct 2, 2017, at 2:19 PM, Nevin Brackett-Rozinsky <nevin.brackettrozinsky@gmail.com <mailto:nevin.brackettrozinsky@gmail.com>> wrote:
> 3. Even though @inlinable will have no effect on declarations which are not public, we should still allow it to be placed there. That way when the access level is later changed to be public, the attribute is already where it should be. This is similar to why we permit, eg., members of an internal type to be declared public, which was discussed and decided previously on Swift Evolution.

This is an interesting point. Do you think the attribute should be completely ignored, or should the restrictions on references to non-public things, etc still be enforced?

Hmm, good question!

I rather like the idea Greg Parker put forth, where non-public @inlinable items can be used by public @inlinable ones, which implies that the restrictions should indeed still apply—something @inlinable can only reference public or @inlinable things.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[Slava_Pestov]

Is this only a problem with fileprivate or does it extend to private members too? I feel like this would be a very valuable feature to support.

Private members too. Consider this example,

struct S {
  private func f() {}
}

The member S.f mangles as _T06struct1SV1f33_AB643CAAAE0894CD0BC8584D7CA3AD23LLyyF. In this case, I suppose we won’t need the private discriminator because there can only be one S.f that’s directly a member of S, and not an extension. However imagine if two different source files both defined extensions of S, with a private member f. You would need to disambiguate them somehow.

The simple-minded way to do this would be to require @_versioned annotations on private and fileprivate members to supply an internally unique alternative name to be used for mangling-as-though-internal (i.e. `@_versioned(my_extension_f)`). Such a function becoming public in an ABI-compatible way would require renaming the "actual" name to the unique @_versioned name.

We have _silgen_name for that, but we really don’t want to expose this more generally because people have been abusing it to make things visible to C, and they should be using @_cdecl instead.

A more elegant refinement could be to have @_versioned private and fileprivate members mangled as though internal, erroring if two or more members with the same name are both @_versioned--would that work?

If you’re going to do that what is the value in having the capability at all?

Slava

···

On Oct 2, 2017, at 8:06 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Mon, Oct 2, 2017 at 9:55 PM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:

On Oct 2, 2017, at 7:52 PM, Kelvin Ma <kelvin13ma@gmail.com <mailto:kelvin13ma@gmail.com>> wrote:

[taylorswift]

Is this only a problem with fileprivate or does it extend to private
members too? I feel like this would be a very valuable feature to support.

···

On Mon, Oct 2, 2017 at 9:43 PM, Slava Pestov <spestov@apple.com> wrote:

It would be a trivial change to allow @_versioned on private and
fileprivate declarations, but there are two pitfalls to keep in mind:

- Private symbols are mangled with a ‘discriminator’ which is basically a
hash of the file name. So now it would be part of the ABI, which seems
fragile — you can’t move the private function to another source file, or
rename the source file.

- Similarly, right now a @_versioned function becoming public is an ABI
compatible change. This would no longer work if you could have private
@_versioned functions, because the symbol name would change if it became
public.

For these reasons we decided against “private versioned” as a concept. I
feel like internal is enough here.

Slava

On Oct 2, 2017, at 4:54 PM, Taylor Swift <kelvin13ma@gmail.com> wrote:

Right now @_versioned is only for internal declarations. We should have
something similar for private and fileprivate declarations. I think most
people use those modifiers for code organization, not binary resilience, so
we would do well to make the two intents separate and explicit.

On Mon, Oct 2, 2017 at 6:42 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Mon, Oct 2, 2017 at 17:41 Taylor Swift <kelvin13ma@gmail.com> wrote:

I think we should try to separate visibility from access control. In
other words, the compiler should be able to see more than the user. I want
to be able to write private and internal code that cannot be called
explicitly in source, but can still be inlined by the compiler. Right now
people are doing this with underscored methods and variable names but I
don’t think that’s a good convention to use. We should have something at
the language level that enforces that something shouldn’t be referenced by
name outside of its scope, but is public for all compilation and ABI
purposes. Maybe an attribute like @visible or a new keyword or something.

Right, that’s @_versioned, essentially.

On Mon, Oct 2, 2017 at 4:45 PM, Xiaodi Wu via swift-evolution < >>> swift-evolution@swift.org> wrote:

This is unduly restrictive; @_versioned (despite being the wrong
spelling) is what we want here. To be callable from an inlinable function,
internal things need only be visible in terms of public ABI, not
necessarily inlinable, just as public things need only be public and not
necessarily inlinable.
On Mon, Oct 2, 2017 at 16:37 Nevin Brackett-Rozinsky via >>>> swift-evolution <swift-evolution@swift.org> wrote:

On Mon, Oct 2, 2017 at 5:21 PM, Slava Pestov <spestov@apple.com> >>>>> wrote:

Thanks for taking a look!

> On Oct 2, 2017, at 2:19 PM, Nevin Brackett-Rozinsky < >>>>>> nevin.brackettrozinsky@gmail.com> wrote:
> 3. Even though @inlinable will have no effect on declarations which
are not public, we should still allow it to be placed there. That way when
the access level is later changed to be public, the attribute is already
where it should be. This is similar to why we permit, eg., members of an
internal type to be declared public, which was discussed and decided
previously on Swift Evolution.

This is an interesting point. Do you think the attribute should be
completely ignored, or should the restrictions on references to non-public
things, etc still be enforced?

Hmm, good question!

I rather like the idea Greg Parker put forth, where non-public
@inlinable items can be used by public @inlinable ones, which implies that
the restrictions should indeed still apply—something @inlinable can only
reference public or @inlinable things.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[xwu (Xiaodi Wu)]

The simple-minded way to do this would be to require @_versioned
annotations on private and fileprivate members to supply an internally
unique alternative name to be used for mangling-as-though-internal (i.e.
`@_versioned(my_extension_f)`). Such a function becoming public in an
ABI-compatible way would require renaming the "actual" name to the unique
@_versioned name.

A more elegant refinement could be to have @_versioned private and
fileprivate members mangled as though internal, erroring if two or more
members with the same name are both @_versioned--would that work?

···

On Mon, Oct 2, 2017 at 9:55 PM, Slava Pestov <spestov@apple.com> wrote:

On Oct 2, 2017, at 7:52 PM, Kelvin Ma <kelvin13ma@gmail.com> wrote:

Is this only a problem with fileprivate or does it extend to private
members too? I feel like this would be a very valuable feature to support.

Private members too. Consider this example,

struct S {
  private func f() {}
}

The member S.f mangles as _T06struct1SV1f33_AB643CAAAE0894CD0BC8584D7CA3AD23LLyyF.
In this case, I suppose we won’t need the private discriminator because
there can only be one S.f that’s directly a member of S, and not an
extension. However imagine if two different source files both defined
extensions of S, with a private member f. You would need to disambiguate
them somehow.

[xwu (Xiaodi Wu)]

Sounds reasonable to me.

···

On Mon, Oct 2, 2017 at 18:54 Taylor Swift <kelvin13ma@gmail.com> wrote:

Right now @_versioned is only for internal declarations. We should have
something similar for private and fileprivate declarations. I think most
people use those modifiers for code organization, not binary resilience, so
we would do well to make the two intents separate and explicit.

On Mon, Oct 2, 2017 at 6:42 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Mon, Oct 2, 2017 at 17:41 Taylor Swift <kelvin13ma@gmail.com> wrote:

I think we should try to separate visibility from access control. In
other words, the compiler should be able to see more than the user. I want
to be able to write private and internal code that cannot be called
explicitly in source, but can still be inlined by the compiler. Right now
people are doing this with underscored methods and variable names but I
don’t think that’s a good convention to use. We should have something at
the language level that enforces that something shouldn’t be referenced by
name outside of its scope, but is public for all compilation and ABI
purposes. Maybe an attribute like @visible or a new keyword or something.

Right, that’s @_versioned, essentially.

On Mon, Oct 2, 2017 at 4:45 PM, Xiaodi Wu via swift-evolution < >>> swift-evolution@swift.org> wrote:

This is unduly restrictive; @_versioned (despite being the wrong
spelling) is what we want here. To be callable from an inlinable function,
internal things need only be visible in terms of public ABI, not
necessarily inlinable, just as public things need only be public and not
necessarily inlinable.
On Mon, Oct 2, 2017 at 16:37 Nevin Brackett-Rozinsky via >>>> swift-evolution <swift-evolution@swift.org> wrote:

On Mon, Oct 2, 2017 at 5:21 PM, Slava Pestov <spestov@apple.com> >>>>> wrote:

Thanks for taking a look!

> On Oct 2, 2017, at 2:19 PM, Nevin Brackett-Rozinsky < >>>>>> nevin.brackettrozinsky@gmail.com> wrote:
> 3. Even though @inlinable will have no effect on declarations which
are not public, we should still allow it to be placed there. That way when
the access level is later changed to be public, the attribute is already
where it should be. This is similar to why we permit, eg., members of an
internal type to be declared public, which was discussed and decided
previously on Swift Evolution.

This is an interesting point. Do you think the attribute should be
completely ignored, or should the restrictions on references to non-public
things, etc still be enforced?

Hmm, good question!

I rather like the idea Greg Parker put forth, where non-public
@inlinable items can be used by public @inlinable ones, which implies that
the restrictions should indeed still apply—something @inlinable can only
reference public or @inlinable things.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution