[Pitch] Make the formal type of 'self' consistent in class methods


(Slava Pestov) #1

Consistent formal type for 'self' in class methods

Proposal: SE-9999 <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/9999-self-formal-type-in-class.md>
Author: Slava Pestov <https://github.com/slavapestov>
Status: Awaiting review
Review manager: TBD

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#introduction>Introduction

This proposal makes the self value behave consistently whether or not it is used from a method with a Self return type.

Swift-evolution thread: Discussion thread topic for that proposal <http://news.gmane.org/gmane.comp.lang.swift.evolution>
<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#motivation>Motivation

Right now, we exhibit inconsistent behavior when self is used as an argument to a generic function, violating the principle of least surprise.

Consider the following code:

class Base {
  @discardableResult
  func methodWithDynamicSelf() -> Self {
    doSomething(self)
    return self
  }

  func methodWithoutDynamicSelf() {
    doSomething(self)
  }
}

class Derived : Base {}

func doSomething<T>(_ t: T) {
  print(T.self)
}

Base().methodWithDynamicSelf()
Base().methodWithoutDynamicSelf()

Derived().methodWithDynamicSelf()
Derived().methodWithoutDynamicSelf()
Currently, it prints the following output:

Base
Base
Derived
Base
Note that there's no inconsistency when the method is called on the base class. When called on the derived class however, we see that in a method with a dynamic Self return type, the type of self is Derived, whereas in a method with any other return type, the type of self is Base.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#proposed-solution>Proposed solution

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

With this proposal, the above code will instead produce the following:

Base
Base
Derived
Derived
Here, the type of self would always be Derived when called on an instance of the derived class.

Of course a more useful program could instead do something with the type parameter T, such as constraining it to a protocol or a class with a required initializer, and then using the type to construct a new instance of the class.

This also dovetails nicely with SE-0068 <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/0068-universal-self.md>.

Finally, it opens the door to generalizing dynamic Self, allowing it to appear in covariant position within parameter types:

class ArtClass {
  func paint(withBrush: (Self) -> ()) { ... }
}
This would allow a class to conform to a protocol with a requirement written like the following, something that is currently not possible at all:

protocol OddProtocol {
  func weaken<X, Y>((Self) -> (X) -> Y) -> (X) -> Y
}

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#detailed-design>Detailed design

There's really not much more to say here. The code for typing self with a dynamic Self is in place already, however enabling this change might expose some new bugs we have not yet encountered, because currently, methods with dynamic Self return type are relatively rare.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#impact-on-existing-code>Impact on existing code

This will have a small impact on existing code that uses a pattern similar to the above.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#alternatives-considered>Alternatives considered

One alternative is to simply do nothing, but this makes the language less consistent than it could be.


(Andrew Trick) #2

We’re currently specializing functions that take `self` as an argument. I don’t think that will be possible after your proposed change.

- Andy

···

On Jun 23, 2016, at 12:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org> wrote:

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.


(Matthew Johnson) #3

+1. I have not encountered this issue myself but it looks like something that would cause a lot of head scratching if I had. It is also something that I am unlikely to remember immediately if I run into it in the future. The current behavior appears broken to me. It will be great to have it fixed.

···

On Jun 23, 2016, at 2:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org> wrote:

Consistent formal type for 'self' in class methods

Proposal: SE-9999 <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/9999-self-formal-type-in-class.md>
Author: Slava Pestov <https://github.com/slavapestov>
Status: Awaiting review
Review manager: TBD

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#introduction>Introduction

This proposal makes the self value behave consistently whether or not it is used from a method with a Self return type.

Swift-evolution thread: Discussion thread topic for that proposal <http://news.gmane.org/gmane.comp.lang.swift.evolution>
<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#motivation>Motivation

Right now, we exhibit inconsistent behavior when self is used as an argument to a generic function, violating the principle of least surprise.

Consider the following code:

class Base {
  @discardableResult
  func methodWithDynamicSelf() -> Self {
    doSomething(self)
    return self
  }

  func methodWithoutDynamicSelf() {
    doSomething(self)
  }
}

class Derived : Base {}

func doSomething<T>(_ t: T) {
  print(T.self)
}

Base().methodWithDynamicSelf()
Base().methodWithoutDynamicSelf()

Derived().methodWithDynamicSelf()
Derived().methodWithoutDynamicSelf()
Currently, it prints the following output:

Base
Base
Derived
Base
Note that there's no inconsistency when the method is called on the base class. When called on the derived class however, we see that in a method with a dynamic Self return type, the type of self is Derived, whereas in a method with any other return type, the type of self is Base.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#proposed-solution>Proposed solution

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

With this proposal, the above code will instead produce the following:

Base
Base
Derived
Derived
Here, the type of self would always be Derived when called on an instance of the derived class.

Of course a more useful program could instead do something with the type parameter T, such as constraining it to a protocol or a class with a required initializer, and then using the type to construct a new instance of the class.

This also dovetails nicely with SE-0068 <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/0068-universal-self.md>.

Finally, it opens the door to generalizing dynamic Self, allowing it to appear in covariant position within parameter types:

class ArtClass {
  func paint(withBrush: (Self) -> ()) { ... }
}
This would allow a class to conform to a protocol with a requirement written like the following, something that is currently not possible at all:

protocol OddProtocol {
  func weaken<X, Y>((Self) -> (X) -> Y) -> (X) -> Y
}

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#detailed-design>Detailed design

There's really not much more to say here. The code for typing self with a dynamic Self is in place already, however enabling this change might expose some new bugs we have not yet encountered, because currently, methods with dynamic Self return type are relatively rare.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#impact-on-existing-code>Impact on existing code

This will have a small impact on existing code that uses a pattern similar to the above.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#alternatives-considered>Alternatives considered

One alternative is to simply do nothing, but this makes the language less consistent than it could be.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Sean Heber) #4

This is certainly surprisingly behavior to me! +1

l8r
Sean

···

On Jun 23, 2016, at 2:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org> wrote:

Consistent formal type for 'self' in class methods

  • Proposal: SE-9999
  • Author: Slava Pestov
  • Status: Awaiting review
  • Review manager: TBD

Introduction

This proposal makes the self value behave consistently whether or not it is used from a method with a Self return type.

Swift-evolution thread: Discussion thread topic for that proposal

Motivation

Right now, we exhibit inconsistent behavior when self is used as an argument to a generic function, violating the principle of least surprise.

Consider the following code:

class
Base {
  
@discardableResult

func methodWithDynamicSelf() ->
Self {
    doSomething(
self
)
    
return self

  }

func methodWithoutDynamicSelf
() {
    doSomething(
self
)
  }
}

class Derived :
Base {}

func doSomething<T>(_ t
: T) {
  
print(T.self
)
}

Base()
.
methodWithDynamicSelf()
Base()
.
methodWithoutDynamicSelf()

Derived()
.
methodWithDynamicSelf()
Derived()
.methodWithoutDynamicSelf()
Currently, it prints the following output:

Base
Base
Derived
Base

Note that there's no inconsistency when the method is called on the base class. When called on the derived class however, we see that in a method with a dynamic Self return type, the type of self is Derived, whereas in a method with any other return type, the type of self is Base.

Proposed solution

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

With this proposal, the above code will instead produce the following:

Base
Base
Derived
Derived

Here, the type of self would always be Derived when called on an instance of the derived class.

Of course a more useful program could instead do something with the type parameter T, such as constraining it to a protocol or a class with a required initializer, and then using the type to construct a new instance of the class.

This also dovetails nicely with SE-0068.

Finally, it opens the door to generalizing dynamic Self, allowing it to appear in covariant position within parameter types:

class
ArtClass {
  
func paint(withBrush: (Self) -> ()) { ...
}
}

This would allow a class to conform to a protocol with a requirement written like the following, something that is currently not possible at all:

protocol
OddProtocol {
  
func weaken<X, Y>((Self) ->
(X) -> Y) -> (X) -> Y
}

Detailed design

There's really not much more to say here. The code for typing self with a dynamic Self is in place already, however enabling this change might expose some new bugs we have not yet encountered, because currently, methods with dynamic Self return type are relatively rare.

Impact on existing code

This will have a small impact on existing code that uses a pattern similar to the above.

Alternatives considered

One alternative is to simply do nothing, but this makes the language less consistent than it could be.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Douglas Gregor) #5

This looks great to me. Much more sane.

···

Sent from my iPhone

On Jun 23, 2016, at 12:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org> wrote:

Consistent formal type for 'self' in class methods
Proposal: SE-9999
Author: Slava Pestov
Status: Awaiting review
Review manager: TBD
Introduction

This proposal makes the self value behave consistently whether or not it is used from a method with a Self return type.

Swift-evolution thread: Discussion thread topic for that proposal

Motivation

Right now, we exhibit inconsistent behavior when self is used as an argument to a generic function, violating the principle of least surprise.

Consider the following code:

class Base {
  @discardableResult
  func methodWithDynamicSelf() -> Self {
    doSomething(self)
    return self
  }

  func methodWithoutDynamicSelf() {
    doSomething(self)
  }
}

class Derived : Base {}

func doSomething<T>(_ t: T) {
  print(T.self)
}

Base().methodWithDynamicSelf()
Base().methodWithoutDynamicSelf()

Derived().methodWithDynamicSelf()
Derived().methodWithoutDynamicSelf()
Currently, it prints the following output:

Base
Base
Derived
Base
Note that there's no inconsistency when the method is called on the base class. When called on the derived class however, we see that in a method with a dynamic Self return type, the type of self is Derived, whereas in a method with any other return type, the type of self is Base.

Proposed solution

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

With this proposal, the above code will instead produce the following:

Base
Base
Derived
Derived
Here, the type of self would always be Derived when called on an instance of the derived class.

Of course a more useful program could instead do something with the type parameter T, such as constraining it to a protocol or a class with a required initializer, and then using the type to construct a new instance of the class.

This also dovetails nicely with SE-0068.

Finally, it opens the door to generalizing dynamic Self, allowing it to appear in covariant position within parameter types:

class ArtClass {
  func paint(withBrush: (Self) -> ()) { ... }
}
This would allow a class to conform to a protocol with a requirement written like the following, something that is currently not possible at all:

protocol OddProtocol {
  func weaken<X, Y>((Self) -> (X) -> Y) -> (X) -> Y
}
Detailed design

There's really not much more to say here. The code for typing self with a dynamic Self is in place already, however enabling this change might expose some new bugs we have not yet encountered, because currently, methods with dynamic Self return type are relatively rare.

Impact on existing code

This will have a small impact on existing code that uses a pattern similar to the above.

Alternatives considered

One alternative is to simply do nothing, but this makes the language less consistent than it could be.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(plx) #6

+1; I *really* appreciate taking a proactive approach to eliminating future sources of bizarre “gotcha” moments!

···

On Jun 23, 2016, at 2:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org> wrote:

Consistent formal type for 'self' in class methods

Proposal: SE-9999 <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/9999-self-formal-type-in-class.md>
Author: Slava Pestov <https://github.com/slavapestov>
Status: Awaiting review
Review manager: TBD

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#introduction>Introduction

This proposal makes the self value behave consistently whether or not it is used from a method with a Self return type.

Swift-evolution thread: Discussion thread topic for that proposal <http://news.gmane.org/gmane.comp.lang.swift.evolution>
<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#motivation>Motivation

Right now, we exhibit inconsistent behavior when self is used as an argument to a generic function, violating the principle of least surprise.

Consider the following code:

class Base {
  @discardableResult
  func methodWithDynamicSelf() -> Self {
    doSomething(self)
    return self
  }

  func methodWithoutDynamicSelf() {
    doSomething(self)
  }
}

class Derived : Base {}

func doSomething<T>(_ t: T) {
  print(T.self)
}

Base().methodWithDynamicSelf()
Base().methodWithoutDynamicSelf()

Derived().methodWithDynamicSelf()
Derived().methodWithoutDynamicSelf()
Currently, it prints the following output:

Base
Base
Derived
Base
Note that there's no inconsistency when the method is called on the base class. When called on the derived class however, we see that in a method with a dynamic Self return type, the type of self is Derived, whereas in a method with any other return type, the type of self is Base.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#proposed-solution>Proposed solution

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

With this proposal, the above code will instead produce the following:

Base
Base
Derived
Derived
Here, the type of self would always be Derived when called on an instance of the derived class.

Of course a more useful program could instead do something with the type parameter T, such as constraining it to a protocol or a class with a required initializer, and then using the type to construct a new instance of the class.

This also dovetails nicely with SE-0068 <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/0068-universal-self.md>.

Finally, it opens the door to generalizing dynamic Self, allowing it to appear in covariant position within parameter types:

class ArtClass {
  func paint(withBrush: (Self) -> ()) { ... }
}
This would allow a class to conform to a protocol with a requirement written like the following, something that is currently not possible at all:

protocol OddProtocol {
  func weaken<X, Y>((Self) -> (X) -> Y) -> (X) -> Y
}

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#detailed-design>Detailed design

There's really not much more to say here. The code for typing self with a dynamic Self is in place already, however enabling this change might expose some new bugs we have not yet encountered, because currently, methods with dynamic Self return type are relatively rare.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#impact-on-existing-code>Impact on existing code

This will have a small impact on existing code that uses a pattern similar to the above.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#alternatives-considered>Alternatives considered

One alternative is to simply do nothing, but this makes the language less consistent than it could be.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Michael Peternell) #7

I'm not sure if this even requires a proposal. This just looks like an ordinary bug in the language implementation. But if this would become a formal proposal, I will support it.

-Michael

···

Am 23.06.2016 um 21:53 schrieb Slava Pestov via swift-evolution <swift-evolution@swift.org>:

Consistent formal type for 'self' in class methods

  • Proposal: SE-9999
  • Author: Slava Pestov
  • Status: Awaiting review
  • Review manager: TBD

Introduction

This proposal makes the self value behave consistently whether or not it is used from a method with a Self return type.

Swift-evolution thread: Discussion thread topic for that proposal

Motivation

Right now, we exhibit inconsistent behavior when self is used as an argument to a generic function, violating the principle of least surprise.

Consider the following code:

class
Base {
  
@discardableResult

func methodWithDynamicSelf() ->
Self {
    doSomething(
self
)
    
return self

  }

func methodWithoutDynamicSelf
() {
    doSomething(
self
)
  }
}

class Derived :
Base {}

func doSomething<T>(_ t
: T) {
  
print(T.self
)
}

Base()
.
methodWithDynamicSelf()
Base()
.
methodWithoutDynamicSelf()

Derived()
.
methodWithDynamicSelf()
Derived()
.methodWithoutDynamicSelf()
Currently, it prints the following output:

Base
Base
Derived
Base

Note that there's no inconsistency when the method is called on the base class. When called on the derived class however, we see that in a method with a dynamic Self return type, the type of self is Derived, whereas in a method with any other return type, the type of self is Base.

Proposed solution

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

With this proposal, the above code will instead produce the following:

Base
Base
Derived
Derived

Here, the type of self would always be Derived when called on an instance of the derived class.

Of course a more useful program could instead do something with the type parameter T, such as constraining it to a protocol or a class with a required initializer, and then using the type to construct a new instance of the class.

This also dovetails nicely with SE-0068.

Finally, it opens the door to generalizing dynamic Self, allowing it to appear in covariant position within parameter types:

class
ArtClass {
  
func paint(withBrush: (Self) -> ()) { ...
}
}

This would allow a class to conform to a protocol with a requirement written like the following, something that is currently not possible at all:

protocol
OddProtocol {
  
func weaken<X, Y>((Self) ->
(X) -> Y) -> (X) -> Y
}

Detailed design

There's really not much more to say here. The code for typing self with a dynamic Self is in place already, however enabling this change might expose some new bugs we have not yet encountered, because currently, methods with dynamic Self return type are relatively rare.

Impact on existing code

This will have a small impact on existing code that uses a pattern similar to the above.

Alternatives considered

One alternative is to simply do nothing, but this makes the language less consistent than it could be.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Vladimir) #8

+1. This should be fixed.

···

On 23.06.2016 22:53, Slava Pestov via swift-evolution wrote:

  Consistent formal type for 'self' in class methods

    * Proposal: SE-9999
      <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/9999-self-formal-type-in-class.md>
    * Author: Slava Pestov <https://github.com/slavapestov>
    * Status: Awaiting review
    * Review manager: TBD

    <https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#introduction>Introduction

  This proposal makes the |self| value behave consistently whether or not
  it is used from a method with a |Self| return type.

  Swift-evolution thread: Discussion thread topic for that proposal
  <http://news.gmane.org/gmane.comp.lang.swift.evolution>

    <https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#motivation>Motivation

  Right now, we exhibit inconsistent behavior when |self| is used as an
  argument to a generic function, violating the principle of least surprise.

  Consider the following code:

  class Base {
    @discardableResult
    func methodWithDynamicSelf() -> Self {
      doSomething(self)
      return self
    }

    func methodWithoutDynamicSelf() {
      doSomething(self)
    }
  }

  class Derived : Base {}

  func doSomething<T>(_ t: T) {
    print(T.self)
  }

  Base().methodWithDynamicSelf()
  Base().methodWithoutDynamicSelf()

  Derived().methodWithDynamicSelf()
  Derived().methodWithoutDynamicSelf()

  Currently, it prints the following output:

  Base
  Derived
  Base

  Note that there's no inconsistency when the method is called on the base
  class. When called on the derived class however, we see that in a method
  with a dynamic |Self| return type, the type of |self| is |Derived|,
  whereas in a method with any other return type, the type of |self| is |Base|.

    <https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#proposed-solution>Proposed
    solution

  The proposal is to change the type of |self| to always be |Self|, which
  can be thought of as a special generic type parameter bound to the
  dynamic type of the instance.

  With this proposal, the above code will instead produce the following:

  Base
  Derived

  Here, the type of |self| would always be |Derived| when called on an
  instance of the derived class.

  Of course a more useful program could instead do something with the type
  parameter |T|, such as constraining it to a protocol or a class with a
  required initializer, and then using the type to construct a new instance
  of the class.

  This also dovetails nicely with SE-0068
  <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/0068-universal-self.md>.

  Finally, it opens the door to generalizing dynamic |Self|, allowing it to
  appear in covariant position within parameter types:

  class ArtClass {
    func paint(withBrush: (Self) -> ()) { ... }
  }

  This would allow a class to conform to a protocol with a requirement
  written like the following, something that is currently not possible at all:

  protocol OddProtocol {
    func weaken<X, Y>((Self) -> (X) -> Y) -> (X) -> Y
  }

    <https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#detailed-design>Detailed
    design

  There's really not much more to say here. The code for typing |self| with
  a dynamic |Self| is in place already, however enabling this change might
  expose some new bugs we have not yet encountered, because currently,
  methods with dynamic |Self| return type are relatively rare.

    <https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#impact-on-existing-code>Impact
    on existing code

  This will have a small impact on existing code that uses a pattern
  similar to the above.

    <https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#alternatives-considered>Alternatives
    considered

  One alternative is to simply do nothing, but this makes the language less
  consistent than it could be.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Slava Pestov) #9

Great to hear some feedback so quickly, especially about something so mundane.

I suspect the real reason it doesn’t work this way now is that ‘Self’ is not fully plumbed through. In particular, if a closure captures the ‘Self’ type, IRGen does not properly codegen it, causing compile-time or run-time crashes:

class MyClass {
  func foo(x: Int) -> Self {

    // Crash!
    _ = { print(self); print(x) }

    return self
  }

  func bar(x: Int) -> MyClass {

    // OK!
    _ = { print(self); print(x) }

    return self
  }
}

Assertion failed: (LocalSelf && "no local self metadata"), function getLocalSelfMetadata, file /Users/slava/new/swift/lib/IRGen/GenType.cpp, line 1805.
0 swift 0x0000000114d5276e llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 46
1 swift 0x0000000114d52c99 PrintStackTraceSignalHandler(void*) + 25
2 swift 0x0000000114d4efc9 llvm::sys::RunSignalHandlers() + 425
3 swift 0x0000000114d53312 SignalHandler(int) + 354
4 libsystem_platform.dylib 0x00007fffc438a01a _sigtramp + 26
5 libsystem_platform.dylib 0x00000000507ca710 _sigtramp + 2353268496
6 swift 0x0000000114d52cbb raise + 27
7 swift 0x0000000114d52d62 abort + 18
8 swift 0x0000000114d52d4e __assert_rtn + 126
9 swift 0x000000010f6e8524 swift::irgen::IRGenFunction::getLocalSelfMetadata() + 100

This comes up most frequently with the ‘weak self / strong self’ dance.

I’m going to fix this bug really soon, and it seems logical to deal with the language wart as well. We need the IRGen fix for SE-0086 as well in any case.

Slava

···

On Jun 23, 2016, at 1:08 PM, Matthew Johnson <matthew@anandabits.com> wrote:

+1. I have not encountered this issue myself but it looks like something that would cause a lot of head scratching if I had. It is also something that I am unlikely to remember immediately if I run into it in the future. The current behavior appears broken to me. It will be great to have it fixed.

On Jun 23, 2016, at 2:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Consistent formal type for 'self' in class methods

Proposal: SE-9999 <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/9999-self-formal-type-in-class.md>
Author: Slava Pestov <https://github.com/slavapestov>
Status: Awaiting review
Review manager: TBD

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#introduction>Introduction

This proposal makes the self value behave consistently whether or not it is used from a method with a Self return type.

Swift-evolution thread: Discussion thread topic for that proposal <http://news.gmane.org/gmane.comp.lang.swift.evolution>
<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#motivation>Motivation

Right now, we exhibit inconsistent behavior when self is used as an argument to a generic function, violating the principle of least surprise.

Consider the following code:

class Base {
  @discardableResult
  func methodWithDynamicSelf() -> Self {
    doSomething(self)
    return self
  }

  func methodWithoutDynamicSelf() {
    doSomething(self)
  }
}

class Derived : Base {}

func doSomething<T>(_ t: T) {
  print(T.self)
}

Base().methodWithDynamicSelf()
Base().methodWithoutDynamicSelf()

Derived().methodWithDynamicSelf()
Derived().methodWithoutDynamicSelf()
Currently, it prints the following output:

Base
Base
Derived
Base
Note that there's no inconsistency when the method is called on the base class. When called on the derived class however, we see that in a method with a dynamic Self return type, the type of self is Derived, whereas in a method with any other return type, the type of self is Base.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#proposed-solution>Proposed solution

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

With this proposal, the above code will instead produce the following:

Base
Base
Derived
Derived
Here, the type of self would always be Derived when called on an instance of the derived class.

Of course a more useful program could instead do something with the type parameter T, such as constraining it to a protocol or a class with a required initializer, and then using the type to construct a new instance of the class.

This also dovetails nicely with SE-0068 <https://github.com/slavapestov/swift-evolution/blob/self-formal-type-in-class/proposals/0068-universal-self.md>.

Finally, it opens the door to generalizing dynamic Self, allowing it to appear in covariant position within parameter types:

class ArtClass {
  func paint(withBrush: (Self) -> ()) { ... }
}
This would allow a class to conform to a protocol with a requirement written like the following, something that is currently not possible at all:

protocol OddProtocol {
  func weaken<X, Y>((Self) -> (X) -> Y) -> (X) -> Y
}

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#detailed-design>Detailed design

There's really not much more to say here. The code for typing self with a dynamic Self is in place already, however enabling this change might expose some new bugs we have not yet encountered, because currently, methods with dynamic Self return type are relatively rare.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#impact-on-existing-code>Impact on existing code

This will have a small impact on existing code that uses a pattern similar to the above.

<https://github.com/slavapestov/swift-evolution/tree/self-formal-type-in-class#alternatives-considered>Alternatives considered

One alternative is to simply do nothing, but this makes the language less consistent than it could be.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Slava Pestov) #10

I’m not sure what that means. Do you currently punt on certain optimizations if a method returns ‘Self’?

It should be possible to keep the reified type information around, by passing in a metatype or something for example. Can you give a concrete code snippet demonstrating the optimization and how this change would inhibit it?

Slava

···

On Jun 23, 2016, at 1:46 PM, Andrew Trick <atrick@apple.com> wrote:

On Jun 23, 2016, at 12:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

We’re currently specializing functions that take `self` as an argument. I don’t think that will be possible after your proposed change.

- Andy


(Matthew Johnson) #11

Great to hear some feedback so quickly, especially about something so mundane.

You caught me at the right time I guess. :slight_smile:

This kind of thing is only mundane until your code doesn't behave as you expect. Then it can become downright maddening! I would never in a million years guess that the return type would affect the interpretation of self in this way.

BTW, allowing Self to appear in covarying positions in argument types and the protocol conformances this will enable are a very nice bonus (even if it doesn't happen right away).

···

Sent from my iPad

On Jun 23, 2016, at 3:12 PM, Slava Pestov <spestov@apple.com> wrote:

I suspect the real reason it doesn’t work this way now is that ‘Self’ is not fully plumbed through. In particular, if a closure captures the ‘Self’ type, IRGen does not properly codegen it, causing compile-time or run-time crashes:

class MyClass {
  func foo(x: Int) -> Self {

    // Crash!
    _ = { print(self); print(x) }

    return self
  }

  func bar(x: Int) -> MyClass {

    // OK!
    _ = { print(self); print(x) }

    return self
  }
}

Assertion failed: (LocalSelf && "no local self metadata"), function getLocalSelfMetadata, file /Users/slava/new/swift/lib/IRGen/GenType.cpp, line 1805.
0 swift 0x0000000114d5276e llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 46
1 swift 0x0000000114d52c99 PrintStackTraceSignalHandler(void*) + 25
2 swift 0x0000000114d4efc9 llvm::sys::RunSignalHandlers() + 425
3 swift 0x0000000114d53312 SignalHandler(int) + 354
4 libsystem_platform.dylib 0x00007fffc438a01a _sigtramp + 26
5 libsystem_platform.dylib 0x00000000507ca710 _sigtramp + 2353268496
6 swift 0x0000000114d52cbb raise + 27
7 swift 0x0000000114d52d62 abort + 18
8 swift 0x0000000114d52d4e __assert_rtn + 126
9 swift 0x000000010f6e8524 swift::irgen::IRGenFunction::getLocalSelfMetadata() + 100

This comes up most frequently with the ‘weak self / strong self’ dance.

I’m going to fix this bug really soon, and it seems logical to deal with the language wart as well. We need the IRGen fix for SE-0086 as well in any case.

Slava

On Jun 23, 2016, at 1:08 PM, Matthew Johnson <matthew@anandabits.com> wrote:

+1. I have not encountered this issue myself but it looks like something that would cause a lot of head scratching if I had. It is also something that I am unlikely to remember immediately if I run into it in the future. The current behavior appears broken to me. It will be great to have it fixed.

On Jun 23, 2016, at 2:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org> wrote:

Consistent formal type for 'self' in class methods
Proposal: SE-9999
Author: Slava Pestov
Status: Awaiting review
Review manager: TBD
Introduction

This proposal makes the self value behave consistently whether or not it is used from a method with a Self return type.

Swift-evolution thread: Discussion thread topic for that proposal

Motivation

Right now, we exhibit inconsistent behavior when self is used as an argument to a generic function, violating the principle of least surprise.

Consider the following code:

class Base {
  @discardableResult
  func methodWithDynamicSelf() -> Self {
    doSomething(self)
    return self
  }

  func methodWithoutDynamicSelf() {
    doSomething(self)
  }
}

class Derived : Base {}

func doSomething<T>(_ t: T) {
  print(T.self)
}

Base().methodWithDynamicSelf()
Base().methodWithoutDynamicSelf()

Derived().methodWithDynamicSelf()
Derived().methodWithoutDynamicSelf()
Currently, it prints the following output:

Base
Base
Derived
Base
Note that there's no inconsistency when the method is called on the base class. When called on the derived class however, we see that in a method with a dynamic Self return type, the type of self is Derived, whereas in a method with any other return type, the type of self is Base.

Proposed solution

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

With this proposal, the above code will instead produce the following:

Base
Base
Derived
Derived
Here, the type of self would always be Derived when called on an instance of the derived class.

Of course a more useful program could instead do something with the type parameter T, such as constraining it to a protocol or a class with a required initializer, and then using the type to construct a new instance of the class.

This also dovetails nicely with SE-0068.

Finally, it opens the door to generalizing dynamic Self, allowing it to appear in covariant position within parameter types:

class ArtClass {
  func paint(withBrush: (Self) -> ()) { ... }
}
This would allow a class to conform to a protocol with a requirement written like the following, something that is currently not possible at all:

protocol OddProtocol {
  func weaken<X, Y>((Self) -> (X) -> Y) -> (X) -> Y
}
Detailed design

There's really not much more to say here. The code for typing self with a dynamic Self is in place already, however enabling this change might expose some new bugs we have not yet encountered, because currently, methods with dynamic Self return type are relatively rare.

Impact on existing code

This will have a small impact on existing code that uses a pattern similar to the above.

Alternatives considered

One alternative is to simply do nothing, but this makes the language less consistent than it could be.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Andrew Trick) #12

We bail out of generic specialization, inlining, and function signature specialization when a type substitution contains dynamic self. (hasDynamicSelfTypes). So, yes we currently almost entirely punt on optimization for methods that return Self.

I don’t have an interesting case to point out. You can look into any trivial example:

func foo<T>(_: T) {}

func method() {
  foo(self)
}

-Andy

···

On Jun 23, 2016, at 1:48 PM, Slava Pestov <spestov@apple.com> wrote:

On Jun 23, 2016, at 1:46 PM, Andrew Trick <atrick@apple.com <mailto:atrick@apple.com>> wrote:

On Jun 23, 2016, at 12:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

We’re currently specializing functions that take `self` as an argument. I don’t think that will be possible after your proposed change.

- Andy

I’m not sure what that means. Do you currently punt on certain optimizations if a method returns ‘Self’?

It should be possible to keep the reified type information around, by passing in a metatype or something for example. Can you give a concrete code snippet demonstrating the optimization and how this change would inhibit it?


(Slava Pestov) #13

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

We’re currently specializing functions that take `self` as an argument. I don’t think that will be possible after your proposed change.

- Andy

I’m not sure what that means. Do you currently punt on certain optimizations if a method returns ‘Self’?

It should be possible to keep the reified type information around, by passing in a metatype or something for example. Can you give a concrete code snippet demonstrating the optimization and how this change would inhibit it?

We bail out of generic specialization, inlining, and function signature specialization when a type substitution contains dynamic self. (hasDynamicSelfTypes). So, yes we currently almost entirely punt on optimization for methods that return Self.

I see. That makes sense.

I think the problem is that if we specialize a top-level function with a substitution involving Self, we have no way to recover what the ‘Self’ type actually is in IRGen. However I think it could be made to work by passing in a metatype for Self, and somehow ensuring we don’t mix up Self from two different contexts...

This is certainly a trickier change than I first imagined, but it would be nice to figure out how to solve this in a principled way so that we can get these optimizations to be more generally applicable. It seems even more surprising, now, if changing the return type of a method inhibits optimizations in a non-obvious way, especially ones that can have a drastic effect on performance.

I think next week I’ll try implementing this proposal behind a staging flag, and play around with the optimizer to see how hard it would be plumb through the relevant type information.

Slava

···

On Jun 23, 2016, at 2:02 PM, Andrew Trick <atrick@apple.com> wrote:

On Jun 23, 2016, at 1:48 PM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:

On Jun 23, 2016, at 1:46 PM, Andrew Trick <atrick@apple.com <mailto:atrick@apple.com>> wrote:

On Jun 23, 2016, at 12:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I don’t have an interesting case to point out. You can look into any trivial example:

func foo<T>(_: T) {}

func method() {
  foo(self)
}

-Andy


(L Mihalkovic) #14

Quick (semi) related question: any particular reason for .Type to be a contextual kwd rather than defined on a protocol? (No concrete def for metatypes?)
Regards
LM
(From mobile)

···

On Jun 23, 2016, at 11:02 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 23, 2016, at 1:48 PM, Slava Pestov <spestov@apple.com> wrote:

On Jun 23, 2016, at 1:46 PM, Andrew Trick <atrick@apple.com> wrote:

On Jun 23, 2016, at 12:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org> wrote:

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

We’re currently specializing functions that take `self` as an argument. I don’t think that will be possible after your proposed change.

- Andy

I’m not sure what that means. Do you currently punt on certain optimizations if a method returns ‘Self’?

It should be possible to keep the reified type information around, by passing in a metatype or something for example. Can you give a concrete code snippet demonstrating the optimization and how this change would inhibit it?

We bail out of generic specialization, inlining, and function signature specialization when a type substitution contains dynamic self. (hasDynamicSelfTypes). So, yes we currently almost entirely punt on optimization for methods that return Self.

I don’t have an interesting case to point out. You can look into any trivial example:

func foo<T>(_: T) {}

func method() {
  foo(self)
}

-Andy

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Andrew Trick) #15

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

We’re currently specializing functions that take `self` as an argument. I don’t think that will be possible after your proposed change.

- Andy

I’m not sure what that means. Do you currently punt on certain optimizations if a method returns ‘Self’?

It should be possible to keep the reified type information around, by passing in a metatype or something for example. Can you give a concrete code snippet demonstrating the optimization and how this change would inhibit it?

We bail out of generic specialization, inlining, and function signature specialization when a type substitution contains dynamic self. (hasDynamicSelfTypes). So, yes we currently almost entirely punt on optimization for methods that return Self.

I see. That makes sense.

I think the problem is that if we specialize a top-level function with a substitution involving Self, we have no way to recover what the ‘Self’ type actually is in IRGen. However I think it could be made to work by passing in a metatype for Self, and somehow ensuring we don’t mix up Self from two different contexts...

This is certainly a trickier change than I first imagined, but it would be nice to figure out how to solve this in a principled way so that we can get these optimizations to be more generally applicable. It seems even more surprising, now, if changing the return type of a method inhibits optimizations in a non-obvious way, especially ones that can have a drastic effect on performance.

I think next week I’ll try implementing this proposal behind a staging flag, and play around with the optimizer to see how hard it would be plumb through the relevant type information.

Slava

Perfect. I didn’t mean to discourage you, just warn you that some benchmarking and optimizer work is also needed.

-Andy

···

On Jun 23, 2016, at 2:06 PM, Slava Pestov <spestov@apple.com> wrote:

On Jun 23, 2016, at 2:02 PM, Andrew Trick <atrick@apple.com <mailto:atrick@apple.com>> wrote:

On Jun 23, 2016, at 1:48 PM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:

On Jun 23, 2016, at 1:46 PM, Andrew Trick <atrick@apple.com <mailto:atrick@apple.com>> wrote:

On Jun 23, 2016, at 12:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I don’t have an interesting case to point out. You can look into any trivial example:

func foo<T>(_: T) {}

func method() {
  foo(self)
}

-Andy


(Slava Pestov) #16

The proposal is to change the type of self to always be Self, which can be thought of as a special generic type parameter bound to the dynamic type of the instance.

We’re currently specializing functions that take `self` as an argument. I don’t think that will be possible after your proposed change.

- Andy

I’m not sure what that means. Do you currently punt on certain optimizations if a method returns ‘Self’?

It should be possible to keep the reified type information around, by passing in a metatype or something for example. Can you give a concrete code snippet demonstrating the optimization and how this change would inhibit it?

We bail out of generic specialization, inlining, and function signature specialization when a type substitution contains dynamic self. (hasDynamicSelfTypes). So, yes we currently almost entirely punt on optimization for methods that return Self.

I see. That makes sense.

I think the problem is that if we specialize a top-level function with a substitution involving Self, we have no way to recover what the ‘Self’ type actually is in IRGen. However I think it could be made to work by passing in a metatype for Self, and somehow ensuring we don’t mix up Self from two different contexts...

This is certainly a trickier change than I first imagined, but it would be nice to figure out how to solve this in a principled way so that we can get these optimizations to be more generally applicable. It seems even more surprising, now, if changing the return type of a method inhibits optimizations in a non-obvious way, especially ones that can have a drastic effect on performance.

I think next week I’ll try implementing this proposal behind a staging flag, and play around with the optimizer to see how hard it would be plumb through the relevant type information.

Slava

Perfect. I didn’t mean to discourage you, just warn you that some benchmarking and optimizer work is also needed.

No worries :slight_smile: Dynamic ‘Self’ is one of the dark corners of Swift I don’t fully understand — so the proposal is as much about cleaning it up as clarifying my own understanding of the issues.

···

On Jun 23, 2016, at 2:08 PM, Andrew Trick <atrick@apple.com> wrote:

On Jun 23, 2016, at 2:06 PM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:

On Jun 23, 2016, at 2:02 PM, Andrew Trick <atrick@apple.com <mailto:atrick@apple.com>> wrote:

On Jun 23, 2016, at 1:48 PM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:

On Jun 23, 2016, at 1:46 PM, Andrew Trick <atrick@apple.com <mailto:atrick@apple.com>> wrote:

On Jun 23, 2016, at 12:53 PM, Slava Pestov via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

-Andy

I don’t have an interesting case to point out. You can look into any trivial example:

func foo<T>(_: T) {}

func method() {
  foo(self)
}

-Andy