[Pitch] Option parameters (potential resilience advantages)


(Matthew Johnson) #1

I've had an idea floating around in my mind for a couple weeks now and would like to find out what others think about it.

The basic idea is really simple. It introduces an `@option` parameter attribute. Parameters with this attribute would behave like parameters with a default value at the call site, however they would not actually have a default value. Instead, the argument seen in the body would be an optional of the type of the parameter value.

func foo(@option value: Int) {
  // value has type Int? in the body of the function
  let valueToUse = value ?? 42
}

At first glance it probably seems like this is of no value. Why not just declare `value: Int? = nil`. Obviously it must bring something to the table to be worth considering.

This idea first occurred to me while thinking about flexible memberwise initialization. One of the use cases I have had in mind while writing that proposal is a theoretical Swift version of UIKit that exposes all of its appearance properties as memberwise initialization parameters with defaults rather than just initializing the members to a default value. Obviously this could lead to a very large stack frame that isn't really necessary. Using the `@option` approach we can avoid the unnecessarily large stack frame.

Instead of always passing a value for every parameter the compiler would use a single dynamic data structure containing the values of all `@option` arguments provided at the call site. The layout of the data structure would be an implementation detail of the language and would be part of the ABI. It would contain a header of some kind indicating which parameters were actually provided with arguments at the call site and any additional information necessary to recover the value. In the body of the function, references to the parameter value would be replaced with a lookup into the data structure which returned an optional value of the type of the parameter.

Perhaps the most interesting thing about `@option` is that careful design of the implementation might allow for `@option` parameters to be added or removed from a function without breaking ABI compatibility:

1. When new parameters are added, existing callers will simply never provide arguments and the implementation will see a nil optional value.
2. When parameters are removed, existing clients will continue to provide them and the data structure will still be populated with them. The implmentation will simply never look for the value. The function will behave as if the parameter was still specified in the signature and simply not used. The client would receive a compiler warning when compiling agains the newer SDK but everything would continue to work without issue until they did so.

If this is possible `@option` parameters would enable more robust evolution for functions containing parameters that may be omitted or have default values.

Another benefit of `@option` is that it provides better encapsulation of default values because they do not appear in function and method signatures. My understanding of the planned mechanism for default arguments is that any changes to default parameter values will not be visible to clients until they recompile against the new version of the library. If this is the case `@option` would also provide additional control to library authors who may wish to modify the default behavior of their implementation without breaking ABI.

Lastly, but potentially useful benefit of `@option` is that default argument values do not allow a function to distinguish between the case when a caller explicitly provides a value that happens to be the same as the default and the case when no argument was provided. `@option` provides this information to the body of the function.

Finally, an interesting observation: changing a parameter from having a default value to `@option` or vice versa would break ABI but would be source-compatible.

I am very interested to hear whether others think this idea is worth pursuing further or not.

Matthew


(Félix Cloutier) #2

The current workaround is to add an overload that calls the "main" version:

class Foo {
  // version 1
  func foo() {
    print(5)
  }
  
  // version 2
  func foo() {
    foo(5)
  }
  
  func foo(value: Int) {
    print(value)
  }
}

How would you say that your solution compares?

Félix

···

Le 28 déc. 2015 à 11:11:57, Matthew Johnson via swift-evolution <swift-evolution@swift.org> a écrit :

I've had an idea floating around in my mind for a couple weeks now and would like to find out what others think about it.

The basic idea is really simple. It introduces an `@option` parameter attribute. Parameters with this attribute would behave like parameters with a default value at the call site, however they would not actually have a default value. Instead, the argument seen in the body would be an optional of the type of the parameter value.

func foo(@option value: Int) {
  // value has type Int? in the body of the function
  let valueToUse = value ?? 42
}

At first glance it probably seems like this is of no value. Why not just declare `value: Int? = nil`. Obviously it must bring something to the table to be worth considering.

This idea first occurred to me while thinking about flexible memberwise initialization. One of the use cases I have had in mind while writing that proposal is a theoretical Swift version of UIKit that exposes all of its appearance properties as memberwise initialization parameters with defaults rather than just initializing the members to a default value. Obviously this could lead to a very large stack frame that isn't really necessary. Using the `@option` approach we can avoid the unnecessarily large stack frame.

Instead of always passing a value for every parameter the compiler would use a single dynamic data structure containing the values of all `@option` arguments provided at the call site. The layout of the data structure would be an implementation detail of the language and would be part of the ABI. It would contain a header of some kind indicating which parameters were actually provided with arguments at the call site and any additional information necessary to recover the value. In the body of the function, references to the parameter value would be replaced with a lookup into the data structure which returned an optional value of the type of the parameter.

Perhaps the most interesting thing about `@option` is that careful design of the implementation might allow for `@option` parameters to be added or removed from a function without breaking ABI compatibility:

1. When new parameters are added, existing callers will simply never provide arguments and the implementation will see a nil optional value.
2. When parameters are removed, existing clients will continue to provide them and the data structure will still be populated with them. The implmentation will simply never look for the value. The function will behave as if the parameter was still specified in the signature and simply not used. The client would receive a compiler warning when compiling agains the newer SDK but everything would continue to work without issue until they did so.

If this is possible `@option` parameters would enable more robust evolution for functions containing parameters that may be omitted or have default values.

Another benefit of `@option` is that it provides better encapsulation of default values because they do not appear in function and method signatures. My understanding of the planned mechanism for default arguments is that any changes to default parameter values will not be visible to clients until they recompile against the new version of the library. If this is the case `@option` would also provide additional control to library authors who may wish to modify the default behavior of their implementation without breaking ABI.

Lastly, but potentially useful benefit of `@option` is that default argument values do not allow a function to distinguish between the case when a caller explicitly provides a value that happens to be the same as the default and the case when no argument was provided. `@option` provides this information to the body of the function.

Finally, an interesting observation: changing a parameter from having a default value to `@option` or vice versa would break ABI but would be source-compatible.

I am very interested to hear whether others think this idea is worth pursuing further or not.

Matthew
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


#3

I'm not sure I understand the use case. Aren't these optimizations that could be better handled by the compiler? Do we really want to provide hints like these manually in our own libraries? Instead of `value: Int? = nil`, why not `value: Int = 42`?

Stephen

···

On Dec 28, 2015, at 11:11 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

I've had an idea floating around in my mind for a couple weeks now and would like to find out what others think about it.

The basic idea is really simple. It introduces an `@option` parameter attribute. Parameters with this attribute would behave like parameters with a default value at the call site, however they would not actually have a default value. Instead, the argument seen in the body would be an optional of the type of the parameter value.

func foo(@option value: Int) {
  // value has type Int? in the body of the function
  let valueToUse = value ?? 42
}

At first glance it probably seems like this is of no value. Why not just declare `value: Int? = nil`. Obviously it must bring something to the table to be worth considering.

This idea first occurred to me while thinking about flexible memberwise initialization. One of the use cases I have had in mind while writing that proposal is a theoretical Swift version of UIKit that exposes all of its appearance properties as memberwise initialization parameters with defaults rather than just initializing the members to a default value. Obviously this could lead to a very large stack frame that isn't really necessary. Using the `@option` approach we can avoid the unnecessarily large stack frame.

Instead of always passing a value for every parameter the compiler would use a single dynamic data structure containing the values of all `@option` arguments provided at the call site. The layout of the data structure would be an implementation detail of the language and would be part of the ABI. It would contain a header of some kind indicating which parameters were actually provided with arguments at the call site and any additional information necessary to recover the value. In the body of the function, references to the parameter value would be replaced with a lookup into the data structure which returned an optional value of the type of the parameter.

Perhaps the most interesting thing about `@option` is that careful design of the implementation might allow for `@option` parameters to be added or removed from a function without breaking ABI compatibility:

1. When new parameters are added, existing callers will simply never provide arguments and the implementation will see a nil optional value.
2. When parameters are removed, existing clients will continue to provide them and the data structure will still be populated with them. The implmentation will simply never look for the value. The function will behave as if the parameter was still specified in the signature and simply not used. The client would receive a compiler warning when compiling agains the newer SDK but everything would continue to work without issue until they did so.

If this is possible `@option` parameters would enable more robust evolution for functions containing parameters that may be omitted or have default values.

Another benefit of `@option` is that it provides better encapsulation of default values because they do not appear in function and method signatures. My understanding of the planned mechanism for default arguments is that any changes to default parameter values will not be visible to clients until they recompile against the new version of the library. If this is the case `@option` would also provide additional control to library authors who may wish to modify the default behavior of their implementation without breaking ABI.

Lastly, but potentially useful benefit of `@option` is that default argument values do not allow a function to distinguish between the case when a caller explicitly provides a value that happens to be the same as the default and the case when no argument was provided. `@option` provides this information to the body of the function.

Finally, an interesting observation: changing a parameter from having a default value to `@option` or vice versa would break ABI but would be source-compatible.

I am very interested to hear whether others think this idea is worth pursuing further or not.

Matthew
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Joe Groff) #4

You can provide resilience with a non-optional parameter by making the default argument the result of calling a resilient function (or evaluating a resilient property):

@availability(x.y)
internal func defaultForFoo() -> Int { return 941 }

public func foo(value: Int = defaultForFoo()) { }

-Joe

···

On Dec 28, 2015, at 11:46 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 28, 2015, at 1:39 PM, Stephen Celis <stephen.celis@gmail.com> wrote:

I'm not sure I understand the use case. Aren't these optimizations that could be better handled by the compiler? Do we really want to provide hints like these manually in our own libraries? Instead of `value: Int? = nil`, why not `value: Int = 42`?

I agree that part of this is simply an optimization. The part that was interesting enough that I thought it is worth sharing is that it could improve resilience in a way that a default value does not allow.

That said, it is not a “proposal”. I’m not sure whether it is really worth considering or not. But I think it is interesting enough to toss out to the community and see what the response is.


(Joe Groff) #5

Yeah, you ought to be able to do this by deprecating the old entry point and implementing it by forwarding to the new more general entry point.

-Joe

···

On Dec 28, 2015, at 12:29 PM, Matthew Johnson <matthew@anandabits.com> wrote:

Sent from my iPhone

On Dec 28, 2015, at 2:04 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Dec 28, 2015, at 11:46 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 28, 2015, at 1:39 PM, Stephen Celis <stephen.celis@gmail.com <mailto:stephen.celis@gmail.com>> wrote:

I'm not sure I understand the use case. Aren't these optimizations that could be better handled by the compiler? Do we really want to provide hints like these manually in our own libraries? Instead of `value: Int? = nil`, why not `value: Int = 42`?

I agree that part of this is simply an optimization. The part that was interesting enough that I thought it is worth sharing is that it could improve resilience in a way that a default value does not allow.

That said, it is not a “proposal”. I’m not sure whether it is really worth considering or not. But I think it is interesting enough to toss out to the community and see what the response is.

You can provide resilience with a non-optional parameter by making the default argument the result of calling a resilient function (or evaluating a resilient property):

@availability(x.y)
internal func defaultForFoo() -> Int { return 941 }

public func foo(value: Int = defaultForFoo()) { }

I don't think I've tried calling a lower visibility function to get a default like that before. That's interesting. Good to know it is possible.

I was especially thinking about the ability to add or remove option parameters without breaking ABI. In that case the existing function might be able to forward to a new overload with a different set of parameters with defaults, but it also seems like that would lead to ambiguity at call sites in many cases when compiling against the new binary.

Will there be a way to "hide" the old function (i.e. give it in some kind of "public for backwards compatibility only" visibility) so call sites are still unambiguous during compilation and the module interface is not unnecessarily cluttered while also retaining the original function for ABI compatibility? (I hope that long sentence is comprehensible)


(Matthew Johnson) #6

I'm not sure I understand the use case. Aren't these optimizations that could be better handled by the compiler? Do we really want to provide hints like these manually in our own libraries? Instead of `value: Int? = nil`, why not `value: Int = 42`?

I agree that part of this is simply an optimization. The part that was interesting enough that I thought it is worth sharing is that it could improve resilience in a way that a default value does not allow.

That said, it is not a “proposal”. I’m not sure whether it is really worth considering or not. But I think it is interesting enough to toss out to the community and see what the response is.

···

On Dec 28, 2015, at 1:39 PM, Stephen Celis <stephen.celis@gmail.com> wrote:

Stephen

On Dec 28, 2015, at 11:11 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

I've had an idea floating around in my mind for a couple weeks now and would like to find out what others think about it.

The basic idea is really simple. It introduces an `@option` parameter attribute. Parameters with this attribute would behave like parameters with a default value at the call site, however they would not actually have a default value. Instead, the argument seen in the body would be an optional of the type of the parameter value.

func foo(@option value: Int) {
  // value has type Int? in the body of the function
  let valueToUse = value ?? 42
}

At first glance it probably seems like this is of no value. Why not just declare `value: Int? = nil`. Obviously it must bring something to the table to be worth considering.

This idea first occurred to me while thinking about flexible memberwise initialization. One of the use cases I have had in mind while writing that proposal is a theoretical Swift version of UIKit that exposes all of its appearance properties as memberwise initialization parameters with defaults rather than just initializing the members to a default value. Obviously this could lead to a very large stack frame that isn't really necessary. Using the `@option` approach we can avoid the unnecessarily large stack frame.

Instead of always passing a value for every parameter the compiler would use a single dynamic data structure containing the values of all `@option` arguments provided at the call site. The layout of the data structure would be an implementation detail of the language and would be part of the ABI. It would contain a header of some kind indicating which parameters were actually provided with arguments at the call site and any additional information necessary to recover the value. In the body of the function, references to the parameter value would be replaced with a lookup into the data structure which returned an optional value of the type of the parameter.

Perhaps the most interesting thing about `@option` is that careful design of the implementation might allow for `@option` parameters to be added or removed from a function without breaking ABI compatibility:

1. When new parameters are added, existing callers will simply never provide arguments and the implementation will see a nil optional value.
2. When parameters are removed, existing clients will continue to provide them and the data structure will still be populated with them. The implmentation will simply never look for the value. The function will behave as if the parameter was still specified in the signature and simply not used. The client would receive a compiler warning when compiling agains the newer SDK but everything would continue to work without issue until they did so.

If this is possible `@option` parameters would enable more robust evolution for functions containing parameters that may be omitted or have default values.

Another benefit of `@option` is that it provides better encapsulation of default values because they do not appear in function and method signatures. My understanding of the planned mechanism for default arguments is that any changes to default parameter values will not be visible to clients until they recompile against the new version of the library. If this is the case `@option` would also provide additional control to library authors who may wish to modify the default behavior of their implementation without breaking ABI.

Lastly, but potentially useful benefit of `@option` is that default argument values do not allow a function to distinguish between the case when a caller explicitly provides a value that happens to be the same as the default and the case when no argument was provided. `@option` provides this information to the body of the function.

Finally, an interesting observation: changing a parameter from having a default value to `@option` or vice versa would break ABI but would be source-compatible.

I am very interested to hear whether others think this idea is worth pursuing further or not.

Matthew
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #7

The current workaround is to add an overload that calls the "main" version:

class Foo {
  // version 1
  func foo() {
    print(5)
  }
  
  // version 2
  func foo() {
    foo(5)
  }
  
  func foo(value: Int) {
    print(value)
  }
}

How would you say that your solution compares?

I don’t quite understand how this is a “workaround”. I don’t believe it would have the potential resilience benefits that prompted me to share the idea.

It also doesn’t cover a case where there might be several option parameters and you want to allow the user to specify any combination of them. Parameters with default values do cover that case but would also be less resilient than the “option” parameter idea.

As I stated a couple of times now, I am not “proposing” this idea, simply sharing it to see what the community response is.

···

On Dec 28, 2015, at 1:08 PM, Félix Cloutier <felixcca@yahoo.ca> wrote:

Félix

Le 28 déc. 2015 à 11:11:57, Matthew Johnson via swift-evolution <swift-evolution@swift.org> a écrit :

I've had an idea floating around in my mind for a couple weeks now and would like to find out what others think about it.

The basic idea is really simple. It introduces an `@option` parameter attribute. Parameters with this attribute would behave like parameters with a default value at the call site, however they would not actually have a default value. Instead, the argument seen in the body would be an optional of the type of the parameter value.

func foo(@option value: Int) {
  // value has type Int? in the body of the function
  let valueToUse = value ?? 42
}

At first glance it probably seems like this is of no value. Why not just declare `value: Int? = nil`. Obviously it must bring something to the table to be worth considering.

This idea first occurred to me while thinking about flexible memberwise initialization. One of the use cases I have had in mind while writing that proposal is a theoretical Swift version of UIKit that exposes all of its appearance properties as memberwise initialization parameters with defaults rather than just initializing the members to a default value. Obviously this could lead to a very large stack frame that isn't really necessary. Using the `@option` approach we can avoid the unnecessarily large stack frame.

Instead of always passing a value for every parameter the compiler would use a single dynamic data structure containing the values of all `@option` arguments provided at the call site. The layout of the data structure would be an implementation detail of the language and would be part of the ABI. It would contain a header of some kind indicating which parameters were actually provided with arguments at the call site and any additional information necessary to recover the value. In the body of the function, references to the parameter value would be replaced with a lookup into the data structure which returned an optional value of the type of the parameter.

Perhaps the most interesting thing about `@option` is that careful design of the implementation might allow for `@option` parameters to be added or removed from a function without breaking ABI compatibility:

1. When new parameters are added, existing callers will simply never provide arguments and the implementation will see a nil optional value.
2. When parameters are removed, existing clients will continue to provide them and the data structure will still be populated with them. The implmentation will simply never look for the value. The function will behave as if the parameter was still specified in the signature and simply not used. The client would receive a compiler warning when compiling agains the newer SDK but everything would continue to work without issue until they did so.

If this is possible `@option` parameters would enable more robust evolution for functions containing parameters that may be omitted or have default values.

Another benefit of `@option` is that it provides better encapsulation of default values because they do not appear in function and method signatures. My understanding of the planned mechanism for default arguments is that any changes to default parameter values will not be visible to clients until they recompile against the new version of the library. If this is the case `@option` would also provide additional control to library authors who may wish to modify the default behavior of their implementation without breaking ABI.

Lastly, but potentially useful benefit of `@option` is that default argument values do not allow a function to distinguish between the case when a caller explicitly provides a value that happens to be the same as the default and the case when no argument was provided. `@option` provides this information to the body of the function.

Finally, an interesting observation: changing a parameter from having a default value to `@option` or vice versa would break ABI but would be source-compatible.

I am very interested to hear whether others think this idea is worth pursuing further or not.

Matthew
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #8

I'm not sure I understand the use case. Aren't these optimizations that could be better handled by the compiler? Do we really want to provide hints like these manually in our own libraries? Instead of `value: Int? = nil`, why not `value: Int = 42`?

I agree that part of this is simply an optimization. The part that was interesting enough that I thought it is worth sharing is that it could improve resilience in a way that a default value does not allow.

That said, it is not a “proposal”. I’m not sure whether it is really worth considering or not. But I think it is interesting enough to toss out to the community and see what the response is.

You can provide resilience with a non-optional parameter by making the default argument the result of calling a resilient function (or evaluating a resilient property):

@availability(x.y)
internal func defaultForFoo() -> Int { return 941 }

public func foo(value: Int = defaultForFoo()) { }

I don't think I've tried calling a lower visibility function to get a default like that before. That's interesting. Good to know it is possible.

I was especially thinking about the ability to add or remove option parameters without breaking ABI. In that case the existing function might be able to forward to a new overload with a different set of parameters with defaults, but it also seems like that would lead to ambiguity at call sites in many cases when compiling against the new binary.

Will there be a way to "hide" the old function (i.e. give it in some kind of "public for backwards compatibility only" visibility) so call sites are still unambiguous during compilation and the module interface is not unnecessarily cluttered while also retaining the original function for ABI compatibility? (I hope that long sentence is comprehensible)

If it is possible to effectively add and remove parameters with default values one way or another without breaking ABI then the option parameter idea is almost certainly not worth the complexity.

···

Sent from my iPhone

On Dec 28, 2015, at 2:04 PM, Joe Groff <jgroff@apple.com> wrote:

On Dec 28, 2015, at 11:46 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:
On Dec 28, 2015, at 1:39 PM, Stephen Celis <stephen.celis@gmail.com> wrote:


(Matthew Johnson) #9

Sent from my iPhone

I'm not sure I understand the use case. Aren't these optimizations that could be better handled by the compiler? Do we really want to provide hints like these manually in our own libraries? Instead of `value: Int? = nil`, why not `value: Int = 42`?

I agree that part of this is simply an optimization. The part that was interesting enough that I thought it is worth sharing is that it could improve resilience in a way that a default value does not allow.

That said, it is not a “proposal”. I’m not sure whether it is really worth considering or not. But I think it is interesting enough to toss out to the community and see what the response is.

You can provide resilience with a non-optional parameter by making the default argument the result of calling a resilient function (or evaluating a resilient property):

@availability(x.y)
internal func defaultForFoo() -> Int { return 941 }

public func foo(value: Int = defaultForFoo()) { }

I don't think I've tried calling a lower visibility function to get a default like that before. That's interesting. Good to know it is possible.

I was especially thinking about the ability to add or remove option parameters without breaking ABI. In that case the existing function might be able to forward to a new overload with a different set of parameters with defaults, but it also seems like that would lead to ambiguity at call sites in many cases when compiling against the new binary.

Will there be a way to "hide" the old function (i.e. give it in some kind of "public for backwards compatibility only" visibility) so call sites are still unambiguous during compilation and the module interface is not unnecessarily cluttered while also retaining the original function for ABI compatibility? (I hope that long sentence is comprehensible)

Yeah, you ought to be able to do this by deprecating the old entry point and implementing it by forwarding to the new more general entry point.

Cool. The option parameter idea seems to have no value to us then. I suspected as much but wasn't sure so thought it was worth seeing what the response was. Thanks for taking time to respond.

···

Sent from my iPhone

On Dec 28, 2015, at 2:52 PM, Joe Groff <jgroff@apple.com> wrote:

On Dec 28, 2015, at 12:29 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Dec 28, 2015, at 2:04 PM, Joe Groff <jgroff@apple.com> wrote:

On Dec 28, 2015, at 11:46 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:
On Dec 28, 2015, at 1:39 PM, Stephen Celis <stephen.celis@gmail.com> wrote: