Prepitch: function wrappers

Similar discussions existed before, but they were made either long before any functionality this idea is based on was introduced to Swift and so they died out. So here goes nothing - let's talk about Python decorators again =)

Decorators in Python are implemented as functions wrapping other functions + some syntactic sugar that looks similar to Swift property wrappers and other attributes. For example in Flask one can implement a following decorator:

from functools import wraps
from flask import g, request, redirect, url_for

def login_required(f):
    @wraps(f)
    def decorated_function(*args, **kwargs):
        if g.user is None:
            return redirect(url_for('login', next=request.url))
        return f(*args, **kwargs)
    return decorated_function

https://flask.palletsprojects.com/en/1.1.x/patterns/viewdecorators/#login-required-decorator

This decorator allows to easily add additional behaviour to decorated functions, i.e. in this case a requirement to be logged in or to be redirected to login page when accessing a specific endpoint of server-side app:

@app.route('/secret_page')
@login_required
def secret_page():
    pass

In fact there is also another decorator here, @app.route, which allows to bind function to the endpoint, which might be as well very desirable for Swift on Server.

Thinking about how this can be implemented in Swift nowadays I came up with a following idea that is based on using dynamicCallable in conjunction with something similar to property wrappers in some way.

Considering above examples we can imagine it to be implemented in Swift like this:

@Route("/secret_page")
@LoginRequired
func secretPage() -> HTML {
  HTML(...)
}

Now to implement this LoginRequired decorator we could write something similar to property wrapper which would at the same time implement @dynamicCallable:

struct LoginRequired {
  let wrappedFunc: (KeyValuePairs<String, Any>) -> HTML
  init(wrappedFunc: @escaping (KeyValuePairs<String, Any>) -> HTML) {
    self.wrappedFunc = wrappedFunc
  }
  func dynamicallyCall(withKeywordArguments args: KeyValuePairs<String, Any>) -> HTML {
    guard Current.user != nil else {
      return Current.app.redirect(to: "login", next: Current.request.url)
    }
    return wrappedFunc(args)
  }
}

(Current here is just some global context, like global g, request in Flask)

This looks very similar to Python code for decorator with KeyValuePairs<String, Any> instead of *kwarg. This makes decorator agnostic of wrapped function parameters, their types and labels, as on practice each wrapped function will have different parameters. (As with dynamic callable KeyValueParis is not required, any array literal convertible type for parameters should work)

Python also allows to define decorators as classes which makes the code even more alike (with dynamicallyCall instead of __call__).

Now for this to really work compiler would need to rewrite original function declaration to something like this:

private var _secretPage: LoginRequired {
  // decorated function moved inside decorator property getter
  // as it is a computed property it will even have access to other members of enclosing type
  func secretPage() -> HTML {
    HTML(...)
  }
  // decorated function is passed to the wrapper via closure
  return LoginRequired(wrappedValue: { params in secretPage() })
}
    
func secretPage() -> String {
  _secretPage()
}

This looks similar to how property wrappers transform code, but a little bit more involved (still doable I think) =)

If decorated function has arguments, named or not named, this is already handled by using KeyValuePairs in dynamicallyCall so only needs some additional code to be generated.

For example if we want to decorate function like this with a URL route that accepts a parameter:

@Route("/user/:username")
@LoginRequired
func profile(username: String) -> User? {
  users.first { $0.name == username }
}

will be translated to:

private var _profile: LoginRequired {
  func profile(username: String) -> User? {
    users.first { $0.name == username }
  }
  return LoginRequired(wrappedValue: { params in profile(username: params[0] as! String) })
}
    
func profile(username: String) -> User? {
  _profile(username: username)
}

Similar to property wrappers it should be possible to pass parameters to function wrappers, but unlike property wrappers function wrappers always have wrapped value, the wrapped function itself, so the convention would be to allow any initialiser parameter and require wrapped value to be the last:

struct Route {
  let path: String
  let wrappedFunc: (KeyValuePairs<String, Any>) -> HTML
  init(_ path: String, wrappedFunc: @escaping (KeyValuePairs<String, Any>) -> HTML) {
    self.wrappedFunc = wrappedFunc
  }
  func dynamicallyCall(withKeywordArguments args: KeyValuePairs<String, Any>) -> HTML {
    guard Current.request.url.path == path else { return Current.request.next() }
    return wrappedFunc(args)
  }
}

Then the code with such wrapper

@Route("/user/:username")
func profile(username: String) -> User? {
  users.first { $0.name == username }
}

would be transformed to following:

private var _profile: Route {
  func profile(username: String) -> User? {
    users.first { $0.name == username }
  }
  // using trailing closure would make code transformation even simpler
  return Route("/user/:username") { params in profile(username: params[0] as! String) }
}
    
func profile(username: String) -> User? {
  _profile(username: username)
}

Some concerns/questions:

  1. return type of wrapped functions is fixed in a wrapped value signature
    A: wrapper can be generic and can provide dynamicallyCall implementations in constrained extensions (this is already supported by dynamicCallable)

  2. type casts and subscripts look nasty
    A: this code is generated by compiler so should be safe...

  3. does such decorator need to have projected value? underscore value to access wrapper itself? access to original function?
    A: not sure about value of projected value here but to be symmetric with property wrappers (maybe even to build on top of it as well) it might make sense to provide the same functionality. Access to original function (not something that is available for property wrappers out of the box, but doable on demand) might be also useful in some edge cases

  4. static callable proposal mentions unifying types as the end goal, can that affect this feature and design and then maybe it's better to wait until this work will be done?
    A: don't know really, wrappers don't have to be structs here so if functions will be able to hold members in future then wrapper would stay a function, then there will be no need for dynamicCallable as a sugar, but still ther will be need for some convention I suppose, and for some code generation as well

  5. does it have to be dynamicCallable?
    A: This way feature will be based on top of existing mechanics. dynamicCallable is a syntactic sugar itself, so it can be desugared in generated code automatically, but wrapper would still need to implement dynamicallyCall methods per convention. But if it's better to come up with a different mechanics it's acceptable. Or maybe I'm not seeing something simpler that does not need any "magic"?

  6. Why not just to wrap function inside it's body?
    A: Surely one can wrap the body of the function in described decorators using higher order function, but this can result in increased nesting, destructs from the actual purpose of the function and complicate editing. In the end, property wrappers were also possible to implement almost in the same way as they work now even before they were introduced as a syntactic sugar =)

That's basically it. I'm not sure about this idea and such design so I'm open to any feedback. In my opinion it worths - Python decorators was one of the nicest things I learned about Python and I heard similar opinions from others - though to be faire I didn't work a lot with Python and when I did I only used decorators in the described context (would love to do that same with Swift on Server!)
Also implementation details seem to align with what was done in property wrappers proposal. So based on existing precedent of property wrappers why not to extend it to functions as well?

UPD:

  1. Wrappers composition
    A: This was eventually solved for property wrappers, though I didn't follow the details. I imagine that compiler can desugar wrappers one by one nesting generated code, eventually transforming code with two wrappers from above examples to something like this:
private var _profile: Route {
  private var _profile: LoginRequired {
    func profile(username: String) -> User? {
      users.first { $0.name == username }
    }
    return LoginRequired(wrappedValue: { params in profile(username: params[0] as! String) })
  }
  
  func profile(username: String) -> User? {
    _profile(username: username)
  }

  // using trailing closure would make code transformation even simpler
  return Route("/user/:username") { params in profile(username: params[0] as! String) }
}
    
func profile(username: String) -> User? {
  _profile(username: username)
}
17 Likes

Technically property wrappers can already do that, but in a limited way as we still don‘t support compound names. I quickly scanned the pitch so please don‘t judge me too hard if I missed something. This design is overall complex and it looses a lot of type safety unlike we‘re now used to from property wrappers. Personally I don‘t like KeyValuePairs type like presented in the pitch. You also haven‘t mentioned anything about composition of these wrapper types, yet you used it everywhere.

If we had compound names then this would be already possible.

@propertyWrapper
struct W {
  var wrappedValue: (A, B) -> Void
}

@W { a, b in ... }
var foo(a:b:): (A, B) -> Void

// usage
foo(a: someA, b: someB)

I don‘t know a reason why we couldn‘t teach the compiler to also allow wrapping functions, however I would be careful here as function builders already used in this place.

@W { a, b in ... }
func foo(a: A, b: B)

There are still things that needs to be resolved first before we can push property wrappers to their limits:

  • generalized coroutines
  • variadic generics
  • compound names + sugar
  • static/shared property wrappers
  • access to enclosing self (would be much better with generalized coroutines)
  • generalized super type constraint
  • function sub-type / constraints
2 Likes

I'm not sure it is really achievable with property wrappers as they are right now, though I think this feature can be built on top of property wrappers, at least based on the same techniques.
So maybe with property wrapper generated code can be simplified to this:

    @Bar
    var _foo: (Foo, String) -> String = { `self`, a in
        "Foo \(a)"
    }
    
    func foo(a: String) -> String {
        _foo(self, a)
    }

And this property wrapper would need to be implemented like this:

@propertyWrapper
struct Bar {
    let wrappedValue: (Foo, String) -> String
    
    init(wrappedValue: @escaping (Foo, String) -> String) {
        self.wrappedValue = { foo, a in
            "Bar " + wrappedValue(foo, a)
        }
    }
}

I guess this would work, but it makes property wrapper too static, it can only be applied on the functions of specific type and belonging to specific instances (self accepted as a first parameter similar to implicit self in methods so that the closure can access enclosing instance in the same way as function). Type of enclosed instance can be generic, but types and labels of parameters would need to be "erased" anyway to make the wrapper universal, and then it becomes even worse than dynamicCallable that solves this issue nicely.

    @Bar
    var _foo: (Foo,/* what to put here to hold parameters?*/) -> String = { `self` /* how this will be passed in?*/ , params in
        "Foo \(/* how to extract parameter that is used here?*/)" /* possible calls to other properties of self would need to have `self.` added to them */
    }

So it seems to me building on property wrappers would complicate things, not sure it is even possible.

Regarding loosing type safety - it's a tradeoff. In provided examples (and I suppose in general) function wrapper would only care about passing parameters to the wrapped function after performing its additional work, they won't care about actual values. Using dynamicCallable for this allows wrapper to be applied on any function regardless its parameters list.
Generated code in this case is based on the code that user writes and it changes with it, preserving type safety in the places where it needs to cast parameters (still if someone can come up with a wait to avoid that preserving dynamism - would be great!)
If type safety is needed though I think callAsFunction may be used for that? It's already working on par with dynamicCallable so I imagine this can be leveraged here as well somehow. But this will again limit wrapper application I suppose. This can be desirable though if for example wrapper requires function to have particular signature because it needs to access its parameters. Then it becomes similar to protocol requirements. I'd say we can have both in the end! =)

Interesting, can you elaborate on what you had in mind for that?

But that's the whole beauty of this feature as it allows us to create statically enforced templates that are well understood and enforced by the compiler to be used correctly, going into the dynamic direction is a no go for me similar to how I dislike the dynamic member lookup which does not use key-paths. This is of course my personal preference and I respect that the dynamic part of the mentioned feature is needed for things like python interop which makes it a pleasure for other to use.

However for property wrappers to support functions in a basic form we don't need that many things (unless you need self from the enclosing host type). 'Compound names' and variadic generics would allow us to create wrappers for 'pure-ish' swift functions. Variadic generics is the keyword here to better support a dynamic yet type-safe list of function parameters.

Support for accessing outer self is already there, but it's experimental and the design of it is clunky. There are much better solutions that we could use in the long term than any of the current rushed approaches. For example instead of static subscripts we could use 'generalized coroutines' so that we could extend 'static/shared' property wrappers with methods like:

static func read(from: Instance) -> shared Value
static func modify(in: inout Instance) -> inout Value

Something along the lines would allow us to create a nice design for static/shared property wrappers which also covers the access to the enclosing self for both objects and value types.


For example:

@ViewBuilder
func foo() -> some View { ... }

This isn't really a property wrapper, nor a function wrapper, it's function builder, but it's used in the same spot where a potential property wrapper could be used if we'd accept such sugar syntax. That said, we should keep that in mind that function builder placement isn't going anywhere now and if we would introduce something similar, we should avoid any possible conflicts up front when we design a new feature.

2 Likes

Interesting, I missed the part where function builders can be applied to functions themselves, I thought it's only applicable to function parameters!

1 Like

There is more. You can apply them to get only properties as well and potentially to get only subscripts I believe:

@ViewBuilder
var body: some View { ... }

This is totally valid Swift code today.

2 Likes

Just to demonstrate how this would look like I've implemented proposed solution with Sourcery template in one of my libraries (branch "function-wrapper").

(If someone wants to try this it requires a custom Sourcery build from its "function-wrapper" branch to be installed in your Application folder)

Client code:

class HelloService: Service {
    //sourcery:wrap:Route: GET/.hello/.string
    func _helloworld(name: String) -> String {
        "Hello \(name)!"
    }
    
    override init() {
        super.init()
        __helloworld.registerRoute(self)
    }
}

Here //sourcery:wrap:Route is used in place of function wrapper @Route. Then __helloworld refers to the wrapper value itself, similar to how property wrappers generate underscored variable that is accessible privately. Underscore in the name of wrapped method is needed as Sourcery does not rewrite client code like compiler would.

Then generated code looks like this:

extension HelloService {
    var __helloworld: Route<HelloService, String> {
        func _helloworld(name: String) -> String {
            "Hello \(name)!"
        }
        return Route(GET/.hello/.string) { params in
            return _helloworld(name: params)
        }
    }
    func helloworld(name: String) -> String {
        __helloworld(name: name)
    }
}

Again, with compiler transforming code there will be one less underscore in the generated code.

Now wrapper code looks like this (it's not a full code and it is slightly transformed, but what's left out is not essential to the discussion and is just implementation details of this particular wrapper):

@dynamicCallable
struct Route<S: Service, T: Encodable> {
    private let route: (KeyValuePairs<String, Any>) -> T
    let registerRoute: (S) -> Void
    
    func dynamicallyCall(withKeywordArguments args: KeyValuePairs<String, Any>) -> T {
        route(args)
    }

    init<A>(_ format: URLFormat<A>, handler: @escaping (A) -> T) {
        self.route = { handler(packKeyValueParams($0)) }
        self.registerRoute = { service in
            service.routes {
                SwiftNIOMock.route(format) { (request, params, response, next) in
                    let result = handler(params)
                    try? response.sendJSON(.ok, value: result)
                    next()
                }
            }
        }
    }
}

With this it's possible to call wrapped method directly the same way as if it was not wrapped, same way as with property wrappers, just with helloworld(name: "Swift"). And it's possible to access wrapper itself via underscored variable to perform some specific actions, same as with property wrappers. Again, don't mind underscore in the original method name =)

I've also been thinking it would be useful to extend property wrappers to create function/method wrappers. Here's some additional use-cases that I've encountered that could benefit.

Logging: Sometimes you want to be able to log entry or exit to some (or all?!) methods in an application. It would be nice to leave the boilerplate out of the logged methods and simply be able to annotate them with "@Logged".

Call deferral: I've had some occasions where I wanted to defer a call until a later point in time (e.g. a network operation when the network is unavailable). If I have a large number of these methods, I end up having a common pattern where I have a "guard" block that places the call into a suspended operation queue. It would also be nice to clean up this boilerplate.

3 Likes

Very interesting to me as well.

Coming from Python, I wonder why function wrappers were not introduced in
the first place rather than the property wrappers, which are a bit more limited.

I‘d love to see this growing into a full pitch.

With function wrappers and new modern concurrency system in Swift we could replace heavyweight construction of starting some task in functions, that are aimed only for starting Task. FE instead of

func viewIsReady() {
   Task {
      let data = await networkService.fetchData()
      await view.showData(data)
   }
}

we could use function wrappers like this

@Task
func viewIsReady() {
   let data = await networkService.fetchData()
   await view.showData(data)
}

much better, isn't it?

It is even more significant with do/catch operations, that already contain another one level of indentation:

from:

func viewIsReady() {
   Task {
      do {
         let data = try await networkService.fetchData()
         await view.showData(data)
      } catch {
         print(error)
      }
   }
}

to:

@Task
func viewIsReady() {
   do {
      let data = try await networkService.fetchData()
      await view.showData(data)
   } catch {
      print(error)
   }
}

The code becomes cleaner and easy to read/maintain.

7 Likes

I have long wondered when property wrappers would be extended to function wrappers, much like what Python Decorators are. In fact, I think they would be even more versatile than property wrappers.

That said though, I would also hope that the primary wart of property wrappers (not being transparently composable) would be fixed before introducing function wrappers.

4 Likes

I guess this can now be achieved with marcos

2 Likes