Compile time support for Aspect-oriented programming

Hi everybody,

I wanted to talk about this for a while now.
With the forum, I feel more able to communicate.

This pitch is a work in progress. There is no implementation at this time.

I look forward to hearing general feedback on this feature from the community.

Thanks,

-- Jeff


TLDR

Imagine that you have to use some third-party SDK like google analytics. You would have to sprinkle analytics code everywhere in your codebase. Imagine now instead of that you only have one file where all the analytics code of your app reside. This is what this feature is about decoupling cross-cutting concern like from your business code.

Compile time support for Aspect-oriented programming

Introduction

This proposal aims at increasing code modularity by introducing some aspect-oriented programming (AOP) traits to the Swift language. AOP helps to unclutter code that otherwise tends to mix business logic with cross-cutting concerns.
Better separation could be achieved in Swift by injecting additional behaviors (code to be executed before or after some functions) at compile time into the existing source code. Thus adding code that is not central to the business logic of a program without cluttering the core code.

The main difference with AOP is that AOP is dynamic and this is static at compile time. And it will not support runtime injection like AOP because meta programming topics are out of scope.

Motivation

Traditional code tends to mix specific business code with cross-cutting concerns like profiling, instrumentation, logging, security, etc., ending up in cluttered code that is complicated to understand or refactor. Using AOP based injection, the code could better isolate those cross-cutting concerns from the business logic: the result is a better readable, maintainable code.

Most of the time the cross-cutting concerns are adding performance, error logging, managing access control or integrating third-party SDKs and you want to seek separation and independence from this code.

Avantages

The main advantages of this feature are:

  • a definite improvement for the developer is to improve focus on a single concern
  • readability and maintainability due to the separation of business logic code and cross-cutting concerns code
  • increased productivity because it's easier to work on code in one central place rather than scattered over the project
  • easily adding functionalities like instrumentation, profiling, logging, debugging, access controls, etc. without having to dive deep into the business code
  • a warranty of execution even if the augmented function has complicated flows with several exit points

Use cases:

  • add debugging information
  • add profiling information
  • add access control for increased security to several classes
  • assert pre- and post-conditions
  • add analytic metrics
  • isolate calls to third-party code in a central place
  • add generic behavior to code without modifying it like memoization

Background

This feature is heavily inspired by Aspect Oriented Programming although I don't think this is Aspect Oriented Programming per se but it carries a lot of similarities.

This feature is built to work at compile time like Codable and Equatable and Hashable conformance(SE-0185).

Separation of concerns is a big common problem in programming that Aspect Oriented Programming helps to reduce.

Other contributors already express the need for this

Other contributors have already expressed the need for this kind of feature on the mailing list Function Decorator sadly this thread doesn't get enough traction. I propose a different approach to fix the same issue. More recently Swift request list ask for Aspect Oriented Programming too.

Sourcery(a very popular metaprogramming toolkit for Swift) provide by default a template for doing it(Decorator.swifttemplate).

Other Languages Does It

Aspect-oriented programming is implemented in many languages.

Python decorator syntax PEP 318 – Decorators for Functions and Methods | peps.python.org / PythonDecorators - Python Wiki

Objective-C use of AOP

Analytics is a kind of cross-cutting concerns because an analytics strategy necessarily affects every part of the system. Analytics thereby crosscuts all classes and methods.

Proposed solution

Teach the compiler how to insert code before/after and around an already existing function.

We propose that a function can have added code to it from an injector. We describe the specific conditions under which a function is augmented below, followed by the details of how the injector is implemented.

Introducing a new expression @decorator which injects code into a function. It can take the parameters before, after and wrap.

Expected behavior

before after wrap
inject code at the beginning of the target function after the before using defer with injected code be able to manipulate the input and the output of a target function by wrapping it in another function it will replace the original function with a new one that will be called instead and call the original function

In the decorator code, you have access to self and all of the decorated function parameters, you can return or throws if you need to.

The decorator function:

  • can be used anywhere
  • can operate on any Swift function in a Class, Struct or Protocol implementation
  • can only work on code that we have the source code of so it does not work on compiled library or frameworks
  • cannot work for getter et setter on protocol
  • we can inject before,after and wrap at the same time
  • cannot work from a library

What the decorator feature needs to work:

  • a way to operate (before|after|wrap)
  • a way to hook onto another function that exists
  • a priority of execution (optional by default the highest if more than two then propose a priority)
  • [unowned self] to do stuff
  • a way to retrieve a function original parameters function-expression-parameters
  • the final function to call in wrap mode

New Compiler error/warning :

  • raise an error when there is a conflict between the prioritys and show where it has already been used
  • if the function selector does not find a function to inject into, the compiler should emit a warning
  • will emit an error if used in wrap mode and the function pass as the parameter is not used

The compiler will do most of the work so we will never have to see the generated version of the code unless we go into the pre-process representation in Xcode.

Detailed design

I am not set on the syntax yet. It's pseudo code.

Syntax 1 - More :snake: like

Pros against syntax 2 :

  • the priority of calls is handled by the call order
  • simple
  • we can see where we inject code

@decorator(before|after|wrap, Class.AllFunction|Struct.AllFunction|Protocol.AllFunctionImplementation) 

/// Example

/// FileA.swift
func decoratorBefore(x : Int) {
    print("before")
}

func decoratorAfter(x : Int) {
    print("after")
}

func decoratorWrap(x : Int, f : (x : Int) -> Bool) -> Bool {
    print("wrap-b")
    let val = f(x)
    print("wrap-a")
    return val
}

/// FileB.swift
@decorator(before, decoratorBefore) 
@decorator(after, decoratorBefore) 
@decorator(wrap, decoratorBefore) 
func functionToDecorate(x : Int) -> Bool {
    print("toto")
    return false
}

Syntax 2 - More macro-ish like

Pros against syntax 1 :

  • we don't need to modify the code where we do the injection
  • it's way more generic
  • the IDE(atom/emacs/xcode) should show that a function is decorated
@decorator(before|after|wrap) where <Class.AllFunction|Struct.AllFunction|Protocol.AllFunctionImplementation> priority 1-255 { [unowned self], <function-expression-parameters>, originalFunction : (<function-expression-parameters>)->()) in
    <# Your Code #>
}

/// Example

/// FileA.swift

/// This will work **before** the function **B** in class **A**
/// have access to self of type **A**
/// **point** is the parameter of the function **B**
@decorator(before) where <A.B> priority 1 { [unowned self] (point : CGPoint) in
    print("before")
}

/// This will work **after** the function **B** in class **A**
/// have access to self of type **A**
/// ** point** is the parameter of the function **B**
@decorator(after) where <A.B> priority 1 { [unowned self] (point : CGPoint) in
    print("after")
}

/// This will work **around** the function **B** in class **A**
/// have access to self of type **A**
/// **point** is the parameter of the function **B**
/// **f** is the function **B**
/// you can have access to function **B** return value
@decorator(wrap) where <A.B> priority 1 { [unowned self] (point : CGPoint, f : (point : CGPoint) -> Bool) in
    print("wrap-b")
    let val = f(point)
    print("wrap-a")
    return val
}

/// FileB.swift
class A {
    func B(point : CGPoint) -> Bool {
        print("toto")
        return false
    }
}

Source compatibility

This is an additive proposal; existing code will continue to work.

Effect on ABI stability

This feature is purely additive and does not change ABI.

Effect on API resilience

N/A.

Alternatives considered

Using precedence name instead of priority

priority seems to be better understood by non-native English speakers.

Using #selector name instead of #decorator

Using #selector was my first idea, and it was a bad one because this does not do the same as the #selector and can be very confusing on what it tries to achieve.

Not having a priority

The priority help to order the injection of those code blocks.

Adding an @notdecorable

This keyword can be used like @objc, but this keyword prevents other to decorate your code.

If someone tries to decorate importantWork() he will receive a compiler error.

Bar.swift

class bar {
    @notdecorable func work() {
        print("bar is working...")
    }
}

Adding an @decorable

This keyword can be used like @objc, but this keyword is necessary to allow other to decorate your code.

Decoration only works on function with this decorator.

Bar.swift

class bar {
    @decorable func work() {
        print("bar is working...")
    }
}
3 Likes

This is an interesting proposal. I use Python's function decorators occasionally, they're definitely an advanced feature that has use.

One question I have though is is this too close to what a macro system should do? From my time lurking on the mailing lists, any macro system, or macro like system is going to become very hotly debated as to whether it is even a good fit for Swift. Personally I think there is value in introducing something like Python's function wrappers.

1 Like

Python decorators are definitely an interesting idea. I do agree, though, that's it's firmly enough in the macro-like sphere of things that are out of scope for Swift 5.

Interesting @xwu I would have put it in Syntactic additions(that can be considered within Swift 5.0 scope :grin:) rather than macro.

Because this feature has for main goal to help people better organize their code by organizing them by cross-cutting concern instead of mixing these concerns with the rest of the code.

Whereas macro is often used to create a shortcut for more complex routine and “reduce” code duplication by hiding it. Macros aim to add simpler code into a function, this feature aims to remove code from functions.

Even if this feature can be used as a macro in some ways it’s not it’s principal goal. But yes I have to admit that there is some gray area here where macros and this feature are overlaps.

BTW if this feature is fleshed out in a reasonable time I would be able to try to do the implementation before March 1 in order to have something that can be considered in time for Swift 5.

FWIW I didn’t find documentation, expectation or a manifesto that explains where the core team wants to go with macros. They talk about it here and there without a clear vision for the moment because they are not in scope for now.

I think we (or at least I) use "macro" as a category pretty loosely to incorporate any sort of metaprogramming. Redoing mirrors or more advanced reflection, for example, are also out of scope. As are property behaviors. But it's definitely an interesting idea :)

Will this work with private members as well, similar to how @testable works? Otherwise, I see the scope of this being extremely limited if you're trying to do any of the things you've mentioned.

This seems like a contradiction. How would you be able to use decorators with SDKs if many of them are distributed as compiled libraries?

This feature doesn't care about the scope of function and will work with any scope even with visible protocol implementation because it injects code within functions.

Sorry, my explanation wasn't very clear on this.

Imagine that you have to use some third party SDK like google analytics. You would have to sprinkle analytics code everywhere in your codebase. Imagine now instead of that you only have one file where all the analytics code of your app reside. This is what this feature is about decoupling cross-cutting concern like from your business code.

In this one file, you use a third party SDK but isolate it from the rest of the code.

But third party SDK cannot use it because it only works at compile time and we cannot use it on them either.

Great this feature didn’t feat inside your definition of macro :laughing: and as explained in the pitch it will not support runtime injection like AOP because runtime meta programming is out of scope.

I think @xwu was saying this is a form of macro, and thus should be put off until after Swift 5.

Thanks for the clarification @nuclearace I did misunderstand that.

But after consideration, I think my point still stands because it's not meta-programming either.
Metaprogramming is like generics, runtime modification or writing a program that writes another program like gyb.

I really consider it to be more a Syntactic additions.

I think I'm with @xwu that this is very macro-like. The sense in which decorators are "like a macro" is that they can be entirely implemented by a macro system. You don't even need a powerful macro system to do that (like Rust has), C's macro system is sufficiently powerful to achieve the same effect as a decorator here.

If macros are on the table for a future Swift release, I think this should probably be deferred until that point. I come from a Python background and would love to have decorators in the language, but I'd rather have them as a side-effect of a fully-featured macro system.

@lukasa What is the difference between syntactic sugar and macro? Where is the delimitation between both?

For me, all syntactic sugar can be replaced by macro as this feature can be.

? is syntactic sugar for Optional. Is it a macro? or syntactic sugar?

Besides the macro / syntactic sugar issue. Do you think it has potential to help Swift developer?

The best answer I can give is that syntactic sugar is a macro that can only be defined by the compiler: that is, the compiler knows that in the face of a certain syntax it should attempt to perform an automatic code transformation. The power of Rust's macro system is that this is essentially programmable by the end user: the end user is capable of using that macro system to provide essentially their own syntactic sugar.

This is why I made my argument contingent on "if macros are on the table for a future Swift release". If the Swift community does not believe Swift-the-language needs (or can support) a powerful macro system, then we should lean more on providing syntactic sugar, and I'm +1 on this proposal. However, if the plan/hope is to provide a powerful macro system then I'd rather we focus on building that, treating this proposal as one use-case the macro system should be able to support.

To answer your optional question, ? is syntactic sugar, but could in principle be implemented as a macro. The advantage of having optionals built into the compiler is that they allow the compiler to reason about them specially, so I'd defend Optional's presence as a compiler-supported macro. But nonetheless, it could be implemented as a macro, if the compiler team were so inclined.

As to whether this is a useful proposal: yes, unquestionably. The Python community has deployed macros to great effect and I see no reason they couldn't be equally as useful in Swift.

1 Like

I completely agree with @lukasa.
This feature may be really useful, but there are probably others that also would. And a common thing between a lot of proposals is that it probably could be done with a good macro system. Even if ti's not a priority it would mean that we could do things like this ourselves without relaying on the language/compiler to implement it. Rust has some interesting features that rely on the macro system and it powers a lot of cool libs in the ecosystem, I can't imagine what the burden on the compiler would be if everything would need to be implemented in it.
I personally would love to see Run Swift code at compile time being considered at some point.

Specifically WRT the proposal, an issue I have is the complexity-to-utility ratio. There’s a lot of complexity introduced here, and yet I’m having trouble seeing how to apply this in non-trivial code to achieve something useful.

Just taking some examples of the use-cases you describe:

Logging: rarely in my code do I simply want to log the direct inputs or outputs of a function. Where a function implements some transform on its inputs, it often has some conditionality and I’ll want to log (for example) which logical branch it went down. Or if a function has conditional side effects, I’ll want to log when those are triggered. I don’t see how this proposal would facilitate such concerns, so I’m back to embedding my logging in the function.

Analytics: when I’ve instrumented an app for analytics, rarely have the analytics events been un-parameterized, and the parameters often are derived from internal state, or otherwise not directly inputs of the function. Again, there are going to be plenty of cases where externalized before/after bindings just wont cut it.

Where I wind up is thinking that either I’ll have to only use this feature occasionally (when it’s capabilities align with the use-case) or I have to significantly restructure my code and logic (perhaps to the detriment of clarity or performance) in order to externalize enough to use the feautre. Neither justifies the complexity in my mind.

1 Like

Much of the value from function decorators is the ability to inspect and take action on the parameters to the function. For example some decorator that does some access control (strawman syntax):

@adminsOnly
func getSomeSensitiveDataFromDatabase(request: User) throws -> SomeData { }

or a decorator that changes some state during a unit test and then tearing it down after the test is run.

class MyTest : TestCase {
  @setConfig(someSettingIDontWantSetInAnotherTest: true)
  func testSomeSetting() { }
}

You might want to take a look at the old "property behaviours" proposal. I would imagine any kind of function decorator system would also be used for properties and other things.

I can definitely see utility of something along these lines, but the feature described by the proposal doesn’t touch on that utility directly.

Notably missing: the ability to abort the function from a decorator (implied by your example of a throwing decorator), or the ability of a decorator to access the instance (self) of the function being decorated would go a long way to increasing the utility of such a system.

Of course both of those things would increase the complexity even more, and have possible impact that would go beyond the relative harmlessness of the original proposal.

@karim :sweat_smile: I'm terrible at explaining this feature.

You can just add a return if you want to abort or throw.

You always have access to self and all of the original function parameters.

I have looked at property behavior and it's a pretty interesting proposal.