Pitch: Introduce custom attributes

Evolution Pitches
1

Hi all!

I'd like to start a real discussion on a topic that's already been brought up several times (see below). It's come to my attention that implementing it might cause issues with the ABI (as it proposes changes to the reflection API), so it might be better for us to discuss it before it's too late (if it isn't already!).

I haven't given this that much thought so far, so any and all comments and suggestions are much appreciated.

Introduce custom attributes

Introduction

Swift currently supports marking declarations with attributes, such as @objc and @deprecated. The compiler uses these attributes to change the way it interprets the corresponding code. However, there is no support for declaring and using new attributes to suit the needs of a project or library. This proposal aims to change that.

Previous discussions on the subject:

Motivation

Adding custom attributes to Swift would enable programmers to explore and use new methods of meta-programming. For instance, someone writing a UI library might want their users to mark properties that can be animated, while someone writing a linter might allow users to mark declarations that the linter should ignore. This proposal would allow for such use cases and many others that haven't yet been thought of.

Proposed solution

This proposal includes adding the following to the language and the compiler:

  • A syntax for declaring new attributes, including possibly associated values and default values.
  • The ability to use these new attributes to annotate declarations.
  • Support for inspecting attributes in runtime using the reflection API.

Detailed design

Custom attributes would be declared using the following syntax:

<accessLevel> attribute[(availableAtRuntime)] @<attributeName>[(<associatedValues>)]

For example:

// Defines an attribute that's public and available at runtime (i.e. should be accessible from the reflection APIs) called animatable
public attribute(availableAtRuntime) @animatable

// Usage
import MyUIFramework
struct Circle {
	@animatable var radius: Double
}

// Defines an attribute that's internal but not available at runtime (i.e. only shows up in the source code) called ignoreRules, which contains associated values
internal attribute @ignoreRules(_ rules: [String], printWarnings: Bool = false)

// Usage
import LinterAttributes

@ignoreRules(["cyclomatic_complexity", "line_count"])
func a() {
	// ...
}

Runtime Access

Certain uses of custom attributes might need runtime access to work. This access will be provided as an addition the reflection API, as in the following example:

class A {
	@animatable var x: Int = 0
}

let mirror = Mirror(reflecting: A())

for child in mirror.childrenWithAttributes {
    print("Property name:", child.label) // "x"
    print("Property value:", child.value) // 0
    print("Property attributes:", child.attributes) // (name: "animatable", associatedValues: [String: Any]())
}

Namespacing

Name conflicts can be solved with simple namespacing when needed:

import UIFrameworkA
import UIFrameworkB

struct Circle {
	@UIFrameworkA.animatable var radius: Double
	@UIFrameworkB.animatable var color: Color
}

Annotating types

The Swift compiler supports annotations for types, as it enables the compiler to provide extra guarantees and features regarding those types. However, it isn't clear at the moment how custom attributes for types would be useful, and therefore it is out of the scope of this proposal.

Source compatibility

This proposal is purely additive.

Effect on ABI stability

This feature would change existing reflection APIs, but it is not clear in which ways. Help in this topic would be appreciated.

Effect on API resilience

Also unknown.

Alternatives considered

There's been discussion before on emulating some of this behavior using protocols, an approach that's more limited but currently feasible. No other alternatives were considered.

24 Likes
2

Some thoughts on what I'd like to be able to accomplish with this:

For example, in Java Spring, you can configure methods to respond to user agent requests on certain paths with an annotation. In Swift, it would hopefully look like this:

@Controller
@RequestMapping("/foo")
class MyWebController {
    @RequestMapping("/hello")
    func fooHello(request: URLRequest) -> String { return "hello" }
}

When the server framework starts up, it loads class definitions, then finds ones marked with @Controller and instantiates them. In the example above, it then maps requests to /foo* to the instance above, looking inside for methods with more specific request mappings (e.g. a request to "http://myserver.com/foo/hello" results in a call to fooHello(request:)). It would be great to be able to implement similar functionality in Swift.

I like the name "annotation" for these. Maybe "attribute" is already too entrenched.

Java allows annotations to be declared much like classes, although they're introduced with @interface. I think I'd prefer annotation:

enum HttpMethod {
    case get
    case put
    case post
    case head
}
annotation RequestMapping {
    let path: String
    let method: HttpMethod = .get
}

This annotation can then be applied in code:

@RequestMapping(path = "/foo", method = .post)

Or, perhaps more Swifty:

@RequestMapping(path: "/foo", method: .post)

Arguments to the applied annotation can go unnamed, in which case they're applied in declaration order, but perhaps an init() method in the definition would help, as it provides for a lot of flexibility, but implies code execution, which I think makes it hard to introspect a class without loading it and executing that code:

annotation RequestMapping {
    init(_ path: String, method: HttpMethod = .get) {
    }

    let path: String
    let method: HttpMethod = .get
}

So maybe it's better to just infer arguments and allow

@RequestMapping("/foo", .put)

Note that most anything in Java can be annotated. Parameters, too. Dunno if Swift introspection supports this yet.

3 Likes
3

Thanks for the input, @JetForMe. The issue that I see with that example is that it would require the reflection mechanisms to provide a way to enumerate all elements in a program (i.e. all classes), which I think isn't possible right now. I'm not sure if that would be an ABI-breaking change or not, but if it was then it probably deserves some serious consideration.

I'm hoping someone who knows what they're talking about will show up to clear these things up.

4 Likes
4

Even if I had to instantiate the class myself, being able to provide the instance for reflection of the rest of it would be a great improvement.

1 Like
5

Random thoughts: it would be really awesome if this feature could subsume one or more existing attributes (e.g. @IBOutlet) that are currently hard coded into the compiler, moving them out to being library features.

It is important to understanding what the reflection model is for properties, to understand how attributes fit into that.

I'm very much a fan of getting this figured out!

-Chris

26 Likes
6

Yeah, it seems to me that these things would be natural (and very interesting) extensions of this system once it was available.

For now, however, I'm worried about what changes (if any) would be required in the ABI to enable this to be implemented, as it seems important to get this part right before we lock the ABI for Swift 5.

If I remember correctly, @Michael_Ilseman was the one who wrote the ABI manifesto... do you know what we'd need here (or can you point us to someone who does)?

1 Like
7

The ABI is effectively locked down, except for critical bug fixes. This seems like it would be ABI-additive, but I haven’t read it in detail. What parts do you think would affect the ABI?

8

I'm not at all sure, but I think the main concern is the reflection API, since we'd need it to (for instance) tell us which attributes have been added to a declaration. Do you think this could be added?

9

Another use case would be to allow custom CodingKeys without having to redefine all of them in an enum. For example:

struct Person: Codable {
  var name: String
  
  @codingKey("nick_name")
  var nickName: String
}
31 Likes
10

Simple and genius example. +1

11

I'm a big fan of this idea. :+1:

I like the syntax you proposed here for declaring and using the attributes. The API for accessing the attributes is not great though, which is Mirror's fault really. Until that is revamped there is not much we can do.

For example, if I have something like this that @lancep proposed:

struct Person: Codable {
  var name: String
  
  @codingKey("nick_name")
  var nickName: String
}

I would need to be able to access the coding key name statically to properly decode this struct. Mirror only works with instances (i.e., a value of Person, not the type) so it wouldn't work.

Maybe key paths would be useful for this?

let attributes = \Person.nickname.attributes
print(attributes) // [Attribute]

Anyway, I love this idea but I think Mirror (and reflection in general) needs some help before it can be a reality.

4 Likes
12

I also believe that we first need to redesign the reflection APIs before we tackle attributes.

3 Likes
13

Is there a pitch or SE describing the work that needs to happen to Mirror and the reflection APIs? I'd really like to see this pitch not end up in the graveyard of Swift pitches lost to time.

1 Like
14

I've been planning on writing that pitch for a while now. I've been researching what's available in Swift's runtime metadata to see what is available to try to come up with the best API possible. But it's not easy.

15

As I understand it, all of the internal work is done now in Swift 5. All the metadata we need is in the binary. We just need to redesign and implement a better reflection API. @Joe_Groff would know more.

16

I am very much in support of adding a proper attributes DSL to Swift. :ok_hand:t2:

Syntax

Syntactically I would prefer attributes to look something like

@(codable: (key: "date_of_birth", formatter: .dayFormatter))

instead of

@codable(key: "date_of_birth", formatter: .dayFormatter)

Attributes would borrow the argument-part's syntax of Swift's func -calls: …(bar: baz, blee: (qux: 42)) and thus :point_right:t2: could be arbitrarily combined/nested through composition :point_left:t2:.

Examples

Availability Checking

I don't think I've ever managed to get non-trivial availability checks right on first try.

Turning Swift's custom-feature #available

if #available(iOS 8.0, *) { … }

into an ordinary conditional compilation via attributes

@(if: (target: (os: "iOS", version: (greaterThanOrEqualTo: 8.1))))
class Foo {
    // implementation that can use iOS 8.0+ APIs
}

would include the attributed item if the pattern matches.

More Target Checks

System and their versions are not the only things one might want to conditionally compile for:

  • @(if: (target: (arch: "x86_64"))) (only include on x86 64bit targets)
  • @(if: (target: (family: "unix"))) (only include on unix-flavored targets)
  • @(if: (target: (env: "musl"))) (only include on targets using musl instead of libc)
  • @(if: (target: (endian: "little"))) (only include on little-endian targets)
  • @(if: (target: (pointerWidth: 32))) (only include on 32bit targets)
  • @(if: (target: (vendor: "apple"))) (only include on apple targets)
  • @(if: (scheme: "test")) (only include on test executions/builds)
  • @(if: ())
  • …

SIMD

Or take SIMD as another possible use of a composition of nested attributes:

@(if: (target_feature: "avx"))
func foo() {
    // implementation that can use `avx`
}

@(if: (not: (target_feature: "avx")))
func foo() {
    // a fallback implementation
}

Opinionated Examples

Being a strong proponent of explicit syntax I personally would love to see Swift take the next step and migrate lots of its current implicit specialized behaviors into attributes and by that open then up for extension to third-parties.

Synthesized Conformance

As such moving the current auto-conformance of protocols

class Foo: Equatable, Hashable { … }

to an explicit "derive" via attributes:

@(derive: [Equatable, Hashable])
class Foo { … }

Would make it clear that Equatable and Hashable are auto-derived, and not just to be found elsewhere in the code-base, when looking at a type declaration.

Conditional Synthesized Conformance

Unlike the current syntax this would also allow for expressing feature-gated conditional derivation/conformance without having to add any custom-syntax/features besides generalized attributes:

@(if: (feature: "some-feature"), then: (derive: [Equatable, Hashable]))
class Foo { … }

Enum allCases

Swift 4.2 addes support for auto-implementation of static let allCased: [Self] on enums.

This could easily be made opt-in and explicit via attributes:

protocol AllCases {
    static let allCased: [Self] { get }
}

@(derive: AllCases)
enum Foo {
    case Bar, Baz, Blee
}

All of this with a single generalized, yet familiar syntax.

2 Likes
17

I definitely agree that improving the reflection APIs is important. Also, it'd be interesting to already have that done so that we could use it as a basis before discussing custom attributes. However, it seems to me that the main downside of doing custom attributes first would be adding a few more awkward APIs to reflection, and that doesn't seem to be worth postponing this discussion indefinitely (again).

If afterwards we do decide to redesign all of the reflection APIs (and I think we should), then we can also redesign the parts that involve custom attributes, no problem.

18

This thread touches on the many different things people want attributes for. I think it might be helpful, instead of seeing "custom attributes" as one monolithic feature, to more incrementally introduce different features that allow attributes to be defined for different purposes. The design constraints and implementation model for an attribute that's primarily intended for discovery by the compiler or source tools is much different from one that extends a type or property's runtime metadata, or one that is dynamically queryable or searchable at runtime. There might be a small amount of basic infrastructure these features all build from, such as maybe a standardized attribute declaration (similar to the operator/precedencegroup decls we have for introducing operators), but I think the requirements diverge pretty quickly from there, and it would be difficult to design a single omnibus "attributes" feature to please all users.

9 Likes
19

I've been thinking about this lately too, largely along the same lines as Chris.

As you mention, merely being able to declare and parse attributes is itself a useful language feature. Uses become visible in libSyntax/SwiftSyntax/SourceKit parse trees, where tools like linters and Sourcery can act on their presence. Even if they don't have any more effect on the compiler than a comment, they would still be useful.

From a parsing perspective, attribute parameters are going to be a challenge. Swift's grammar does not have a general "attribute argument list" term; each attribute has its own custom argument grammar parsed by bespoke code. The upshot is, we can probably only expect to generalize a subset of the argument syntaxes supported by built-in attributes; I wouldn't expect to see @available or @objc become custom attributes, at least not without using some kind of escape hatch to trigger their custom parsing behavior.

One thing that nobody has mentioned yet is restricting or validating the declarations an attribute can be applied to. It may not make sense to apply @animatable to an initializer or @ignoreRules to an instance variable. If we look at Apple attributes this might subsume, @IBOutlet doesn't make sense on a class, method, static variable, or constant. We need some way to specify these validation rules.

Another feature that I think would be valuable is having one attribute imply others. For example, @IBOutlet should imply @objc; @NSManaged might imply something like @objc dynamic @_semantics("objc.generated_at_runtime_so_dont_emit_anything"). Your custom attributes should have this power, too.

Getting parsing, validation, and implied attributes right seems like a large enough proposal on its own. Because of that, and because the rest of our metadata story isn't there yet, I think it would make sense to defer runtime metadata to a second proposal. By the time those pieces are in place, we can have the basic declaration syntax for an attribute ready to go, so we can just add a way to say an attribute's uses are available at runtime. Maybe you'd put an attribute on your attribute to create a property of your properties?

17 Likes
20

Is there no way that every attribute could be a custom attribute?
I'm thinking of even such things as "public", "private", etc. Another way to say this is that a custom attribute should be as first-class as anything already in the language. There's power in uniformity. The existing attributes could just be defined in a preexisting library; it's analogous to the data structures. Some very basic ones, like Vector, may have to be built in, but most can just be implemented in the language. I can sorta see how to do it for the visibility attributes...