Pitch: Introduce custom attributes

Evolution Pitches
21

I think you could at least have every attribute be declared in the language (presumably in Policy.swift in the standard library), but treat certain attributes specially. (Step zero for this is probably something like "open Attr.def, write attribute @<name> { _builtinAttrID: <id> } for each entry, and make the compiler parse that". Then you gradually lean on _builtinAttrID less and less, but you might never get to not leaning on it at all.)

1 Like
22

I strongly disagree with your proposed syntax, I can see your intention, having @ kind of be the function, but that looks pretty bad to me, without mentioning it adds unnecessary parenthesis.

I like the idea of having #available use this. But again I disagree with your syntax.
Having it be called if would require another syntax for checking inside implementations (or a pretty weird call). where it currently is the same, such as

if #available(iOS 8.1, *) {

} else {

}

Also, would prefer to avoid String based API's whenever possible, and again less parenthesis (although the composition would be good). Something like:

@available(.iOS(version: .greaterThanOrEqualTo(8.1)))
class Foo {
    // implementation that can use iOS 8.0+ APIs
}

But a problem that brings, as seen by the "inside implementation" example is that it would require the custom attributes to be able to have a return type too, Bool in this case, which could be interesting, but opens another can of worms.

Just for completion, would like to add that I really liked your examples for "deriving" implementations, apart (again) from the specific syntax >.>

5 Likes
23

Thanks everybody for the feedback. Here's a second version of the pitch.

I've come to agree with the sentiment that including custom attributes that are available at runtime might be a bit much. Therefore, I'll try to make a first proposal only for compile-time attribtues. I don't really understand the steps in the Swift compiler, but it seems to me that attributes should at least be available in libSyntax and SourceKit so that tools can officially deal with them. Any clarifications here are appreciated.

(As an aside, I've never done this before. Would it be better if I hosted the pitch in github or in a gist somewhere as it evolves?)

Introduce custom attributes

Introduction

Swift currently supports marking declarations with attributes, such as @objc and @deprecated. The compiler uses these attributes to change the way it interprets the corresponding code. However, there is no support for declaring and using new attributes to suit the needs of a project or library. This proposal aims to change that.

Previous discussions on the subject:

Motivation

Adding custom attributes to Swift would enable programmers to explore and use new methods of meta-programming. For instance, someone writing a linter might allow users to mark declarations that the linter should ignore, while someone writing a JSON parser might let users mark their properties directly with the right coding keys. This proposal would allow for such use cases and many others that haven't yet been thought of.

Proposed solution

This document proposes adding the following to the language and the compiler:

  • A syntax for declaring new attributes, including:
    • A list of the types of declarations on which they can be used.
    • Associated values and default values (optionally).
  • The ability to use these new attributes to annotate declarations.

It deliberately does not include support for inspecting the attributes in runtime (see Alternatives Considered).

Detailed design

Syntax

Custom attributes would be declared using the following syntax:

<accessLevel> attribute[(listOfSupportedDeclarations)] @<attributeName>[(<associatedValues>)]

And used as in the examples:

// Defines a public attribute called codingKey
internal attribute(.variables) @codingKey(_ key: String = "key")

// Usage
import JSONFramework

class NetworkHandler {
	@codingKey("url_path") var url: URL
}

// Defines an internal attribute called ignoreRules
internal attribute(.functions, .variables, .classes) @ignoreRules(_ rules: [String], printWarnings: Bool = false)

// Usage
import LinterAttributes

@ignoreRules(["cyclomatic_complexity", "line_count"])
func a() {
	// ...
}

Usage

The declared attributes should be made public by both libSyntax and SourceKit, allowing developers to use them to access the attribute information.

Namespacing

Name conflicts can be solved with simple namespacing when needed:

import JSONFrameworkA
import JSONFrameworkB

struct Circle {
	@JSONFrameworkA.codingKey("circle_radius") var radius: Double
	@JSONFrameworkB.codingKey("color_name") var color: Color
}

Annotating types

The Swift compiler supports annotations for types, as it enables the compiler to provide extra guarantees and features regarding those types. However, it isn't clear at the moment how custom attributes for types would be useful, and therefore it is out of the scope of this proposal.

Source compatibility

This proposal is purely additive.

Effect on ABI stability

No effect.

Effect on API resilience

No effect.

Alternatives considered

Currently available workarounds

There's been discussion before on emulating some of this behavior using protocols, an approach that's more limited but currently feasible.

Runtime support

An initial version of this proposal suggested that attributes should be made available at runtime through the reflection API. This idea was positively received in the discussions but was ultimately left for a future proposal, which makes this proposal smaller and allows time to improve the reflection APIs before runtime attributes are considered.

7 Likes
24

May I suggest an alternate syntax:

// Defines an internal attribute called ignoreRules
internal attribute ignoreRules(_ rules: [String], printWarnings: Bool = false): FuncAttribute, VarAttribute, ClassAttribute

Explanation for changes:

  • We know it is an attribute from attribute, we don't need @ here.
  • The name is more important than what it applies to, put the name first.
  • Since the supported declarations were pushed to last,
    and are somewhat protocol-like, enhance the similarity.
9 Likes
25

I think that's a good first step.

The pitch looks great, I like everything in there :+1: Just think the actual syntax could be tweaked a tiny bit.

I like these changes a lot :+1:

Maybe instead of looking like protocol conformance, we could make these look like the get / set modifiers in protocol properties? We could also use keyword names so there are no extra protocols to memorize.

// Defines an internal attribute called ignoreRules
internal attribute ignoreRules(_ rules: [String]) { func var class }
12 Likes
26

I think this is a great improvement to the syntax.

27

It makes sense to simplify this proposal by relegating runtime access to a future proposal, but I think its still important that we think about them. For example, I've always really liked the fact that C# Custom Attributes are defined as standard types and their constructors:

  • it doesn't introduce a new syntax
  • those types serve both as definition and as runtime metadata
  • behaviour can be attached to them

Here's what it could look like in Swift:

@attribute(name: "codingKey", usage: .variables)
internal struct CodingKeyAttribute {
    let key: String

    init(key: String = "key") {
        self.key = key
    }

    var snakeCase: String {
        // ...
    }
}

@attribute(name: "ignoreRules", usage: [.functions, .variables, .classes])
internal struct IgnoreRulesAttribute {
    private let rules: [String]
    private let printWarnings: Bool

    init(_ rules: [String], printWarnings: Bool = false) {
        self.rules = rules
        self.printWarnings = printWarnings
    }
}

Once we allow accessing them at runtime, we could reuse the type as metadata:

for attribute in myObject.reflectionMetadata.attributes {
    if let codingKey = attribute as? CodingKeyAttribute {
        print(codingKey.snakeCase)
    } else if let ignoreRules = attribute as? IgnoreRulesAttribute {
        // Ignore
    }
}
7 Likes
28

I probably jumping into cold water now, I quickly read the latest proposal version and I'm still confused how this should work.

  • What do we gain from custom attributes if they have no behavior at all?
  • Can I write the attribute declaration in multiple lines (with a line width of 80 characters this would be required)?
  • Is it a future direction to allow defining the behavior of an attribute?

I'm here because I just debugged an issue I had with RxSwift library that caused a crash. In RxSwift you have to work with objects called DisposeBag a lot to be able to break strong reference cycles and clean up subscriptions.

I ultimately would want either an attribute for a stored property or property behaviors that would allow me, in defer like fashion of execution, force a stored property to be the top most stored property as it can change the behavior of a serial program.

A quick example:

class Host {
  let crasher = Crasher()
  // This position will allow `crasher` to produce side-effects during `deinit`
  // and *can* lead to a crash of the program.
  let disposeBag = DisposeBag() 
  ...
}

As I mentioned above I would want to write a property behavior or an attribute near that property to force it to be the top most.

@storedPropertyOrder(index: 0) let disposeBag = DisposeBag()

Sure I can 'just' re-order the properties, but from a readability point I would mix all my properties groups which I would like to avoid if possible.

  • Is something like this a feasible future of this feature?

For curios people, here is the issue with a code sample that produces different behavior depending on the oder of the stored properties: Crashed on main queue as destroyed dispose bag did not dispose the subscription Ā· Issue #1900 Ā· ReactiveX/RxSwift Ā· GitHub

29

I considered this at first, but ultimately it seemed to me that it would

  • Not make a lot of sense for attributes that donā€™t have runtime behaviors, as they would create code that never gets interacted with (except for the compiler deriving them into an attribute).
  • Maybe pollute the code with attribute types that would rarely be used in practice. For instance, it could be rare for a user to want to instantiate the CodingKeys type themselves, but it would show up on their autocomplete nonetheless.

Iā€™d love some more feedback on this, but in any case itā€™ll be good to mention in ā€œAlternatives Consideredā€.

Iā€™ll assume youā€™re talking about runtime behavior. Attributes that donā€™t have runtime behavior would still show up in the source code, allowing devs to create tools that interacted with them before theyā€™re compiled (using SourceKit for instance).

Iā€™m not familiar with RxSwift, but there could be a tool that looks at your code and makes sure that all variable declarations marked with @storedPropertyOrder(index: 0) are indeed at index 0, throwing a warning or an error if theyā€™re not.

That said, it seems that your property index example would need special support from the compiler to be implemented, even if we did have runtime support for attributes.

Sure! I thought that was implicit, but maybe itā€™ll be better to mention this in the proposal.

1 Like
30

In a future proposal where attributes can be accessed at runtime, I'd expect all custom attributes to be accessible that way (even those from the Standard Library). If that's what happens, we need a runtime metadata type for attributes anyway.

I don't see this as an issue, and if it really is, it can solved at the SourceKit level, the same way that all internal Standard Library types/properties that are prefixed with an underscore don't show up in code completion.

1 Like
31

I wouldn't mind that syntax at all, especially if it were easier to implement in the compiler. It feels a bit weird to me to define an attribute using what looks like an attribute, but I can get over that.

One other idea, we continue to have the attribute syntax look like a function, but make it return an associated type. The type it returns is what Swift stores in the in the runtime reflection API when that attribute is used.

public attribute(var) codingKey(_ name: String) -> CodingKeyAttribute { 
    return CodingKeyAttribute(name: name)
}

public struct CodingKeyAttribute: CustomStringConvertible {
    public let name: String
    public var description: String { ... }
}

The only thing is I'm not exactly sure how you would access something like this using SwiftSyntax. :thinking:

1 Like
32

Are there existing examples of developers writing tools that interact with comments in source-code? It seems to me that anything one can do with a non-functional custom attribute, one can also do with a custom comment. If this use-case is important and powerful, then Iā€™d like to see what people are already doing with it, since thereā€™s nothing stopping them today.

33

See Sourcery and SwiftLint

3 Likes
34

Oh yeah, we'd definitely need a way to access attributes, I just think it could be simpler. For instance, it could be a bit like the current "children" property in Mirrors:

// In the reflection API
struct Attribute {
    let name: String
    let associatedValues: [String: Any]
}

// Declare the attribute
internal attribute codingKey(_ key: String): VarAttribute

// Use it at runtime
for attribute in myObject.reflectionMetadata.attributes {
    if let attribute.label == "codingKey" {
        let keyToUseInJSON = attribute.associatedValues["key"]
        // ...
    }
}

In my view this has the advantage of being simpler and more in line with the current reflection API, though it definitely has the disadvantage of being less type safe.

I find it hard to justify to myself involving all the functionalities of a struct for attributes that are only meant for the source code. For instance, what does it mean to implement methods in a source-code-only attribute? And what if we never get around to proposing runtime attributes, or if the community ultimately decides it doesn't want them in the language?

It sure is important to consider how this scales into runtime attributes, but also to bear in mind that this proposal has to be able to stand on its own.

This seems to be a valid point. I believe that custom attributes would make it easier for tools like SwiftLint and Sourcery to allow for more complex functionalities (using different types, limiting the kind of declarations they support, etc). However, it's worth noting that they already get a lot done by only using comments, which means that if this proposal is to move forward it should be aware of that and present a clear improvement.

35

It is extremely common for linters and formatters across all languages to look for special comments indicating that a style rule should be modified or disabled for a particular section of code.

The Swift project also encourages incremental improvements over huge, complicated features added all at once; they're easier to design and review, less likely to contain undetected flaws, and the incremental features with their partial functionality can be delivered before the huge feature would be finished. This is especially true when some part of the huge feature depends on some other feature that hasn't been designed yet, which is true of runtime attribute metadataā€”you'll need a new reflection design if you want to actually use it, and that design isn't here yet. From that perspective, factoring this into two proposalsā€”one adding attribute declarations and another adding runtime metadata and APIs to use itā€”makes a lot of sense.

(It might also make sense to add the metadata but not the APIs; Swift has already done this for types and their members generally.)

1 Like
36

That's a good idea too. I was hoping to limit the introduction of new declaration statements. If it feels weird to you to define an attribute using what looks like an attribute, there's always the possibility of making it a protocol conformance:

protocol Attribute {
    var name: String { get }
    var usage: AttributeUsage { get }
}

public struct CodingKeyAttribute: Attribute {
    public let name = "codingKey"
    public let usage = [.properties]
    public let key: String

    init(key: String) {
        self.key = key
    }
}

That's the crux of the problem. It's not because I'm accessing runtime metadata that I want it to be any less type-safe.

6 Likes
37

I've been quiet so far, but am very much supportive of introducing custom attribute support. An immediate motivating use case for me is being able to use custom attributes with Sourcery templates. This would be greatly preferable to using comment-based annotations. One huge win would be the ability of the compiler to validate that custom attributes usage is actually valid. I have some annotation validation logic in my Sourcery templates but not nearly as much as I should. This proposal (and a Sourcery update to take advantage of it) could help a lot with that.

This part felt weird to me as well. I really like the protocol direction a lot, especially because it turns attributes into strongly typed values. That will be extremely useful for both static and dynamic metaprogramming when Swift receives those features. One small tweak I would make is to move the name and usage requirements to be static.

In this model, an attribute is just a value type and attribute usage initializes a value of the corresponding type. Usage sites would need to be limited to compiler evaluable initializers / case constructors. It would be unfortunate to block this feature while waiting for compile time interpreter but this model is compelling enough that it might be worth doing that if necessary. I wonder if there would be a way to provide a limited form of this feature in the meantime by restricting initializer / case argument types.

4 Likes
38

Agreed. So weā€™d be looking at something like:

public enum AttributeUsage {
    case types
    case properties
    case functions
}

public protocol Attribute {
    static var name: String { get }
    static var usage: AttributeUsage { get }
}

// Example

public struct CodingAttribute: Attribute {
    public static let name = "coding"
    public static let usage = .properties
    public let key: String?

    public init(key: String? = nil, ignored: Bool = false) {
        self.key = key
        self.ignored = ignored
    }
}

struct Person: Decodable {
    @coding(key: "first_name")
    let firstName: String

    @coding(key: "last_name")
    let lastName: String

    @coding(ignored: true)
    let age: Int
}

Other question: how precise do we want to be in AttributeUsage? Do we want to allow some attributes to apply only to protocols? To enums? To structs? To extensions? To static members? Etc...

1 Like
39

The name should be just "coding" not "codingKey" I think.

40

Corrected the example. Thanks.