Allow stored properties in extensions

Extensions are a useful way to organize related properties and methods within a source file. Stored instance properties, however, are currently not allowed to be written in an extension:

struct Planet {
  let name: String
}

extension Planet {
  var mass: Double  // error: extensions must not contain stored properties
}

One common pattern is conforming to a a type to a protocol using an extension, and defining all of the members required by the protocol in that extension. Extensions currently aren't allowed to define stored properties, though, so organizing code in this way is not always possible:

protocol AstronomicalObject  {
  var mass: Double { get }
  var parent: AstronomicalObject? { get }
  func terraform()
}

struct Planet {
  let name: String
}

extension Planet: AstronomicalObject {
  var mass: Double // error: extensions must not contain stored properties
  let parent: AstronomicalObject? // error: extensions must not contain stored properties
  func terraform() { /* add an atmosphere, which increases the mass */ }
}

Instead, the stored properties must be defined in the type's base declaration:

struct Planet {
  let name: String
  let atmosphere: Atmosphere?
  // Implements AstronomicalObject:
  var mass: Double
  let parent: AstronomicalObject?
}

extension Planet: AstronomicalObject {
  func terraform() { /* add an atmosphere */ }
}

This limitation is mostly artificial. We could lift this restriction and allow stored properties to be defined in extensions, as long as the extension is in the same file as the type being extended. This would increase flexibility and consistency within individual source files.

I prototyped this change in apple/swift#61593 -- would love to hear what folks think!

Detailed design

We would allow stored properties in extensions of class, struct or actor types which are defined in the same file as the type being extended. The example above would now be permitted to compile:

protocol AstronomicalObject  {
  var mass: Double { get }
  var parent: AstronomicalObject? { get }
  func terraform()
}

struct Planet {
  let name: String
}

extension Planet: AstronomicalObject {
  var mass: Double
  let parent: AstronomicalObject?
  func terraform() { /* add an atmosphere, which increases the mass */ }
}

These properties become part of the type's memory layout as if they were defined in the body of the type, and the ordering of the storage would be based on the ordering of the property declarations in the source file.

For structs, stored properties defined in extensions are included as parameters in the implicit synthesized constructor. Like with properties defined in the type body, the ordering of the parameters in the synthesized constructor declaration would be based on the ordering of the property declarations in the source file. For example:

struct Planet {
  let name: String
}

extension Planet {
  let mass: Double
}

let planet = Planet(name: "Earth", mass: 6e24)

Attempting to extend a type in a different file or module would not be allowed, since there would no longer be a clear, deterministic ordering for the type's stored properties. Attempting to do this would result in the following error:

extension String {
  var identifier: String? // error: only extensions defined in the same file as the extended type are permitted to define stored properties
}

Stored properties are never allowed in enums or protocols, so they can't be permitted in extensions of these types:

enum PlanetaryBodyKind {
  case planet
  case moon
}

extension PlanetaryBodyKind {
  let mass: Double // error: stored property is not permitted in extension of enum
}
protocol AstronomicalObject {
  var mass: Double { get }
}

extension AstronomicalObject {
  var parent: PlanetaryBody? // error: stored property is not permitted in extension of protocol
}

We also wouldn't allow stored properties in constrained extensions of generic types, since there isn't any design precedent for allowing a type to have different sets of stored properties based on the specific generic instantiation:

struct Generic<T> { }

extension Generic {
  let t: T // Fine, since the extension is unconstrained and applied to all instantiations of `Generic`
}

extension Generic where T == Int {
  let u: T // error: stored property is not permitted in constrained extension
}
17 Likes

Not really opposed to this right off the bat—I've definitely run into this issue when writing conformances that required stored properties myself. A couple of thoughts/questions, though.

It feels a bit weird that this would get picked up in the memberwise initializer. One way to look at the memberwise initializer today is that it includes all the stored properties of the struct, but of course an equally correct way to phrase this is that it includes all the properties in the main type body. I like being able to quickly click into a struct and see what I'll need to initialize, and a part of me feels like something would be lost if the structure of the member wise initializer would have to be pieced together mentally from stored property definitions across the file.

Maybe this just isn't that big of a loss since you can always type, say, Planet( and have autocomplete spit out the memberwise init for you anyway, but I can imagine an alternative design where stored properties in extensions wouldn't be part of the memberwise init signature, and instead would be required to have a default value or else result in a "struct has no initializers" error (with a tailored diagnostic explaining why). I don't really know if that would be better or worse, but I'm not immediately sure that including these properties in the memberwise initializer is the right choice and would want to be convinced one way or the other.


A somewhat orthogonal question is whether protocol conformances are the only situation where the inability to define stored properties in extensions is limiting. It's certainly the one that I've encountered, but I think this pitch would be more strongly motivated if there were independent reasons to want this functionality.

If declaring stored properties for conformances is the only motivation, why is allowing stored properties in extensions superior to something like, say, providing a syntax to infer stored properties for unfulfilled protocol requirements? I'm not necessarily saying that I'd prefer such an approach, but if protocol conformance declarations are the only motivating example then is there a solution here that is more directly tied to the problem we're trying to solve, such as only allowing stored properties in extensions when that extension declares a protocol conformance?

Another way of phrasing this discomfort, I suppose, is that I find this code:

pretty undesirable and I'd like to avoid it if possible. Maybe I'm making a bigger deal out of this than it would actually be in practice, but if we could come up with a design that solves the problem posed in a targeted way without creating a total free for all about where in a file stored properties might appear, I think I'd find that preferable.

9 Likes

My reason for wanting this functionality is that it completes the only remaining scenario I have encountered for which it is impossible to refactor a fileprivate member without using that access modifier, which the core team has long stated is meant to be rarely needed.

Members declared in a private extension have fileprivate visibility, but as it is not possible currently to refactor stored properties into an extension, there remain uses of fileprivate which cannot be replaced. With this proposal, we would finally complete the core team’s vision.


I’d echo @Chris_Lattner3’s points elsewhere that I don’t think Swift’s design philosophy requires us to go out of our way to prohibit obviously silly uses of features which have non-silly uses. And a very non-silly use, to my mind, related to the above point about access modifiers, would be as follows:

internal struct Planet {
  let name: String
}

private extension Planet {
  let mass: Double
  let perihelion: Double
}

…and I don’t think there would be much of a principled distinction that would permit all uses like this but also prohibit all silliness.

6 Likes

I should add, another useful benefit of this proposal, which admittedly could be better served with other sugar but which we would get “for free,” is a way to implement deeply nested types without excessive indenting.

That is, for a type Foo.Bar.Baz.Boo, we’d only have to declare struct Boo { } with excess indentation, but everything about the type could be implemented in extension Foo.Bar.Baz.Boo { }.

9 Likes

If the memberwise initializer only included properties defined in the main body, then in this example the memberwise initializer would be Planet.init(name:):

struct Planet {
  let name: String
}

extension Planet {
  let mass: Double
}

That would leave the mass property uninitialized, which would be a problem. Like you mentioned, to prevent this we'd need to either:

  1. no longer provide a synthesized memberwise initializer for structs that defined stored members in extensions, or
  2. require that stored members defined in extensions have a default value

I think both of these restrictions would be limiting and unintuitive to users, in comparison to allowing these properties defined in extensions to appear as parameters in the memberwise initializer.

I understand what you're getting at here, however I think this would also apply today to any large struct declaration, since the set of stored properties could be listed anywhere in the body (which can be arbitrarily long). I see how potentially having stored properties defined further away from the type body could make this harder, but as you mention I think autocomplete is sufficiently helpful here.

Protocol conformances are a useful example here because the group of declarations are almost always semantically related eachother, and because this is a very common organization pattern in my experience.

This is also relevant as a general-purpose code organization tool, even outside of the protocol conformance use case. For example, a pattern I use frequently is defining certain algorithms or operations (and their helper functions or sub-operations) in an extension:

struct MyComplexObject { ... }

extension MyComplexObject {
  func complexAlgorithm() { ... }

  private func helperFunction() { ... }
  private func subOperation() { .... }
}

Imagine you needed to track some state related to this algorithm. As a contrived example we can just count the number of times it's been run. Today we would have to put this property in the main struct body, outside of the extension where all of the other related code lives. If we permitted stored properties to be defined in extensions, this could instead be grouped with the related code:

struct MyComplexObject { ... }

extension MyComplexObject {
  func complexAlgorithm() { 
    algorithmRunCount += 1
    ...
  }

  private var algorithmRunCount = 0

  private func helperFunction() { ... }
  private func subOperation() { .... }
}
4 Likes

Can you expand on that a bit: why ordering is important. Can we tolerate dropping ordering in case of external file vars and, perhaps, keeping it in other cases (if it's really needed)?

I'd also not include these vars in member wise initialisers, even if they are in the same file. The consequence would be - they will always be in a form with initialisation expression:

struct Foo {
   var x: Int
    // the generated member wise init: init(x: Int) { self.x = x }
}

// the same or a different file:
extension Foo {
    var y: Int = 42
    var z: Int // Error, must be initialized
}

See, I don't know that I'd consider this a "win." I guess there's a certain philosophical position here that the main type definition simply shouldn't be special at all, and the file is the true unit of organization that matters with respect to how you are allowed to declare a particular type, but I'm not certain I totally buy in. I'm not certain I don't either, but I do have a gut reaction that tells me the main type definition really ought to be 'special' in that it defines the 'core' of the type with helper functionality relegated to extensions.

It's not just the memberwise init for which knowledge of stored properties is important—these are also the properties used for the synthesis of Equatable, Hashable, and Codable. For someone inspecting a type to determine how such conformance will behave, the scope of their inspection under this change will increase from the main type definition to the entire file. Not insurmountable, of course, but also not something that should be trivially discounted, I think (it also provides a very compelling rationale as to why file scope is the absolute maximum scope we should consider enabling this functionality for, IMO).

This is true, but not particularly compelling as a reason to make it easier to spread the stored properties across the entire file.

Hm. At the point where you have multiple conceptually separate pieces of functionality which each require their own state tracking it feels to me like you should start considering breaking your type into multiple separate types, rather than just organizing your code so that the storage is located nearby in source.

One other thought is that our current resilience story promises that in a resilient library:

Top-level declarations in a source file can be re-ordered, and moved between source files in the same framework.

which would no longer be true under this proposal without some further restrictions. I think it would be reasonable to simply require that frozen types do declare all their storage "up front."

4 Likes

Good question -- I think this depends on whether the definition order of stored properties affects the ABI of a type. e.g. is it ABI-compatible to change a type definition from:

struct Planet {
  let name: String
  let mass: Double
}

to

struct Planet {
  let mass: Double
  let name: String
}

If this is an ABI-compatible change then we may not necessarily need to have a well-defined property ordering. I don't actually know the answer to this -- are these any good resources that describe the types of changes to a type which are / aren't ABI compatible?

Property ordering of course also matters for a struct's memberwise initializer. If we choose to include these stored properties defined in extensions as parameters in the memberwise initializer, then the ordering does matter. This wouldn't isn't source-breaking for API consumers, since synthesized initializers are internal, however:

  1. we'd need some sort of stable ordering to prevent unnecessary churn / source-breakage within the module itself
  2. it seems plausible that we'll eventually have support for public synthesized memberwise initializers, at which point changing the order of properties in a struct would be source breaking for consumers of an API.

I'm not sure if it's the most comprehensive or up-to-date document but the portion I quoted in my reply just above comes from here. (And, in short, property ordering in non-frozen structs is not ABI-visible.)

2 Likes

Thanks for the link! This does answer our question:

Members inside a type or extension can be re-ordered, with the exception of stored properties

So we definitely need to define a clear and unambiguous ordering for a type's stored properties, since changing the ordering is not ABI-stable.

edit: oh, does this only apply to structs explicitly marked as frozen? Good to know.

Interesting, this is good to know. Is this a promise / contract or merely a description of the current behavior? Would we need to completely disallow this feature in resilient libraries to fulfill this behavior?

to me, the real problem we are trying to solve here is excessive indentation, whose own root cause in turn is swift’s penchant for long-winded API names, which rapidly eat up the 80 characters available in a typical editor gutter. so the entire motivation for this feature seems to be driven by the need to claw back some of those 80 characters that get taken up by the indentation.

with that in mind, it would appear to me that the most sensible solution is to go one step further, and allow:

struct Foo.Bar.Baz.Boo
{
}

while leaving all our other existing restrictions on source locations of stored properties in place.

6 Likes

It is not often done this way, but it is possible to create var like properties in extensions today - properties that for all intents and purposes are like real "var" but are implemented via different mechanisms (e.g. external global key-value table or objc associated value, or user defaults, or taking advantage of the fact that class type allows for some user accessible key-value storage, etc). These properties should also be part of Eq/hash/Codable, so that particular problem is not new.

Store them alphabetically? (without respect to locale and other unicode variables of course.)

I think this feature should be introduced as a partial like feature with similar limitations as dotnet. Partial Classes and Methods - C# Programming Guide | Microsoft Learn

At the very least I would suggest that partial like extensions should be restricted to only non alias names.

1 Like

I don't think there's a formal correctness issue with modifying the rules about what changes are ABI compatible, but we'd want to be very careful about any such changes to avoid 'pulling the rug' out from under anyone who has been relying on certain transformations being ABI-compatible. IMO it would be best to keep the policies the same and just disable this functionality for @frozen structs.

IMO, the sorts of properties you describe should absolutely not be part of the synthesized conformance to the protocols mentioned. If users want those properties included they should be required to implement Equatable et al manually, in which case the problem of determining the behavior of the conformance is solved manually because it is written out explicitly.

This appears to be essentially the same rationale by which synthesized Equatable and Hashable conformance was restricted to conformance declarations on the main type. The restriction of course has since been expanded to extensions in the same file.

As you know, the same history is applicable to the private scope, which was eventually expanded so that members are accessible in all same-file extensions.

Over and over again, we are rediscovering the file scope as the meaningful sub-modular unit by which to divide code, even as many first attempts try to eschew it. I think it is fair to say that it has stood the test of time.

Given that it is possible to declare synthesized Equatable and Hashable conformances in same-file extensions, this promise as understood in that manner was broken from the get-go, was it not?

Sorry, I see your meaning, and that is a good point about extension members being reordered in the same file—this could the addressed by always laying out members declared in extensions in alphabetical rather than source order, rather than banning this for resilient types outright, methinks?

1 Like

This is compelling, but I'm not sure I quite agree that these two examples are analogous with the case at hand. In the case of Equatable/Hashable there were compelling functional reasons to allow the conformance synthesis with same-file extensions:

Synthesis is supported in same-file extensions to ensure that generic types can synthesize a conditional conformance, since the properties may only satisfy the requirements for synthesis (see below) with extra bounds:

struct Bad<T>: Equatable { // synthesis not possible, T is not Equatable
    var x: T
}

struct Good<T> {
   var x: T
}
extension Good: Equatable where T: Equatable {} // synthesis works, T is Equatable

And as far as same-file-extension-private-access goes, I don't view the downsides of that decision as being quite as problematic as they are here. Answering the question "what are all the stored properties of this type?" becomes nontrivially harder under this proposal, but answering the question "where does this particular private member get used?" amounts to a 'find' operation within the file with or without same-file-extension access.

I'm mostly focusing on the "re-ordered" part of that sentence, which AFAICT is not violated by synthesized Equatable/Hashable conformances on same-file extensions. Point taken that the "moved between source files" is perhaps not strictly true thanks to such conformances, but at least in that situation the failure mode is a compilation failure and not an ABI break.

EDIT: oop, wires crossed!

Yes, I think this would work, though I'd probably err on the side of disallowing it at first and seeing if we really feel like we need it for @frozen structs. Also, with unicode-supported source files, we'd have to make sure that we define 'alphabetical' in a way that would not change based on the unicode version that the source file is compiled against. :slightly_smiling_face: I have a vague memory that Swift already has some odd behavior around unicode-equivalent identifiers in source?

2 Likes

Agreed, that makes sense to me.

Totally agree, this is a compelling reason to keep this functionality limited to a single file and not the entire module.

I don't think this is a good idea, for a few reasons:

Primarily, I feel like it trades a win for a loss in a way that is at best a wash, and probably a net loss. While it is nice to declare variables locally to their use, it's also nice to go to a type and answer, in one place, "what is this thing". That can be done if you declare all the storage in one place.

My preference for this comes from working on the standard library, back in the days when significant refactors were frequent. I would often find myself wanting to go to a type and ask "what is this thing made up of". But I couldn't: the type's storage declarations were scattered throughout the file in exactly the way proposed here – Stored properties were declared just above where they were first used by some methods. Note some methods, because often that storage was then used much later down in the file too, after some other unrelated stuff. This worked because of another stylistic approach at the time, which was to declare (nearly) all the protocols a type conformed to at the top, instead of grouping them later with extensions.

I found this so confusing that I went through and reorganized most types in the standard library to do two things:

  1. Declare the minimum possible things (really just stored properties and some type aliases and the most fundamental of inits) and then close the definition.
  2. Everything else, either protocol conformances, inner type definitions, or just other methods, were grouped together into short extensions.

Having done this, the standard library code became far easier to work on, at least for me.

Now, I realize this proposal is in part motivated by #2 here. If you break conformances up into extensions, and just declare storage with the extension that needs it, then this makes those extensions more coherent.

But I think it would be a big loss, at least for the standard library code, to not be able to go to the struct definition and see "what is this type made out of – what is the essense of this thing". This is a really important question to answer, especially for low-level code. Is Array just a pointer to a buffer? I can tell you that easily because the declaration of Array is nice and minimal.[1] Same goes for Slice, or String or String.Index.

I admit this is a stylistic preference, but I'd really push to keep any codebase written in this style. I don't think we want to offer more choice in this area.

I might be biased by the fact that I don't tend to work on much business logic code, where perhaps storage required by conformances is more common than in the std lib. But I would also fear that on a sprawling codebase, making storage declaration easier would lead to bloated types where you don't even realize how big the types are getting. And it's also the case that often, storage might be primarily associated with some protocol conformance, but also touched in other places. And when that happens, you're back to the question "what is this type and what does it contain" and I wouldn't want that smeared all over the file.

Secondly (and probably much less importantly) I think this will cause confusion when people try to do this outside the file. A frequently requested feature is to be able to add storage to protocols, or to add storage in extensions even outside the type's module. This is often coming from folks used to a language where everything is a pointer, and they're looking for for sugar for some kind of global lookup table based on the class reference (asking "how would adding storage to Int work?" usually helps). By allowing the syntax in a specific circumstance, you're likely going to upset people hoping for this feature even more.


  1. OK not as nice and minimal as you might wish if you work on it... but it used to be so much worse. ↩︎

39 Likes

This is very well put indeed. I tend to do the same in my projects for big types: only vars and init in main type body, everything else in extensions, which has an added bonus that those extensions could be moved to separate files (aside the infamous problems with overexposing of member visibility). To keep things semi structured I comment the relevant section appropriately:

struct Foo {
    var x: Int
    var y: Int

    // MARK: Foo.Protocol name, or an "informal" extension name, or a file name if extension is in a file
    var z: Int
}

here z is perhaps only used in the relevant extension / file, by specifying the extension / file name it is easier to find such dependencies.

There's also a potential collision course with the "strict conformances" feature we may have one day, a variation of which was pitched recently. In the strictest form it is "a conformance extension only has protocol conformance and nothing else".

It seems to me that the objections/concerns* apply most strongly to structs, and conversely that the points in favor are most relevant for classes. I'd suggest limiting the feature to classes, at least initially.


*Which I agreed with; I'm not particularly enthusiastic about this idea.