SE-0210: Add an offset(of:) method to MemoryLayout

Hello Swift Community,

The review of SE-0210: "Add an offset(of:) method to MemoryLayout begins now and runs through April 25th, 2018.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to avoid this forum or just keep your feedback private, directly to me via email, twitter, or direct message. If emailing me directly, please put “SE-0210: "Add an offset(of:) method to MemoryLayout” in the subject line. Your feedback is equally valuable to us no matter how we receive it.

What goes into a review of a proposal?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift.

When reviewing a proposal, here are some questions to consider:

What is your evaluation of the proposal?

Is the problem being addressed significant enough to warrant a change to Swift?

Does this proposal fit well with the feel and direction of Swift?

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Thanks,
Doug
Review Manager

9 Likes

A +1 from me: this proposal is simple, has practical use cases, and enables previously-unsupported functionality in Swift.

As a side note, I think the subscript in the alternatives considered would be a useful (albeit niche) addition; I've actually had a use-case recently for it:

extension UnsafeMutablePointer {
    func initialize<Base>(from base : UnsafePointer<Base>, path: KeyPath<Base, Pointee>, count: Int) {
        for i in 0..<count {
              self.advanced(by: i).initialize(to: base[i][keyPath: path])
        }
}

would become:

extension UnsafeMutablePointer {
    func initialize<Base>(from base : UnsafePointer<Base>, path: KeyPath<Base, Pointee>, count: Int) {
        for i in 0..<count {
              self.advanced(by: i).initialize(from: base.advanced(by: i)[path])
        }
}

which I think is a little clearer. That may be something that's better left to user-code extensions, though, since the subscripts themselves are fairly trivial to implement.

What is your evaluation of the proposal?

+1

Is the problem being addressed significant enough to warrant a change to Swift?

Yes

Does this proposal fit well with the feel and direction of Swift?

Yes

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

Yes, C.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Followed the original thread and prefered the MemoryLayout option to the alternatives.

What is your evaluation of the proposal?

+1

Is the problem being addressed significant enough to warrant a change to Swift?

Motivation section is clear.

I also recently had a "problem" of several nested withUnsafeMutablePointer(to:) to multiple properties of a single value, that this proposal will flatten.

Does this proposal fit well with the feel and direction of Swift?

It's great to see high-level keypaths meet low-level memory layout, and a brilliant demonstration of inner consistency! Kudos!

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

n/d

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

quick

I'm a +1 on this proposal in general.

@Joe_Groff Can you explain to me a little what it means that this only applies to struct properties? I was under the assumption that this API was for querying the basic byte index of properties inside of a struct. In which case I'm a little confused as to why it says it only applies to struct properties. Excuse my ignorance in this area, but aren't class properties inside of a struct just a word-sized pointer to the actual instance of the object? If so, what stops us from querying for the offset to the pointer?

Is this some limitation of key paths that could be fixed in the future?

Also, is there something that stops us from applying this same kind of logic to class instances?

Sorry, by "struct properties" I meant "properties inside a struct". You should be able to get the offset of any (non-observed, non-reabstracted, non-bitpacked) stored property in a struct regardless of the property's type.

4 Likes

What is your evaluation of the proposal?

+1 to adding the ability to access the offset. But I do have some questions about the specific design choices.

Maybe I'm missing something obvious, but the example in the proposal doesn't seem to match the declaration:

extension MemoryLayout {
  func offset(of key: PartialKeyPath<T>) -> Int?
}

This declaration returns Int? rather than Int. This is necessary due to the choice to use a function which accepts an arbitrary PartialKeyPath<T>. However, the example doesn't appear to handle the optional result:

var root: T, value: U
var key: WritableKeyPath<T, U>
// Mutation through the key path...
root[keyPath: \.key] = value
// ...is exactly equivalent to mutation through the offset pointer...
withUnsafePointer(to: &root) {
  (UnsafeMutableRawPointer($0) + MemoryLayout<T>.offset(of: \.key))
    // ...which can be assumed to be bound to the target type
    .assumingMemoryBound(to: U.self).pointee = value
}

Let's rewrite the example to handle the optional:

withUnsafePointer(to: &root) {
   guard let offset = MemoryLayout<T>.offset(of: \.key) else {
      /// what do we do here???
      return 
   }
  (UnsafeMutableRawPointer($0) + offset)
    // ...which can be assumed to be bound to the target type
    .assumingMemoryBound(to: U.self).pointee = value
}

I am wondering if a design that was focused on providing offsets for properties that can be statically verified to be directly addressable is viable (and therefore return Int rather than Int?, producing a compiler error if the specified property is not directly addressable). If it is, should that be offered in place of or along side the dynamic version? If a design along these lines isn't viable, what specific challenges are there?

Is the problem being addressed significant enough to warrant a change to Swift?

Yes, definitely.

Does this proposal fit well with the feel and direction of Swift?

In general, yes. But see the questions listed above.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I have used C. From a programmer's point of view, the most significant difference is that offsets in C are not optional. Swift is a higher level language that cannot provide an offset for every property, but perhaps it would be better to statically verify the direct addressability of a property when possible to avoid the need to manually handle nil in all code that works with offsets.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

A quick, but detailed reading.

2 Likes

Doesn't that run into the issue where a third party vended lib changes a once inline property to be a computed property? That would probably run into undefined behavior if you return a straight Int.

If you know the property is stored, you can safely force-unwrap the result. Maybe in the future with a more protocol-based key path design, we could statically distinguish addressable stored key paths from others. Returning Optional seems to me like a reasonable tradeoff between expressivity and type safety in the meantime.

Perhaps, unless that lib used @frozen to make offsets part of the public ABI. But that would obviously never be an issue for code in the same module.

Thanks, that answers my questions.

Is it not a programmer error, then, to use this new API with properties that do not have any offset? Is it possible that the type comes at runtime, and that one has to check the optional?

If not, shouldn't we fatalError/preconditionFail instead of returning an optional?

I'm inclined to agree that in most use cases, code will apply this with fields known to be stored and have no reasonable fallback for when an offset isn't formable. I can see conceivable use cases where knowing a property is offsetable might let you directly access a property as an optimization, but where you could still do normal property access from the base value as a fallback, though. Whether the result should be Optional or trap to me depends on how common those cases are, and how big the benefit of conditionally taking advantage of offsetability is.

Oh, I overlooked the fact that if the root type is usually statically known, the key path is not. I now see how the optional can be used. Thanks for your answer.