Guarantees of Tuples as Fixed Sized (stack allocated) Arrays


(Johannes Weiss) #1

Hi swift-users,

(sorry for the cross post to swift-dev, but wasn't sure where it belongs)

I tried to find guarantees about the memory layout Swift tuples but couldn't find anything. The reason I ask is because I'd like to use them as fixed sized (stack allocated) arrays. I'm pretty sure they're actually not guaranteed to be stack allocated but highly likely I assume :).

Am I correct in assuming that

    let swift_events: (kevent, kevent) = ...

has the same memory layout as

    struct kevent c_events[2] = ...

? In other words, is this legal:

    var events = (kevent(), kevent())
    withUnsafeMutableBytes(of: &events) { event_ptr in
        precondition(MemoryLayout<kevent>.size * 2 == event_ptr.count)
        if let ptr = event_ptr.baseAddress?.bindMemory(to: kevent.self, capacity: 2) {
            return kevent(someFileDescriptor, ptr, 2, ptr, 2, nil)
        }
    }

I'm assuming yes but I'd like to make sure.

Many thanks,
  Johannes


(Martin R) #2

As far as I know, the only guarantee is made for structures imported from C. From https://lists.swift.org/pipermail/swift-users/Week-of-Mon-20160516/001968.html :

   Swift structs have unspecified layout. If you depend on a specific layout, you should define the struct in C and import it into Swift for now.

and from https://lists.swift.org/pipermail/swift-users/Week-of-Mon-20160516/001980.html :

   That's not necessary. You can leave the struct defined in C and import it into Swift. Swift will respect C's layout.

Regards, Martin

···

On 28. Apr 2017, at 13:03, Johannes Weiss via swift-users <swift-users@swift.org> wrote:

Hi swift-users,

(sorry for the cross post to swift-dev, but wasn't sure where it belongs)

I tried to find guarantees about the memory layout Swift tuples but couldn't find anything. The reason I ask is because I'd like to use them as fixed sized (stack allocated) arrays. I'm pretty sure they're actually not guaranteed to be stack allocated but highly likely I assume :).

Am I correct in assuming that

   let swift_events: (kevent, kevent) = ...

has the same memory layout as

   struct kevent c_events[2] = ...

? In other words, is this legal:

   var events = (kevent(), kevent())
   withUnsafeMutableBytes(of: &events) { event_ptr in
       precondition(MemoryLayout<kevent>.size * 2 == event_ptr.count)
       if let ptr = event_ptr.baseAddress?.bindMemory(to: kevent.self, capacity: 2) {
           return kevent(someFileDescriptor, ptr, 2, ptr, 2, nil)
       }
   }

I'm assuming yes but I'd like to make sure.

Many thanks,
Johannes

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(John McCall) #3

Hi swift-users,

(sorry for the cross post to swift-dev, but wasn't sure where it belongs)

I tried to find guarantees about the memory layout Swift tuples but couldn't find anything. The reason I ask is because I'd like to use them as fixed sized (stack allocated) arrays. I'm pretty sure they're actually not guaranteed to be stack allocated but highly likely I assume :).

Tuples are guaranteed to use a standard C-style layout wherever that layout is ABI-observable, e.g. when you construct an UnsafePointer to one.

Note that that layout isn't ABI-observable when the tuple is, say, just stored in a struct. The compiler is allowed to break up the tuple and store its components however it likes. Of course, if the compiler does that, and you turn around and construct an UnsafePointer to that property, then the compiler has to reconstitute the tuple into an ABI-compliant temporary that it can give you a pointer to; this is yet another reason why you can't maintain permanent unsafe pointers to components of a struct.

John.

···

On Apr 28, 2017, at 7:03 AM, Johannes Weiss via swift-dev <swift-dev@swift.org> wrote:

Am I correct in assuming that

   let swift_events: (kevent, kevent) = ...

has the same memory layout as

   struct kevent c_events[2] = ...

? In other words, is this legal:

   var events = (kevent(), kevent())
   withUnsafeMutableBytes(of: &events) { event_ptr in
       precondition(MemoryLayout<kevent>.size * 2 == event_ptr.count)
       if let ptr = event_ptr.baseAddress?.bindMemory(to: kevent.self, capacity: 2) {
           return kevent(someFileDescriptor, ptr, 2, ptr, 2, nil)
       }
   }

I'm assuming yes but I'd like to make sure.

Many thanks,
Johannes

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev


(Johannes Weiss) #4

Hi Martin,

As far as I know, the only guarantee is made for structures imported from C. From https://lists.swift.org/pipermail/swift-users/Week-of-Mon-20160516/001968.html :

   Swift structs have unspecified layout. If you depend on a specific layout, you should define the struct in C and import it into Swift for now.

Thanks! Yes I know structs have unspecified layout but that doesn't necessarily mean tuples have that too. Especially if you consider that C structs having fixed sized arrays in them get imported as tuples. See for example this:

typedef struct __siginfo {
  int si_signo; /* signal number */
  int si_errno; /* errno association */
  int si_code; /* signal code */
  pid_t si_pid; /* sending process */
  uid_t si_uid; /* sender's ruid */
  int si_status; /* exit value */
  void *si_addr; /* faulting instruction */
  union sigval si_value; /* signal value */
  long si_band; /* band event for SIGPOLL */
  unsigned long __pad[7]; /* Reserved for Future Use */
} siginfo_t;

gets imported as

--- SNIP ---
public struct __siginfo {
    public var si_signo: Int32 /* signal number */
    public var si_errno: Int32 /* errno association */
    public var si_code: Int32 /* signal code */
    public var si_pid: pid_t /* sending process */
    public var si_uid: uid_t /* sender's ruid */
    public var si_status: Int32 /* exit value */
    public var si_addr: UnsafeMutableRawPointer! /* faulting instruction */
    public var si_value: sigval /* signal value */
    public var si_band: Int /* band event for SIGPOLL */
    public var __pad: (UInt, UInt, UInt, UInt, UInt, UInt, UInt) /* Reserved for Future Use */
    public init()
    public init(si_signo: Int32, si_errno: Int32, si_code: Int32, si_pid: pid_t, si_uid: uid_t, si_status: Int32, si_addr: UnsafeMutableRawPointer!, si_value: sigval, si_band: Int, __pad: (UInt, UInt, UInt, UInt, UInt, UInt, UInt))
}
public typealias siginfo_t = __siginfo
--- SNAP ---

So the `unsigned long __pad[7]` becomes `public var __pad: (UInt, UInt, UInt, UInt, UInt, UInt, UInt)`

and given that, this should really be legal, right?

var random_uint_7_tuple: (UInt, UInt, UInt, UInt, UInt, UInt, UInt) = (1, 2, 3, 4, 5, 6, 7)
var si = siginfo_t()
_ = withUnsafeMutablePointer(to: &si.__pad) { (si_pad_ptr) -> Void in
    return withUnsafeMutablePointer(to: &random_uint_7_tuple) { (tuple_ptr) -> Void in
        si_pad_ptr.assign(from: tuple_ptr, count: 1)
    }
}

from this example I believe we can see that the Swift values of type `(UInt, UInt, UInt, UInt, UInt, UInt, UInt)` MUST have the same layout as `unsigned long[7]` imported from C, right? withUnsafeMutablePointer() to those values gives me a UnsafeMutablePointer<(UInt, UInt, UInt, UInt, UInt, UInt, UInt)> in both cases.

That's why I think inducing unspecified tuple layout from unspecified struct layout doesn't work. Structs are nominal types (so I can't recreate the same type that was imported from C in pure Swift. Tuples on the other hand are structural types which means I am able to recreate the same type that is imported from C in Swift (which is what I did in the example above).

Does that make sense?

Cheers,
  Johannes

···

On 28 Apr 2017, at 12:59, Martin R <martinr448@gmail.com> wrote:

and from https://lists.swift.org/pipermail/swift-users/Week-of-Mon-20160516/001980.html :

   That's not necessary. You can leave the struct defined in C and import it into Swift. Swift will respect C's layout.

Regards, Martin

On 28. Apr 2017, at 13:03, Johannes Weiss via swift-users <swift-users@swift.org> wrote:

Hi swift-users,

(sorry for the cross post to swift-dev, but wasn't sure where it belongs)

I tried to find guarantees about the memory layout Swift tuples but couldn't find anything. The reason I ask is because I'd like to use them as fixed sized (stack allocated) arrays. I'm pretty sure they're actually not guaranteed to be stack allocated but highly likely I assume :).

Am I correct in assuming that

   let swift_events: (kevent, kevent) = ...

has the same memory layout as

   struct kevent c_events[2] = ...

? In other words, is this legal:

   var events = (kevent(), kevent())
   withUnsafeMutableBytes(of: &events) { event_ptr in
       precondition(MemoryLayout<kevent>.size * 2 == event_ptr.count)
       if let ptr = event_ptr.baseAddress?.bindMemory(to: kevent.self, capacity: 2) {
           return kevent(someFileDescriptor, ptr, 2, ptr, 2, nil)
       }
   }

I'm assuming yes but I'd like to make sure.

Many thanks,
Johannes

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Johannes Weiss) #5

Thanks very much John, that's what I expected and hoped for as it's enough to call kevent() for a (kevent_t, kevent_t) used as struct kevent[2] .

···

On 29 Apr 2017, at 00:28, John McCall <rjmccall@apple.com> wrote:

On Apr 28, 2017, at 7:03 AM, Johannes Weiss via swift-dev <swift-dev@swift.org> wrote:
Hi swift-users,

(sorry for the cross post to swift-dev, but wasn't sure where it belongs)

I tried to find guarantees about the memory layout Swift tuples but couldn't find anything. The reason I ask is because I'd like to use them as fixed sized (stack allocated) arrays. I'm pretty sure they're actually not guaranteed to be stack allocated but highly likely I assume :).

Tuples are guaranteed to use a standard C-style layout wherever that layout is ABI-observable, e.g. when you construct an UnsafePointer to one.

Note that that layout isn't ABI-observable when the tuple is, say, just stored in a struct. The compiler is allowed to break up the tuple and store its components however it likes. Of course, if the compiler does that, and you turn around and construct an UnsafePointer to that property, then the compiler has to reconstitute the tuple into an ABI-compliant temporary that it can give you a pointer to; this is yet another reason why you can't maintain permanent unsafe pointers to components of a struct.

John.

Am I correct in assuming that

  let swift_events: (kevent, kevent) = ...

has the same memory layout as

  struct kevent c_events[2] = ...

? In other words, is this legal:

  var events = (kevent(), kevent())
  withUnsafeMutableBytes(of: &events) { event_ptr in
      precondition(MemoryLayout<kevent>.size * 2 == event_ptr.count)
      if let ptr = event_ptr.baseAddress?.bindMemory(to: kevent.self, capacity: 2) {
          return kevent(someFileDescriptor, ptr, 2, ptr, 2, nil)
      }
  }

I'm assuming yes but I'd like to make sure.

Many thanks,
Johannes

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev


(Jordan Rose) #6

Ah, is this true for non-homogeneous tuples? I thought we only guaranteed it for homogeneous ones.

Jordan

···

On Apr 28, 2017, at 16:28, John McCall via swift-users <swift-users@swift.org> wrote:

On Apr 28, 2017, at 7:03 AM, Johannes Weiss via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
Hi swift-users,

(sorry for the cross post to swift-dev, but wasn't sure where it belongs)

I tried to find guarantees about the memory layout Swift tuples but couldn't find anything. The reason I ask is because I'd like to use them as fixed sized (stack allocated) arrays. I'm pretty sure they're actually not guaranteed to be stack allocated but highly likely I assume :).

Tuples are guaranteed to use a standard C-style layout wherever that layout is ABI-observable, e.g. when you construct an UnsafePointer to one.


(John McCall) #7

It's true of all tuples; we have to have some agreement on tuple layout that works generically, and the simple C left-to-right layout is the obvious one. Of course, like I said, we totally reserve the right to not actually store tuples that way in specific classes, structs, enums, locals, argument, results, or whatever else; but in places where a universal ABI is necessary, including pointers, we do make that guarantee.

This notion of universal ABI generally also affects things like generic data structures, which can mean that e.g. a [(MyClass, Bool)] might be stored less efficiently than a [MyPair] where
  struct MyPair { var object: MyClass; var flag: Bool }
But that's just the price of supporting dynamic generics.

John.

···

On May 2, 2017, at 4:38 PM, Jordan Rose <jordan_rose@apple.com> wrote:

On Apr 28, 2017, at 16:28, John McCall via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

On Apr 28, 2017, at 7:03 AM, Johannes Weiss via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
Hi swift-users,

(sorry for the cross post to swift-dev, but wasn't sure where it belongs)

I tried to find guarantees about the memory layout Swift tuples but couldn't find anything. The reason I ask is because I'd like to use them as fixed sized (stack allocated) arrays. I'm pretty sure they're actually not guaranteed to be stack allocated but highly likely I assume :).

Tuples are guaranteed to use a standard C-style layout wherever that layout is ABI-observable, e.g. when you construct an UnsafePointer to one.

Ah, is this true for non-homogeneous tuples? I thought we only guaranteed it for homogeneous ones.