[Swift4][Pitch] Control struct layout with @layout, @offset, @order


(Russ Bishop) #1

I wanted to gauge the interest in supporting explicit struct layout and alignment.

The general idea would be to support attributes to create packed structs and set alignment. It will be critical for certain kinds of interop and systems programming in pure Swift. I don鈥檛 know about you but I want to write a kernel module in Swift someday :slight_smile: I鈥檓 bringing it up because it probably affects ABI stability and resilience and so meets the requirements for Swift 4 phase 1.

Hopefully this could be a jumping-off point for further discussion. If the core team doesn鈥檛 think this affects ABI stability then we can postpone discussing it until Swift 4 phase 1 wraps up.

@layout(style: auto, alignment: natural)

enum LayoutStyle {
聽聽聽聽/// Compiler is free to order the fields as it likes
聽聽聽聽case automatic
聽聽聽聽/// Places a struct's members in source order
聽聽聽聽case sequential
聽聽聽聽/// Requires each member to have an @order attribute
聽聽聽聽case ordered
聽聽聽聽/// Requires each member to have an @offset attribute
聽聽聽聽case explicit
}

Only structs with certain @layout attributes would be exportable to non-Swift languages (assuming such support is added at some point). A struct defined with ordered or explicit layout and as resilient could possibly avoid indirect access to members since the ABI contract guarantees the location of each member.

enum Alignment {
聽聽聽聽/// Compiler decides
聽聽聽聽case natural
聽聽聽聽/// Align as if the natural alignment were specified bytes
聽聽聽聽case bytes(Int)
}

@order(<int>)

Specifies the order of the member in memory. Order must start at zero and increase monotonically without gaps. This makes accidental changes to the ordering errors.

@offset(<int>)

Specifies the offset from the start of the struct where this member should be placed. The size of the struct becomes the highest offset plus the size of the member with that offset.

It is an open question whether overlapping alignment should be allowed. If it is allowed I鈥檇 propose that for any set of members overlapping all of the members must be entirely contained within the largest overlapping member (I鈥檓 thinking of C-style unions when the struct format is imposed by something outside your control).

Thoughts?

Russ


(Mark Lacey) #2

I wanted to gauge the interest in supporting explicit struct layout and alignment.

The general idea would be to support attributes to create packed structs and set alignment. It will be critical for certain kinds of interop and systems programming in pure Swift. I don鈥檛 know about you but I want to write a kernel module in Swift someday :slight_smile: I鈥檓 bringing it up because it probably affects ABI stability and resilience and so meets the requirements for Swift 4 phase 1.

Explicit layout matters for the ABI once the feature exists, but it seems purely additive, i.e. out of scope for Swift 4 phase 1. You can ship without any explicit layout support and add it later without impacting code that ships before the feature exists.

Mark

路路路

On Aug 17, 2016, at 11:28 AM, Russ Bishop via swift-evolution <swift-evolution@swift.org> wrote:

Hopefully this could be a jumping-off point for further discussion. If the core team doesn鈥檛 think this affects ABI stability then we can postpone discussing it until Swift 4 phase 1 wraps up.

@layout(style: auto, alignment: natural)

enum LayoutStyle {
聽聽聽聽/// Compiler is free to order the fields as it likes
聽聽聽聽case automatic
聽聽聽聽/// Places a struct's members in source order
聽聽聽聽case sequential
聽聽聽聽/// Requires each member to have an @order attribute
聽聽聽聽case ordered
聽聽聽聽/// Requires each member to have an @offset attribute
聽聽聽聽case explicit
}

Only structs with certain @layout attributes would be exportable to non-Swift languages (assuming such support is added at some point). A struct defined with ordered or explicit layout and as resilient could possibly avoid indirect access to members since the ABI contract guarantees the location of each member.

enum Alignment {
聽聽聽聽/// Compiler decides
聽聽聽聽case natural
聽聽聽聽/// Align as if the natural alignment were specified bytes
聽聽聽聽case bytes(Int)
}

@order(<int>)

Specifies the order of the member in memory. Order must start at zero and increase monotonically without gaps. This makes accidental changes to the ordering errors.

@offset(<int>)

Specifies the offset from the start of the struct where this member should be placed. The size of the struct becomes the highest offset plus the size of the member with that offset.

It is an open question whether overlapping alignment should be allowed. If it is allowed I鈥檇 propose that for any set of members overlapping all of the members must be entirely contained within the largest overlapping member (I鈥檓 thinking of C-style unions when the struct format is imposed by something outside your control).

Thoughts?

Russ

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Slava Pestov) #3

Hi Russ,

Keep in mind you can control layout of structs by defining them in C (using __attribute__((packed)) and so on), and using them as imported types in Swift.

As Mark mentioned, we can add this later if we need it, so it鈥檚 probably out of scope for now.

Slava

路路路

On Aug 17, 2016, at 11:28 AM, Russ Bishop via swift-evolution <swift-evolution@swift.org> wrote:

I wanted to gauge the interest in supporting explicit struct layout and alignment.

The general idea would be to support attributes to create packed structs and set alignment. It will be critical for certain kinds of interop and systems programming in pure Swift. I don鈥檛 know about you but I want to write a kernel module in Swift someday :slight_smile: I鈥檓 bringing it up because it probably affects ABI stability and resilience and so meets the requirements for Swift 4 phase 1.

Hopefully this could be a jumping-off point for further discussion. If the core team doesn鈥檛 think this affects ABI stability then we can postpone discussing it until Swift 4 phase 1 wraps up.

@layout(style: auto, alignment: natural)

enum LayoutStyle {
聽聽聽聽/// Compiler is free to order the fields as it likes
聽聽聽聽case automatic
聽聽聽聽/// Places a struct's members in source order
聽聽聽聽case sequential
聽聽聽聽/// Requires each member to have an @order attribute
聽聽聽聽case ordered
聽聽聽聽/// Requires each member to have an @offset attribute
聽聽聽聽case explicit
}

Only structs with certain @layout attributes would be exportable to non-Swift languages (assuming such support is added at some point). A struct defined with ordered or explicit layout and as resilient could possibly avoid indirect access to members since the ABI contract guarantees the location of each member.

enum Alignment {
聽聽聽聽/// Compiler decides
聽聽聽聽case natural
聽聽聽聽/// Align as if the natural alignment were specified bytes
聽聽聽聽case bytes(Int)
}

@order(<int>)

Specifies the order of the member in memory. Order must start at zero and increase monotonically without gaps. This makes accidental changes to the ordering errors.

@offset(<int>)

Specifies the offset from the start of the struct where this member should be placed. The size of the struct becomes the highest offset plus the size of the member with that offset.

It is an open question whether overlapping alignment should be allowed. If it is allowed I鈥檇 propose that for any set of members overlapping all of the members must be entirely contained within the largest overlapping member (I鈥檓 thinking of C-style unions when the struct format is imposed by something outside your control).

Thoughts?

Russ

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Joe Groff) #4

Property behaviors could conceivably get you at least some of the way here. You could do something like this:

protocol ManualLayout {
聽聽associatedtype Storage
聽聽var storage: Storage
}

struct Foo: ManualLayout {
聽聽var storage: [4 x Int32] // give me 16 bytes of 32-bit-aligned storage

聽聽var [offset(0)] foo: Int32
聽聽var [offset(5)] bar: Int32
聽聽var [offset(10)] bas: Int32
}

-Joe

路路路

On Aug 17, 2016, at 11:28 AM, Russ Bishop via swift-evolution <swift-evolution@swift.org> wrote:

I wanted to gauge the interest in supporting explicit struct layout and alignment.

The general idea would be to support attributes to create packed structs and set alignment. It will be critical for certain kinds of interop and systems programming in pure Swift. I don鈥檛 know about you but I want to write a kernel module in Swift someday :slight_smile: I鈥檓 bringing it up because it probably affects ABI stability and resilience and so meets the requirements for Swift 4 phase 1.

Hopefully this could be a jumping-off point for further discussion. If the core team doesn鈥檛 think this affects ABI stability then we can postpone discussing it until Swift 4 phase 1 wraps up.

@layout(style: auto, alignment: natural)

enum LayoutStyle {
聽聽聽聽/// Compiler is free to order the fields as it likes
聽聽聽聽case automatic
聽聽聽聽/// Places a struct's members in source order
聽聽聽聽case sequential
聽聽聽聽/// Requires each member to have an @order attribute
聽聽聽聽case ordered
聽聽聽聽/// Requires each member to have an @offset attribute
聽聽聽聽case explicit
}

Only structs with certain @layout attributes would be exportable to non-Swift languages (assuming such support is added at some point). A struct defined with ordered or explicit layout and as resilient could possibly avoid indirect access to members since the ABI contract guarantees the location of each member.

enum Alignment {
聽聽聽聽/// Compiler decides
聽聽聽聽case natural
聽聽聽聽/// Align as if the natural alignment were specified bytes
聽聽聽聽case bytes(Int)
}

@order(<int>)

Specifies the order of the member in memory. Order must start at zero and increase monotonically without gaps. This makes accidental changes to the ordering errors.

@offset(<int>)

Specifies the offset from the start of the struct where this member should be placed. The size of the struct becomes the highest offset plus the size of the member with that offset.

It is an open question whether overlapping alignment should be allowed. If it is allowed I鈥檇 propose that for any set of members overlapping all of the members must be entirely contained within the largest overlapping member (I鈥檓 thinking of C-style unions when the struct format is imposed by something outside your control).

Thoughts?


(Chris Lattner) #5

Agreed. This is something that we鈥檇 like Swift to support someday, but it isn鈥檛 on the critical path, and it would be a distraction to dive deep on it right now.

-Chris

路路路

On Aug 17, 2016, at 11:37 AM, Mark Lacey via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 17, 2016, at 11:28 AM, Russ Bishop via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I wanted to gauge the interest in supporting explicit struct layout and alignment.

The general idea would be to support attributes to create packed structs and set alignment. It will be critical for certain kinds of interop and systems programming in pure Swift. I don鈥檛 know about you but I want to write a kernel module in Swift someday :) I鈥檓 bringing it up because it probably affects ABI stability and resilience and so meets the requirements for Swift 4 phase 1.

Explicit layout matters for the ABI once the feature exists, but it seems purely additive, i.e. out of scope for Swift 4 phase 1. You can ship without any explicit layout support and add it later without impacting code that ships before the feature exists.


(Russ Bishop) #6

No problem, I'll put it on the back burner for the time being.

Russ

路路路

On Aug 19, 2016, at 12:50 PM, Joe Groff <jgroff@apple.com> wrote:

On Aug 17, 2016, at 11:28 AM, Russ Bishop via swift-evolution <swift-evolution@swift.org> wrote:

I wanted to gauge the interest in supporting explicit struct layout and alignment.

The general idea would be to support attributes to create packed structs and set alignment. It will be critical for certain kinds of interop and systems programming in pure Swift. I don鈥檛 know about you but I want to write a kernel module in Swift someday :slight_smile: I鈥檓 bringing it up because it probably affects ABI stability and resilience and so meets the requirements for Swift 4 phase 1.

Hopefully this could be a jumping-off point for further discussion. If the core team doesn鈥檛 think this affects ABI stability then we can postpone discussing it until Swift 4 phase 1 wraps up.

@layout(style: auto, alignment: natural)

enum LayoutStyle {
聽聽聽/// Compiler is free to order the fields as it likes
聽聽聽case automatic
聽聽聽/// Places a struct's members in source order
聽聽聽case sequential
聽聽聽/// Requires each member to have an @order attribute
聽聽聽case ordered
聽聽聽/// Requires each member to have an @offset attribute
聽聽聽case explicit
}

Only structs with certain @layout attributes would be exportable to non-Swift languages (assuming such support is added at some point). A struct defined with ordered or explicit layout and as resilient could possibly avoid indirect access to members since the ABI contract guarantees the location of each member.

enum Alignment {
聽聽聽/// Compiler decides
聽聽聽case natural
聽聽聽/// Align as if the natural alignment were specified bytes
聽聽聽case bytes(Int)
}

@order(<int>)

Specifies the order of the member in memory. Order must start at zero and increase monotonically without gaps. This makes accidental changes to the ordering errors.

@offset(<int>)

Specifies the offset from the start of the struct where this member should be placed. The size of the struct becomes the highest offset plus the size of the member with that offset.

It is an open question whether overlapping alignment should be allowed. If it is allowed I鈥檇 propose that for any set of members overlapping all of the members must be entirely contained within the largest overlapping member (I鈥檓 thinking of C-style unions when the struct format is imposed by something outside your control).

Thoughts?

Property behaviors could conceivably get you at least some of the way here. You could do something like this:

protocol ManualLayout {
associatedtype Storage
var storage: Storage
}

struct Foo: ManualLayout {
var storage: [4 x Int32] // give me 16 bytes of 32-bit-aligned storage

var [offset(0)] foo: Int32
var [offset(5)] bar: Int32
var [offset(10)] bas: Int32
}

-Joe