How much memory does withMemoryRebound bind


(Etan Kissling) #1

Hi,

When calling POSIX accept, the common way is to
sockaddr_storage addr = {0};
sockaddr_len addrlen = 0;
int clientFd = accept(serverFd, (sockaddr *) &addr, &addrlen);

In Swift, this translates to
var addr = sockaddr_storage()
var addrlen = sockaddr_len(0)
int clientFd = withUnsafeMutablePointer(to: addr) {
    $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { addr in
        Foundation.accept(socket, addr, &addrlen)
    }
}

Since sockaddr is smaller than sockaddr_storage, I wonder if this is correct.

If withMemoryRebound would be the same as the simple C cast, it would be okay.
However, since it also requires passing the capacity, I wonder if there may be cases
where it actually copies out the memory region, which could lead to memory corruption.

==> How can I guarantee that withMemoryRebound binds the complete sockaddr_storage,
       and prevent cases where only the first MemoryLayout<sockaddr>.size bytes are bound?

Thanks

Etan


(Rien) #2

I used the code from http://blog.obdev.at/representing-socket-addresses-in-swift-using-enums/ in my package SwifterSockets (see github link below)

It does not answer your question exactly, but I think it is a rather better approach to sockaddr usage.

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Swiftrien
Project: http://swiftfire.nl

···

On 29 Dec 2016, at 14:27, Etan Kissling via swift-users <swift-users@swift.org> wrote:

Hi,

When calling POSIX accept, the common way is to
sockaddr_storage addr = {0};
sockaddr_len addrlen = 0;
int clientFd = accept(serverFd, (sockaddr *) &addr, &addrlen);

In Swift, this translates to
var addr = sockaddr_storage()
var addrlen = sockaddr_len(0)
int clientFd = withUnsafeMutablePointer(to: addr) {
   $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { addr in
       Foundation.accept(socket, addr, &addrlen)
   }
}

Since sockaddr is smaller than sockaddr_storage, I wonder if this is correct.

If withMemoryRebound would be the same as the simple C cast, it would be okay.
However, since it also requires passing the capacity, I wonder if there may be cases
where it actually copies out the memory region, which could lead to memory corruption.

==> How can I guarantee that withMemoryRebound binds the complete sockaddr_storage,
      and prevent cases where only the first MemoryLayout<sockaddr>.size bytes are bound?

Thanks

Etan
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Etan Kissling) #3

I meant the question in a more generalized sense.
The sockaddr example is just one that is easily understandable :slight_smile:

Thanks for the link though, interesting read!

···

On 29 Dec 2016, at 14:47, Rien <Rien@Balancingrock.nl> wrote:

I used the code from http://blog.obdev.at/representing-socket-addresses-in-swift-using-enums/ in my package SwifterSockets (see github link below)

It does not answer your question exactly, but I think it is a rather better approach to sockaddr usage.

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Swiftrien
Project: http://swiftfire.nl

On 29 Dec 2016, at 14:27, Etan Kissling via swift-users <swift-users@swift.org> wrote:

Hi,

When calling POSIX accept, the common way is to
sockaddr_storage addr = {0};
sockaddr_len addrlen = 0;
int clientFd = accept(serverFd, (sockaddr *) &addr, &addrlen);

In Swift, this translates to
var addr = sockaddr_storage()
var addrlen = sockaddr_len(0)
int clientFd = withUnsafeMutablePointer(to: addr) {
  $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { addr in
      Foundation.accept(socket, addr, &addrlen)
  }
}

Since sockaddr is smaller than sockaddr_storage, I wonder if this is correct.

If withMemoryRebound would be the same as the simple C cast, it would be okay.
However, since it also requires passing the capacity, I wonder if there may be cases
where it actually copies out the memory region, which could lead to memory corruption.

==> How can I guarantee that withMemoryRebound binds the complete sockaddr_storage,
     and prevent cases where only the first MemoryLayout<sockaddr>.size bytes are bound?

Thanks

Etan
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Rien) #4

Ah, ok.

In that case, I believe it is correct because sockaddr_storage is in fact big enough to hold either the IPv4 or IPv6 structure.
When bits go unused, that causes no harm. And when “addr” goes out of scope, it will be completely deallocated, so no memory leak either.

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Swiftrien
Project: http://swiftfire.nl

···

On 29 Dec 2016, at 15:52, Etan Kissling <etan@scriptreactor.com> wrote:

I meant the question in a more generalized sense.
The sockaddr example is just one that is easily understandable :slight_smile:

Thanks for the link though, interesting read!

On 29 Dec 2016, at 14:47, Rien <Rien@Balancingrock.nl> wrote:

I used the code from http://blog.obdev.at/representing-socket-addresses-in-swift-using-enums/ in my package SwifterSockets (see github link below)

It does not answer your question exactly, but I think it is a rather better approach to sockaddr usage.

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Swiftrien
Project: http://swiftfire.nl

On 29 Dec 2016, at 14:27, Etan Kissling via swift-users <swift-users@swift.org> wrote:

Hi,

When calling POSIX accept, the common way is to
sockaddr_storage addr = {0};
sockaddr_len addrlen = 0;
int clientFd = accept(serverFd, (sockaddr *) &addr, &addrlen);

In Swift, this translates to
var addr = sockaddr_storage()
var addrlen = sockaddr_len(0)
int clientFd = withUnsafeMutablePointer(to: addr) {
$0.withMemoryRebound(to: sockaddr.self, capacity: 1) { addr in
     Foundation.accept(socket, addr, &addrlen)
}
}

Since sockaddr is smaller than sockaddr_storage, I wonder if this is correct.

If withMemoryRebound would be the same as the simple C cast, it would be okay.
However, since it also requires passing the capacity, I wonder if there may be cases
where it actually copies out the memory region, which could lead to memory corruption.

==> How can I guarantee that withMemoryRebound binds the complete sockaddr_storage,
    and prevent cases where only the first MemoryLayout<sockaddr>.size bytes are bound?

Thanks

Etan
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Etan Kissling) #5

Thanks once more!

It all depends on how withMemoryRebound works.

If there is any way where it actually copies the bound memory to a separate place, then passes that copy to the closure,
it would not be a good idea to write past the end of the bound memory.

If withMemoryRebound is a simple cast like in C, everything would be fine (as we know there is an underlying sockaddr_storage big enough),
but since we only pass the smaller size to withMemoryRebound, I'm not sure of the semantics.

In the end, if it would be a simple cast, I wonder what the "capacity" parameter of withMemoryReboundis actually used for...

I also "believe" it is correct but I want to know for sure :slight_smile:

···

On 29 Dec 2016, at 16:17, Rien <Rien@Balancingrock.nl> wrote:

Ah, ok.

In that case, I believe it is correct because sockaddr_storage is in fact big enough to hold either the IPv4 or IPv6 structure.
When bits go unused, that causes no harm. And when “addr” goes out of scope, it will be completely deallocated, so no memory leak either.

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Swiftrien
Project: http://swiftfire.nl

On 29 Dec 2016, at 15:52, Etan Kissling <etan@scriptreactor.com> wrote:

I meant the question in a more generalized sense.
The sockaddr example is just one that is easily understandable :slight_smile:

Thanks for the link though, interesting read!

On 29 Dec 2016, at 14:47, Rien <Rien@Balancingrock.nl> wrote:

I used the code from http://blog.obdev.at/representing-socket-addresses-in-swift-using-enums/ in my package SwifterSockets (see github link below)

It does not answer your question exactly, but I think it is a rather better approach to sockaddr usage.

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Swiftrien
Project: http://swiftfire.nl

On 29 Dec 2016, at 14:27, Etan Kissling via swift-users <swift-users@swift.org> wrote:

Hi,

When calling POSIX accept, the common way is to
sockaddr_storage addr = {0};
sockaddr_len addrlen = 0;
int clientFd = accept(serverFd, (sockaddr *) &addr, &addrlen);

In Swift, this translates to
var addr = sockaddr_storage()
var addrlen = sockaddr_len(0)
int clientFd = withUnsafeMutablePointer(to: addr) {
$0.withMemoryRebound(to: sockaddr.self, capacity: 1) { addr in
    Foundation.accept(socket, addr, &addrlen)
}
}

Since sockaddr is smaller than sockaddr_storage, I wonder if this is correct.

If withMemoryRebound would be the same as the simple C cast, it would be okay.
However, since it also requires passing the capacity, I wonder if there may be cases
where it actually copies out the memory region, which could lead to memory corruption.

==> How can I guarantee that withMemoryRebound binds the complete sockaddr_storage,
   and prevent cases where only the first MemoryLayout<sockaddr>.size bytes are bound?

Thanks

Etan
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Guillaume Lessard) #6

Hi Etan,

`withMemoryRebound` does not copy memory.
The proposal for UnsafeRawPointer contains information about the memory model (as related to pointers):
https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md

(also, the method is defined in the following file:
https://github.com/apple/swift/blob/master/stdlib/public/core/UnsafePointer.swift.gyb)

The capacity parameter of withMemoryRebound allows rebinding a contiguous buffer at once; it might be nice if it had a default value of 1.

Cheers,
Guillaume Lessard


(Etan Kissling) #7

Thank you very much!

···

On 29 Dec 2016, at 23:03, Guillaume Lessard via swift-users <swift-users@swift.org> wrote:

Hi Etan,

`withMemoryRebound` does not copy memory.
The proposal for UnsafeRawPointer contains information about the memory model (as related to pointers):
https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md

(also, the method is defined in the following file:
https://github.com/apple/swift/blob/master/stdlib/public/core/UnsafePointer.swift.gyb)

The capacity parameter of withMemoryRebound allows rebinding a contiguous buffer at once; it might be nice if it had a default value of 1.

Cheers,
Guillaume Lessard

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Andrew Trick) #8

Yep. To further reassure you, `withMemoryRebound` could not be implemented as a copy without breaking various semantics.

Specifically, `forwardPointer` below is semantically equivalent to calling `f` directly.

func forwardPointer(_ p: UnsafePointer<Int64>, to f: (UnsafePointer<Int64>) -> ()) {
  p.withMemoryRebound(to: Int32.self, capacity: 1) {
    $0.withMemoryRebound(to: Int64.self, capacity: 1) {
      f($0)
    }
  }
}

[For an instant, the high bytes are initialized to untyped raw bits. The documentation skirts around this case, but the model is consistent and verifiable.]

The `capacity` label serves to distinguish this cast from the familiar C pointer cast where it’s customary to dereference multiple elements from the resulting typed pointer without specifying the array size. A default capacity=1 would be convenient but misleading.

-Andy

···

On Dec 29, 2016, at 2:03 PM, Guillaume Lessard via swift-users <swift-users@swift.org> wrote:

Hi Etan,

`withMemoryRebound` does not copy memory.
The proposal for UnsafeRawPointer contains information about the memory model (as related to pointers):
https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md

(also, the method is defined in the following file:
https://github.com/apple/swift/blob/master/stdlib/public/core/UnsafePointer.swift.gyb)

The capacity parameter of withMemoryRebound allows rebinding a contiguous buffer at once; it might be nice if it had a default value of 1.

Cheers,
Guillaume Lessard