Why does withUnsafePointer(to:) require a var argument?


(Rick M) #1

We have withUnsafePointer(to:) and withUnsafeMutablePointer(to:). Why does the first take an inout parameter? The function names imply that the first will not modify the pointer (which I take to mean its contents), and it makes it quite clunky to pass in constant things.

···

--
Rick Mann
rmann@latencyzero.com


(Florent Bruneau) #2

Hi Rick,

My understanding on this is that withUnsafePointer() requires an inout argument because it has to take a reference to the variable in order to be able to derive its pointer. The languages requires inout arguments to be vars, leading to withUnsafePointer() requiring the passed object to be a var.

There may be other subtleties I'm not aware of, though.

···

Le 27 avr. 2017 à 02:42, Rick Mann via swift-users <swift-users@swift.org> a écrit :

We have withUnsafePointer(to:) and withUnsafeMutablePointer(to:). Why does the first take an inout parameter? The function names imply that the first will not modify the pointer (which I take to mean its contents), and it makes it quite clunky to pass in constant things.

--
Rick Mann
rmann@latencyzero.com

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

--
Florent Bruneau


(Rien) #3

1) When you obtain a pointer, it can no longer be ensured by the compiler that you won’t write to it.
2) A ‘let’ variable (constant) allows way more optimizations than a ‘var’. I would not be surprised if the majority of ‘let’ constants never see any memory allocation at all.

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Balancingrock
Project: http://swiftfire.nl - A server for websites build in Swift

···

On 27 Apr 2017, at 08:31, Florent Bruneau via swift-users <swift-users@swift.org> wrote:

Hi Rick,

My understanding on this is that withUnsafePointer() requires an inout argument because it has to take a reference to the variable in order to be able to derive its pointer. The languages requires inout arguments to be vars, leading to withUnsafePointer() requiring the passed object to be a var.

There may be other subtleties I'm not aware of, though.

Le 27 avr. 2017 à 02:42, Rick Mann via swift-users <swift-users@swift.org> a écrit :

We have withUnsafePointer(to:) and withUnsafeMutablePointer(to:). Why does the first take an inout parameter? The function names imply that the first will not modify the pointer (which I take to mean its contents), and it makes it quite clunky to pass in constant things.

--
Rick Mann
rmann@latencyzero.com

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

--
Florent Bruneau

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Rick M) #4

1) When you obtain a pointer, it can no longer be ensured by the compiler that you won’t write to it.
2) A ‘let’ variable (constant) allows way more optimizations than a ‘var’. I would not be surprised if the majority of ‘let’ constants never see any memory allocation at all.

In my case, it's a field in a struct that's allocated elsewhere, and most certainly exists, and is passed as a parameter, so it's 'let'. I really want to be able to say "I promise I won't write to this", which seems reasonable if the language is willing to give me an unsafe pointer in the first place.

How is the immutable version of withUnsafePointer different from the mutable one? It passes the argument to the closure as a mutable pointer, which must mean something different.

···

On Apr 26, 2017, at 23:37 , Rien via swift-users <swift-users@swift.org> wrote:

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Balancingrock
Project: http://swiftfire.nl - A server for websites build in Swift

On 27 Apr 2017, at 08:31, Florent Bruneau via swift-users <swift-users@swift.org> wrote:

Hi Rick,

My understanding on this is that withUnsafePointer() requires an inout argument because it has to take a reference to the variable in order to be able to derive its pointer. The languages requires inout arguments to be vars, leading to withUnsafePointer() requiring the passed object to be a var.

There may be other subtleties I'm not aware of, though.

Le 27 avr. 2017 à 02:42, Rick Mann via swift-users <swift-users@swift.org> a écrit :

We have withUnsafePointer(to:) and withUnsafeMutablePointer(to:). Why does the first take an inout parameter? The function names imply that the first will not modify the pointer (which I take to mean its contents), and it makes it quite clunky to pass in constant things.

--
Rick Mann
rmann@latencyzero.com

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

--
Florent Bruneau

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

--
Rick Mann
rmann@latencyzero.com


(Rien) #5

1) When you obtain a pointer, it can no longer be ensured by the compiler that you won’t write to it.
2) A ‘let’ variable (constant) allows way more optimizations than a ‘var’. I would not be surprised if the majority of ‘let’ constants never see any memory allocation at all.

In my case, it's a field in a struct that's allocated elsewhere, and most certainly exists, and is passed as a parameter, so it's 'let'. I really want to be able to say "I promise I won't write to this", which seems reasonable if the language is willing to give me an unsafe pointer in the first place.

IIRC this is the reason for the mutable and non-mutable version of the pointer. With the non-mutable pointer you cannot write to the pointee.
(At least not without some additional code)
But it is another step entirely to then allow a let assignment to be used as the pointee. I assume the Swift team decided to keep things simple and not allow let’s to be used as pointee’s. It could probably be done in theory, but I imagine that to be a nightmare case for compiler writers...

Regards,
Rien.

···

On 27 Apr 2017, at 09:54, Rick Mann <rmann@latencyzero.com> wrote:

On Apr 26, 2017, at 23:37 , Rien via swift-users <swift-users@swift.org> wrote:

How is the immutable version of withUnsafePointer different from the mutable one? It passes the argument to the closure as a mutable pointer, which must mean something different.

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Balancingrock
Project: http://swiftfire.nl - A server for websites build in Swift

On 27 Apr 2017, at 08:31, Florent Bruneau via swift-users <swift-users@swift.org> wrote:

Hi Rick,

My understanding on this is that withUnsafePointer() requires an inout argument because it has to take a reference to the variable in order to be able to derive its pointer. The languages requires inout arguments to be vars, leading to withUnsafePointer() requiring the passed object to be a var.

There may be other subtleties I'm not aware of, though.

Le 27 avr. 2017 à 02:42, Rick Mann via swift-users <swift-users@swift.org> a écrit :

We have withUnsafePointer(to:) and withUnsafeMutablePointer(to:). Why does the first take an inout parameter? The function names imply that the first will not modify the pointer (which I take to mean its contents), and it makes it quite clunky to pass in constant things.

--
Rick Mann
rmann@latencyzero.com

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

--
Florent Bruneau

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

--
Rick Mann
rmann@latencyzero.com


(Alex Blewitt) #6

The let constant may not even be stored in a single place; if it's known to be constant it can be in-lined at the point of use and potentially unpacked and dead code elimination throw away the unused members, for example.

If you want to pass in a let constant into the pointer, you can create a copy of it locally in a local variable and then use that instead. However this will be in the local scope, so the pointer isn't valid after it returns.

Alex

···

On 27 Apr 2017, at 09:41, Rien via swift-users <swift-users@swift.org> wrote:

On 27 Apr 2017, at 09:54, Rick Mann <rmann@latencyzero.com> wrote:

On Apr 26, 2017, at 23:37 , Rien via swift-users <swift-users@swift.org> wrote:

1) When you obtain a pointer, it can no longer be ensured by the compiler that you won’t write to it.
2) A ‘let’ variable (constant) allows way more optimizations than a ‘var’. I would not be surprised if the majority of ‘let’ constants never see any memory allocation at all.

In my case, it's a field in a struct that's allocated elsewhere, and most certainly exists, and is passed as a parameter, so it's 'let'. I really want to be able to say "I promise I won't write to this", which seems reasonable if the language is willing to give me an unsafe pointer in the first place.

IIRC this is the reason for the mutable and non-mutable version of the pointer. With the non-mutable pointer you cannot write to the pointee.
(At least not without some additional code)
But it is another step entirely to then allow a let assignment to be used as the pointee. I assume the Swift team decided to keep things simple and not allow let’s to be used as pointee’s. It could probably be done in theory, but I imagine that to be a nightmare case for compiler writers...


(Rick M) #7

...

The let constant may not even be stored in a single place; if it's known to be constant it can be in-lined at the point of use and potentially unpacked and dead code elimination throw away the unused members, for example.

If you want to pass in a let constant into the pointer, you can create a copy of it locally in a local variable and then use that instead. However this will be in the local scope, so the pointer isn't valid after it returns.

Ah, so this brings up another issue, then. Many of the calls in the C library take a pointer to some memory and hang on to it, filling it in at a later point (they make network requests). I've been doing it like this, and it's been working, but I wonder if this is fragile:

class
MyClass
{
    func
    execute()
    {
        self.dataBuffer = Data(count: kLGSImageDataSize)
        precondition(self.dataBuffer != nil, "Unable to allocate image buffer (\(kLGSImageDataSize) bytes)")
    
        var params = c_library_params_t()
        params.data_capacity = self.dataBuffer!.count
    
        self.dataBuffer?.withUnsafeMutableBytes
            { (inBuffer) -> Void in
                // This call returns immediately, but assumes
                // it can write to inBuffer later…
            
                self.request = c_library_call(&params, inBuffer)
            }
    
        if self.request == nil
        {
            // Error
        }
    }

    var dataBuffer: Data?
}

···

On Apr 27, 2017, at 01:48 , Alex Blewitt <alblue@apple.com> wrote:

--
Rick Mann
rmann@latencyzero.com


(Hooman Mehr) #8

You should be able to type your `dataBuffer ` as [Int8] (Byte array). Then you won’t need `withUnsafeMutableBytes`. You can simply call it like this:

self.request = c_library_call(&params, dataBuffer) // Call as if it is a C array

It works because of C interoperability compiler magic.

As long as the instance holding `dataBuffer` is not deallocated and you have not resized the array, the pointer should remain valid.

···

On Apr 27, 2017, at 4:38 PM, Rick Mann via swift-users <swift-users@swift.org> wrote:

On Apr 27, 2017, at 01:48 , Alex Blewitt <alblue@apple.com> wrote:

...

The let constant may not even be stored in a single place; if it's known to be constant it can be in-lined at the point of use and potentially unpacked and dead code elimination throw away the unused members, for example.

If you want to pass in a let constant into the pointer, you can create a copy of it locally in a local variable and then use that instead. However this will be in the local scope, so the pointer isn't valid after it returns.

Ah, so this brings up another issue, then. Many of the calls in the C library take a pointer to some memory and hang on to it, filling it in at a later point (they make network requests). I've been doing it like this, and it's been working, but I wonder if this is fragile:

class
MyClass
{
   func
   execute()
   {
       self.dataBuffer = Data(count: kLGSImageDataSize)
       precondition(self.dataBuffer != nil, "Unable to allocate image buffer (\(kLGSImageDataSize) bytes)")

       var params = c_library_params_t()
       params.data_capacity = self.dataBuffer!.count

       self.dataBuffer?.withUnsafeMutableBytes
           { (inBuffer) -> Void in
               // This call returns immediately, but assumes
               // it can write to inBuffer later…

               self.request = c_library_call(&params, inBuffer)
           }

       if self.request == nil
       {
           // Error
       }
   }

   var dataBuffer: Data?
}

--
Rick Mann
rmann@latencyzero.com

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Rien) #9

To address your question:

https://developer.apple.com/reference/foundation/data/1779823-withunsafemutablebytes

"Warning
The byte pointer argument should not be stored and used outside of the lifetime of the call to the closure."

Which is exactly what you are doing, hence the code is unsafe (no pun intended).

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Balancingrock
Project: http://swiftfire.nl - A server for websites build in Swift

···

On 28 Apr 2017, at 01:38, Rick Mann <rmann@latencyzero.com> wrote:

On Apr 27, 2017, at 01:48 , Alex Blewitt <alblue@apple.com> wrote:

...

The let constant may not even be stored in a single place; if it's known to be constant it can be in-lined at the point of use and potentially unpacked and dead code elimination throw away the unused members, for example.

If you want to pass in a let constant into the pointer, you can create a copy of it locally in a local variable and then use that instead. However this will be in the local scope, so the pointer isn't valid after it returns.

Ah, so this brings up another issue, then. Many of the calls in the C library take a pointer to some memory and hang on to it, filling it in at a later point (they make network requests). I've been doing it like this, and it's been working, but I wonder if this is fragile:

class
MyClass
{
   func
   execute()
   {
       self.dataBuffer = Data(count: kLGSImageDataSize)
       precondition(self.dataBuffer != nil, "Unable to allocate image buffer (\(kLGSImageDataSize) bytes)")

       var params = c_library_params_t()
       params.data_capacity = self.dataBuffer!.count

       self.dataBuffer?.withUnsafeMutableBytes
           { (inBuffer) -> Void in
               // This call returns immediately, but assumes
               // it can write to inBuffer later…

               self.request = c_library_call(&params, inBuffer)
           }

       if self.request == nil
       {
           // Error
       }
   }

   var dataBuffer: Data?
}

--
Rick Mann
rmann@latencyzero.com


(Rick M) #10

So, the dataBuffer has a maximum initial size, then I resize it down to the actual resulting size (the data comes from a sensor and real-world vagaries change the resulting volume of data). Then it eventually gets passed to existing Objective-C++ code, and on to C++ code that wants a void* and length.

Because this runs on iOS, I'd like to reduce the amount of time I hang on to memory as much as possible (there's a lot of parallel processing happening in our pipeline). I'd also like to avoid copying the data.

The resizing was the main reason I chose to use Data.

···

On Apr 27, 2017, at 18:56 , Hooman Mehr <hooman@mac.com> wrote:

You should be able to type your `dataBuffer ` as [Int8] (Byte array). Then you won’t need `withUnsafeMutableBytes`. You can simply call it like this:

self.request = c_library_call(&params, dataBuffer) // Call as if it is a C array

It works because of C interoperability compiler magic.

As long as the instance holding `dataBuffer` is not deallocated and you have not resized the array, the pointer should remain valid.

On Apr 27, 2017, at 4:38 PM, Rick Mann via swift-users <swift-users@swift.org> wrote:

On Apr 27, 2017, at 01:48 , Alex Blewitt <alblue@apple.com> wrote:

...

The let constant may not even be stored in a single place; if it's known to be constant it can be in-lined at the point of use and potentially unpacked and dead code elimination throw away the unused members, for example.

If you want to pass in a let constant into the pointer, you can create a copy of it locally in a local variable and then use that instead. However this will be in the local scope, so the pointer isn't valid after it returns.

Ah, so this brings up another issue, then. Many of the calls in the C library take a pointer to some memory and hang on to it, filling it in at a later point (they make network requests). I've been doing it like this, and it's been working, but I wonder if this is fragile:

class
MyClass
{
  func
  execute()
  {
      self.dataBuffer = Data(count: kLGSImageDataSize)
      precondition(self.dataBuffer != nil, "Unable to allocate image buffer (\(kLGSImageDataSize) bytes)")

      var params = c_library_params_t()
      params.data_capacity = self.dataBuffer!.count

      self.dataBuffer?.withUnsafeMutableBytes
          { (inBuffer) -> Void in
              // This call returns immediately, but assumes
              // it can write to inBuffer later…

              self.request = c_library_call(&params, inBuffer)
          }

      if self.request == nil
      {
          // Error
      }
  }

  var dataBuffer: Data?
}

--
Rick Mann
rmann@latencyzero.com

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

--
Rick Mann
rmann@latencyzero.com


(Rick M) #11

You should be able to type your `dataBuffer ` as [Int8] (Byte array). Then you won’t need `withUnsafeMutableBytes`. You can simply call it like this:

self.request = c_library_call(&params, dataBuffer) // Call as if it is a C array

Hmm. I'm not able to pass self.dataBuffer or self.dataBuffer! to the call. It's expecting `UnsafeMutableRawPointer!`.

···

On Apr 27, 2017, at 18:56 , Hooman Mehr <hooman@mac.com> wrote:

It works because of C interoperability compiler magic.

As long as the instance holding `dataBuffer` is not deallocated and you have not resized the array, the pointer should remain valid.

--
Rick Mann
rmann@latencyzero.com


(Rick M) #12

Yeah, okay. So: how do I do this in a way that is safe?

···

--
Rick Mann
rmann@latencyzero.com

On Apr 27, 2017, at 23:00, Rien <Rien@Balancingrock.nl> wrote:

To address your question:

https://developer.apple.com/reference/foundation/data/1779823-withunsafemutablebytes

"Warning
The byte pointer argument should not be stored and used outside of the lifetime of the call to the closure."

Which is exactly what you are doing, hence the code is unsafe (no pun intended).

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Balancingrock
Project: http://swiftfire.nl - A server for websites build in Swift

On 28 Apr 2017, at 01:38, Rick Mann <rmann@latencyzero.com> wrote:

On Apr 27, 2017, at 01:48 , Alex Blewitt <alblue@apple.com> wrote:

...

The let constant may not even be stored in a single place; if it's known to be constant it can be in-lined at the point of use and potentially unpacked and dead code elimination throw away the unused members, for example.

If you want to pass in a let constant into the pointer, you can create a copy of it locally in a local variable and then use that instead. However this will be in the local scope, so the pointer isn't valid after it returns.

Ah, so this brings up another issue, then. Many of the calls in the C library take a pointer to some memory and hang on to it, filling it in at a later point (they make network requests). I've been doing it like this, and it's been working, but I wonder if this is fragile:

class
MyClass
{
  func
  execute()
  {
      self.dataBuffer = Data(count: kLGSImageDataSize)
      precondition(self.dataBuffer != nil, "Unable to allocate image buffer (\(kLGSImageDataSize) bytes)")

      var params = c_library_params_t()
      params.data_capacity = self.dataBuffer!.count

      self.dataBuffer?.withUnsafeMutableBytes
          { (inBuffer) -> Void in
              // This call returns immediately, but assumes
              // it can write to inBuffer later…

              self.request = c_library_call(&params, inBuffer)
          }

      if self.request == nil
      {
          // Error
      }
  }

  var dataBuffer: Data?
}

--
Rick Mann
rmann@latencyzero.com


(Rien) #13

You will need to take full control over allocation and deallocation.

Here is a piece of code from my project SwifterSockets (on github, see the link in my signature)

public func tipReceiverLoop(
    socket: Int32,
    bufferSize: Int,
    duration: TimeInterval,
    receiver: ReceiverProtocol?) {
    
...
    
    // Allocate the data buffer
    
    let buffer = UnsafeMutableRawPointer.allocate(bytes: bufferSize, alignedTo: 1)
    
    // ===============================================================================
    // This loop stays active as long as the consumer wants more and no error occured.
    // ===============================================================================
    
    var cont = true
    repeat {
        
...
        // ================================================
        // Wait until select signals incoming data activity
        // ================================================
        
        let selres = waitForSelect(socket: socket, timeout: timeout, forRead: true, forWrite: false)
        
        switch selres {
            
...
        case .ready:
            
            // =============
            // Call the recv
            // =============
            
            let bytesRead = Darwin.recv(socket, buffer.assumingMemoryBound(to: UInt8.self), bufferSize, 0)
            
            switch bytesRead {
                
...
            case 1 ... Int.max: // Callback for the received data
                cont = receiver?.receiverData(UnsafeBufferPointer<UInt8>(start: buffer.assumingMemoryBound(to: UInt8.self), count: bytesRead)) ?? true
                
... }
        }
        
    } while cont
    
    // Deallocate the data buffer
    
    buffer.deallocate(bytes: bufferSize, alignedTo: 1)
}

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Balancingrock
Project: http://swiftfire.nl - A server for websites build in Swift

···

On 28 Apr 2017, at 08:02, Rick Mann <rmann@latencyzero.com> wrote:

Yeah, okay. So: how do I do this in a way that is safe?

--
Rick Mann
rmann@latencyzero.com

On Apr 27, 2017, at 23:00, Rien <Rien@Balancingrock.nl> wrote:

To address your question:

https://developer.apple.com/reference/foundation/data/1779823-withunsafemutablebytes

"Warning
The byte pointer argument should not be stored and used outside of the lifetime of the call to the closure."

Which is exactly what you are doing, hence the code is unsafe (no pun intended).

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Balancingrock
Project: http://swiftfire.nl - A server for websites build in Swift

On 28 Apr 2017, at 01:38, Rick Mann <rmann@latencyzero.com> wrote:

On Apr 27, 2017, at 01:48 , Alex Blewitt <alblue@apple.com> wrote:

...

The let constant may not even be stored in a single place; if it's known to be constant it can be in-lined at the point of use and potentially unpacked and dead code elimination throw away the unused members, for example.

If you want to pass in a let constant into the pointer, you can create a copy of it locally in a local variable and then use that instead. However this will be in the local scope, so the pointer isn't valid after it returns.

Ah, so this brings up another issue, then. Many of the calls in the C library take a pointer to some memory and hang on to it, filling it in at a later point (they make network requests). I've been doing it like this, and it's been working, but I wonder if this is fragile:

class
MyClass
{
func
execute()
{
     self.dataBuffer = Data(count: kLGSImageDataSize)
     precondition(self.dataBuffer != nil, "Unable to allocate image buffer (\(kLGSImageDataSize) bytes)")

     var params = c_library_params_t()
     params.data_capacity = self.dataBuffer!.count

     self.dataBuffer?.withUnsafeMutableBytes
         { (inBuffer) -> Void in
             // This call returns immediately, but assumes
             // it can write to inBuffer later…

             self.request = c_library_call(&params, inBuffer)
         }

     if self.request == nil
     {
         // Error
     }
}

var dataBuffer: Data?
}

--
Rick Mann
rmann@latencyzero.com


(Hooman Mehr) #14

You should be able to type your `dataBuffer ` as [Int8] (Byte array). Then you won’t need `withUnsafeMutableBytes`. You can simply call it like this:

self.request = c_library_call(&params, dataBuffer) // Call as if it is a C array

Hmm. I'm not able to pass self.dataBuffer or self.dataBuffer! to the call. It's expecting `UnsafeMutableRawPointer!`.

Oh, if the original C declaration is not `const`, it needs & prefix:

self.request = c_library_call(&params, &dataBuffer)

Also let me clarify that documentation states that it does not guarantee the pointer will stay valid after the call returns, but if the optimizations are on, and you don’t mess with the array, I think it should work. Basically, I wanted to point out an alternative to withUnsafe… stuff, not to address your validity question.

···

On Apr 27, 2017, at 7:31 PM, Rick Mann <rmann@latencyzero.com> wrote:

On Apr 27, 2017, at 18:56 , Hooman Mehr <hooman@mac.com> wrote:

It works because of C interoperability compiler magic.

As long as the instance holding `dataBuffer` is not deallocated and you have not resized the array, the pointer should remain valid.

--
Rick Mann
rmann@latencyzero.com


(Quinn “The Eskimo!”) #15

While I understand what you’re trying to achieve here you have to be careful that you don’t do a bunch of convoluted stuff only to find that, under the covers, the system is doing a copy for you anyway.

iOS’s memory manager, like most memory managers, will often put allocations in buckets based on their size. Thus, reducing the size of an allocation may not actually yield a useful result. Specifically:

* If the memory is in a zone that uses fixed sized buckets, it may not be possible to reduce the memory that you freed by reducing the size

* The system may not like wasting that much memory and actually trigger a copy [1]

The actual behaviour you get definitely varies based on the original size, the new size, and the version of the OS on which you’re running. It may also vary based on the current state of the heap.

The upshot of this is that reducing the size of a memory block may not actually yield the results you’re looking for. It might be better to simplify your code by:

1. Allocating one maximally sized buffer initially

2. Once you’ve read a ‘packet’ of data into that buffer, create a minimally sized data object from that and pass along to the rest of your code

3. Reuse the buffer from 1 for subsequent reads

This will always trigger a copy but a) that copy might have happened anyway, b) it will avoid wasting any memory, and c) it’s much simpler.

Share and Enjoy

···

On 28 Apr 2017, at 03:09, Rick Mann via swift-users <swift-users@swift.org> wrote:

Because this runs on iOS, I'd like to reduce the amount of time I hang on to memory as much as possible (there's a lot of parallel processing happening in our pipeline). I'd also like to avoid copying the data.

The resizing was the main reason I chose to use Data.

--
Quinn "The Eskimo!" <http://www.apple.com/developer/>
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

[1] Consider this small test project:

static void test(size_t size) {
    void * buffer = malloc(size);
    void * initial = buffer;
    buffer = realloc(buffer, size / 2);
    assert(buffer != NULL);
    if (buffer == initial) {
        fprintf(stderr, "%8zu -> %-8zu didn't move\n", size, size / 2);
    } else {
        fprintf(stderr, "%8zu -> %-8zu moved\n", size, size / 2);
    }
    free(buffer);
}

int main(int argc, char **argv) {
    #pragma unused(argc)
    #pragma unused(argv)
    test(1024);
    return EXIT_SUCCESS;
}

When run on macOS 10.12.4 it prints:

    1024 -> 512 moved


(Quinn “The Eskimo!”) #16

D’oh!

For “to reduce the memory” read "to reuse the memory”. Sorry about the confusion.

Share and Enjoy

···

On 28 Apr 2017, at 09:10, Quinn The Eskimo! via swift-users <swift-users@swift.org> wrote:

* If the memory is in a zone that uses fixed sized buckets, it may not be possible to reduce the memory that you freed by reducing the size

--
Quinn "The Eskimo!" <http://www.apple.com/developer/>
Apple Developer Relations, Developer Technical Support, Core OS/Hardware