NSMutableData memory leak


(Ian Partridge) #1

import Foundation
while true {
  var myData: NSMutableData? = NSMutableData(capacity: 0)
  myData = nil
}

Running this infinite loop with swift-corelibs-foundation shows a steady
memory leak, with the process's RSS increasing over time. No leak is seen
with Foundation on Darwin.

Instrumenting with Valgrind's massif profiler shows this stacktrace is
responsible for the leak:

67.36% (114,349B) (heap allocation functions) malloc/new/new[],
--alloc-fns, etc.
->65.01% (110,352B) 0x59F7A89: _CFDataInit
  ->65.01% (110,352B) 0x5B8A8DF:
_TTSf4n_n_n_g_n___TFC10Foundation6NSDatacfT5bytesGSqGSpT___6lengthSi4copySb11deallocatorGSqFTGSpT__Si_T___S0_
    ->65.01% (110,352B) 0x5B873ED:
_TFC10Foundation13NSMutableDataCfT8capacitySi_GSqS0__
      ->65.01% (110,352B) 0x40105D: main

I've stepped through the code with a debugger and observed that the
requested capacity is thrown away
<https://github.com/apple/swift-corelibs-foundation/blob/df239bbbdf5bcdd9ea31c394c6af4dd7c328f99d/Foundation/NSData.swift#L904>
[1]
to begin with. The leak occurs regardless of the capacity requested.

The deinitializer for NSData does call through to _CFDeinit(), which does
then call the finalize()
<https://github.com/apple/swift-corelibs-foundation/blob/ea6179dd35be2c7d9a8f953579f626a5f1be6511/CoreFoundation/Base.subproj/CFRuntime.c#L1773>
[2]
function and hence through to __CFDataDeallocate()
<https://github.com/apple/swift-corelibs-foundation/blob/ea3014bd7883e428727272118cbf37dc56522be6/CoreFoundation/Collections.subproj/CFData.c#L294>
[3].
However, once in __CFDataDeallocate(), the code to free the buffer is
skipped, because __kCFDontDeallocate is set.

If I hack _CFDataInit() so that __kCFDontDeallocate isn't set (by
commenting out this line
<https://github.com/apple/swift-corelibs-foundation/blob/ea3014bd7883e428727272118cbf37dc56522be6/CoreFoundation/Collections.subproj/CFData.c#L337>
[4])
then I get crashes elsewhere - so this obviously isn't the right approach.

I can see that some work has been done in this area
<https://github.com/apple/swift-corelibs-foundation/commit/ea3014bd7883e428727272118cbf37dc56522be6>
[5]
previously by Philippe so I'm wondering if anyone can advise on what might
be going on here?

The init?(length:) initializer avoids CFData entirely and calls malloc()
and free() directly. I'm not sure why that approach was taken and whether
it's relevant to my issue.

Any help would be gratefully received!

Thanks,

[1]
https://github.com/apple/swift-corelibs-foundation/blob/df239bbbdf5bcdd9ea31c394c6af4dd7c328f99d/Foundation/NSData.swift#L904
[2]
https://github.com/apple/swift-corelibs-foundation/blob/ea6179dd35be2c7d9a8f953579f626a5f1be6511/CoreFoundation/Base.subproj/CFRuntime.c#L1773
[3]
https://github.com/apple/swift-corelibs-foundation/blob/ea3014bd7883e428727272118cbf37dc56522be6/CoreFoundation/Collections.subproj/CFData.c#L294
[4]
https://github.com/apple/swift-corelibs-foundation/blob/ea3014bd7883e428727272118cbf37dc56522be6/CoreFoundation/Collections.subproj/CFData.c#L337
[5]
https://github.com/apple/swift-corelibs-foundation/commit/ea3014bd7883e428727272118cbf37dc56522be6

···

--
Ian Partridge


(Philippe Hausler) #2

I think there is probably some likely issue with the fact that NSMutableData is a subclass and the layout initialization is not properly setup during the init for that object.

__kCFDontDeallocate is used to mark the allocated memory as managed by the deallocator blocks. So I bet the problem is that the code-path for NSMutableData(capacity: 0) is hitting the don’t deallocate code path incorrectly (where as the other versions are hitting the appropriate flagging; hence the crashes)

Does this also occur in the Darwin builds of this? (Using SwiftFoundtion instead of Foundation)

···

On May 16, 2016, at 9:34 AM, Ian Partridge via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:

import Foundation

while true {
  var myData: NSMutableData? = NSMutableData(capacity: 0)
  myData = nil
}

Running this infinite loop with swift-corelibs-foundation shows a steady memory leak, with the process's RSS increasing over time. No leak is seen with Foundation on Darwin.

Instrumenting with Valgrind's massif profiler shows this stacktrace is responsible for the leak:

67.36% (114,349B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->65.01% (110,352B) 0x59F7A89: _CFDataInit
  ->65.01% (110,352B) 0x5B8A8DF: _TTSf4n_n_n_g_n___TFC10Foundation6NSDatacfT5bytesGSqGSpT___6lengthSi4copySb11deallocatorGSqFTGSpT__Si_T___S0_
    ->65.01% (110,352B) 0x5B873ED: _TFC10Foundation13NSMutableDataCfT8capacitySi_GSqS0__
      ->65.01% (110,352B) 0x40105D: main
I've stepped through the code with a debugger and observed that the requested capacity is thrown away <https://github.com/apple/swift-corelibs-foundation/blob/df239bbbdf5bcdd9ea31c394c6af4dd7c328f99d/Foundation/NSData.swift#L904> [1] to begin with. The leak occurs regardless of the capacity requested.

The deinitializer for NSData does call through to _CFDeinit(), which does then call the finalize() <https://github.com/apple/swift-corelibs-foundation/blob/ea6179dd35be2c7d9a8f953579f626a5f1be6511/CoreFoundation/Base.subproj/CFRuntime.c#L1773> [2] function and hence through to __CFDataDeallocate() <https://github.com/apple/swift-corelibs-foundation/blob/ea3014bd7883e428727272118cbf37dc56522be6/CoreFoundation/Collections.subproj/CFData.c#L294> [3]. However, once in __CFDataDeallocate(), the code to free the buffer is skipped, because __kCFDontDeallocate is set.

If I hack _CFDataInit() so that __kCFDontDeallocate isn't set (by commenting out this line <https://github.com/apple/swift-corelibs-foundation/blob/ea3014bd7883e428727272118cbf37dc56522be6/CoreFoundation/Collections.subproj/CFData.c#L337> [4]) then I get crashes elsewhere - so this obviously isn't the right approach.

I can see that some work has been done in this area <https://github.com/apple/swift-corelibs-foundation/commit/ea3014bd7883e428727272118cbf37dc56522be6> [5] previously by Philippe so I'm wondering if anyone can advise on what might be going on here?

The init?(length:) initializer avoids CFData entirely and calls malloc() and free() directly. I'm not sure why that approach was taken and whether it's relevant to my issue.

Any help would be gratefully received!

Thanks,

[1] https://github.com/apple/swift-corelibs-foundation/blob/df239bbbdf5bcdd9ea31c394c6af4dd7c328f99d/Foundation/NSData.swift#L904
[2] https://github.com/apple/swift-corelibs-foundation/blob/ea6179dd35be2c7d9a8f953579f626a5f1be6511/CoreFoundation/Base.subproj/CFRuntime.c#L1773
[3] https://github.com/apple/swift-corelibs-foundation/blob/ea3014bd7883e428727272118cbf37dc56522be6/CoreFoundation/Collections.subproj/CFData.c#L294
[4] https://github.com/apple/swift-corelibs-foundation/blob/ea3014bd7883e428727272118cbf37dc56522be6/CoreFoundation/Collections.subproj/CFData.c#L337
[5] https://github.com/apple/swift-corelibs-foundation/commit/ea3014bd7883e428727272118cbf37dc56522be6

--
Ian Partridge

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev


(Ian Partridge) #3

Yes, I have been investigating this using a new TestNSData test under XCode.

···

On 16 May 2016 at 17:43, Philippe Hausler <phausler@apple.com> wrote:

I think there is probably some likely issue with the fact that NSMutableData is a subclass and the layout initialization is not properly setup during the init for that object.

__kCFDontDeallocate is used to mark the allocated memory as managed by the deallocator blocks. So I bet the problem is that the code-path for NSMutableData(capacity: 0) is hitting the don’t deallocate code path incorrectly (where as the other versions are hitting the appropriate flagging; hence the crashes)

Does this also occur in the Darwin builds of this? (Using SwiftFoundtion instead of Foundation)

--
Ian Partridge


(Ian Partridge) #4

We're continuing to investigate this, and have raised SR-1552 to track
it - https://bugs.swift.org/browse/SR-1552

···

--
Ian Partridge