Crash when adding particular Strings to Arrays with C++ interop

efroemling · October 24, 2023, 3:17am

I'm constructing Swift String Arrays in C++ to pass into Swift code and I'm finding that adding particular strings seems to choke things up and leads to a crash when the Array goes out of scope. I can repro this in a fresh Xcode 15.0.1 (15A507) template project by turning on C++ interop and wiring up the code below.

Am I doing something wrong here, or does this seem like a bug? These strings all seem to be valid utf-8, and everything seems to work correctly if I take out the offending few.

Swift code:

static public func printStrings(strings: [String]) {
  for s in strings {
    print("GOT STRING", s)
  }
  print("DONE PRINTING")
}

C++ code:

{
  auto array = swift::Array<swift::String>::init();

  // These are all good.
  array.append("தமிழ்");
  array.append("简体中文");
  array.append("繁體中文");
  array.append("हिंदी");

  // This one's trouble.
  array.append("ภาษาไทย");
    
  // As is this one.
  array.append("🤷🏼‍♂️");

  // (note: still crashes without this)
  SwiftTestProject::printStrings(array)

  // Crashes here when tearing down array.
}

output:

GOT STRING தமிழ்
GOT STRING 简体中文
GOT STRING 繁體中文
GOT STRING हिंदी
GOT STRING
GOT STRING
DONE PRINTING
<EXC_BAD_ACCESS in swift::Array<swift::String>::~Array()>

wadetregaskis · October 24, 2023, 6:20am

I wonder if it'd help to prefix the literals with u8? (e.g. u8"ภาษาไทย")

Maybe in the absence of that explicit encoding indicator, C++ is somehow mangling the encoding? Bit odd that it would be only for some non-ASCII characters, though.

You could also try using Unicode escape sequences for at least one of your problematic strings, and see if that helps. e.g. u8"\ud83e\udd37\ud83c\udffc\u200d\u2642\ufe0f" for your shrug emoji there.

michelf · October 24, 2023, 10:59am

That sounds like an access to deallocated memory to me.

What's the difference between the good and bad strings?

// These are all good.
"தமிழ்".utf8.count // → 15
"简体中文".utf8.count // → 12
"繁體中文".utf8.count // → 12
"हिंदी".utf8.count // → 15

// This one's trouble.
"ภาษาไทย".utf8.count // → 21
    
// As is this one.
"🤷🏼‍♂️".utf8.count // → 17

All the bad ones are above 15 bytes. I believe 15 is the upper limit of String's inline storage (for non-ASCII strings). Above that threshold String's storage is on the heap.

tera · October 24, 2023, 11:09am

Well spotted. Then I guess just this will crash as well?

array.append("0123456789ABCDEF");

michelf · October 24, 2023, 11:12am

I think ASCII strings are packed more tightly so they can still fit in the inline storage above 15. ASCII is a 7-bit encoding, so that leaves extra bits. I'm not sure of the exact threshold but 17 or 18 bytes sounds likely for ASCII strings.

tera · October 24, 2023, 11:18am

15 is the threshold:

withUnsafeBytes(of: "0123456789ABCDE") { p in
    dumpHex(p.baseAddress!, 16) // 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 ef
    // 16 here is for MemoryLayout<String>.size
}
withUnsafeBytes(of: "0123456789ABCDEF") { p in
    dumpHex(p.baseAddress!, 16) // 10 00 00 00 00 00 00 d0 50 7c 03 00 01 00 00 80
}

michelf · October 24, 2023, 11:58am

I might have mixed things with NSString's tagged pointer representation. Looks like the actual limit for Swift.String inline storage is either 15, 14, or 10 depending on the platform.

efroemling · October 24, 2023, 1:01pm

Just confirmed that 15 does seem to be the cutoff here.

  auto array = swift::Array<swift::String>::init();
  array.append("0123456789ABCDE"); // len 15
  array.append("0123456789ABCDEF"); // len 16

Gives me:

GOT STRING 0123456789ABCDE
GOT STRING 
DONE PRINTING.
<EXC_BAD_ACCESS in swift::Array<swift::String>::~Array()>

(Also just noticed that the bad prints are full of U+FFFD (replacement character) if that means anything).

Alex_L · October 24, 2023, 4:33pm

Thanks for reporting this issue. Could you please file an issue on GitHub - apple/swift: The Swift Programming Language, and we'll work on fixing it.

efroemling · October 24, 2023, 6:19pm

Ok, issue filed. Thanks everyone for taking a look at this.

github.com/apple/swift

C++ interop crash adding Strings with > 15 chars to Swift Array.

opened 06:18PM - 24 Oct 23 UTC

efroemling

bug triage needed

**Description** When creating a Swift Array of Strings from C++ code (as described [in the guide](https://www.swift.org/documentation/cxx-interop/#using-swift-array-in-c)), adding strings with a size greater than 15 bytes causes a crash. **Steps to reproduce** The following steps reproduce the crash for me using XCode 15.0.1 (15A507) on an M1 Max MBP running Sonoma 14.1: - Create a new ‘Command Line Tool’ project named “TestCommand”. - Create a new C++ file named ‘test’ (with header). Click ‘Create Bridging Header’ when it asks. - In TestCommand-Bridging-Header.h, add: ```c++ #include "test.hpp" ``` - In test.hpp, add: ```c++ void doTest(); ``` - In test.cpp, add: ```c++ #include <TestCommand-Swift.h> void doTest() { // Pass an array back to our Swift code to print. auto array = swift::Array<swift::String>::init(); array.append("foo"); TestCommand::printStrings(array); } ``` - In main.swift, add: ```swift // Expose a print call for C++ to use. public func printStrings(_ strings: [String]) { for s in strings { print("GOT STRING", s) } print("DONE PRINTING.") } // Call our c++ code. doTest() ``` - Under build settings for the TestCommand target, set C++ and Objective-C Interoperability to “C++/Objective C++” - You should now be able to build and run the project, which should give the following output: ``` Hello, World! GOT STRING foo DONE PRINTING. Program ended with exit code: 0 ``` - Now, to trigger the crash, simply change the "foo" in test.cpp to "123456789ABCDEFG" and run again. You should see the following output followed by an EXC_BAD_ACCESS crash at the end of the doTest function. ``` Hello, World! GOT STRING DONE PRINTING. <crashes here in swift::Arrayswift::String::~Array()> ``` As per the discussion of this crash [on the swift forums](https://forums.swift.org/t/crash-when-adding-particular-strings-to-arrays-with-c-interop/68021), it appears that the threshold for causing a crash is strings > 15 characters which corresponds to String's inline storage size.  **Environment** - Swift compiler version info: swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1) - Xcode version info: Xcode 15.0.1 Build version 15A507 - Deployment target: macOS Sonoma 14.1 (23B73)

Alex_L · October 24, 2023, 7:40pm

Thank you!

efroemling · October 24, 2023, 8:17pm

But wait, there's more!

So I'm now looking into workarounds and trying to pass my stuff in to Swift as std::vectors instead of converting to Swift arrays, but I'm running into a different (possibly unrelated?) issue there.

I'm defining a class in swift:

public class TestClass {
  
  public init(_ values: CppFloatVec) {
    for v in values {
      print("GOT VALUE", v)
    }
    print("DONE PRINTING.")
  }
  
  static public func create(_ values: CppFloatVec) -> TestClass {
    return TestClass(values)
  }
}

I'm defining CppFloatVec in the common header:

using CppFloatVec = std::vector<float>;

And I'm creating an instance of that class from C++.

void doTest() {
  auto vals = std::vector<float>{1.0f};
  auto obj = TestCommand::TestClass::init(vals);
}

I get the object created and printing vals correctly, but at the end of doTest when obj goes out of scope I get a debugger break in the std::vector destructor:
TestCommand(39267,0x1e1271ec0) malloc: *** error for object 0x600001298050: pointer being freed was not allocated

Interestingly, if I change the line to call TestCommand::TestClass::create(vals) instead of calling the initializer directly then I don't get the error.

Any idea what would be happening here? I'm putting together a separate bug report & repro steps for this one, but thought I'd mention it here first in case it seems like there's some common factor with the original issue.

efroemling · October 24, 2023, 8:43pm

Ok, second bug & from-scratch repro case is up:

github.com/apple/swift

C++ interop pointer double freed when passing std::vector<float> to Swift initializer

opened 08:37PM - 24 Oct 23 UTC

efroemling

bug triage needed

**Description** If I define a Swift class that accepts a std::vector\<float\> as the argument in its initializer, I can successfully create an instance of that class from C++, but when that object is destructed I get the following error: *** error for object 0x600001298050: pointer being freed was not allocated. **Steps to reproduce** The following steps reproduce the crash for me using XCode 15.0.1 (15A507) on an M1 Max MBP running Sonoma 14.1: - Create a new ‘Command Line Tool’ project named “TestCommand”. - Create a new C++ file named ‘test’ (with header). Click ‘Create Bridging Header’ when it asks. - In TestCommand-Bridging-Header.h, add: ```c++ #include "test.hpp" ``` - In test.hpp, add: ```c++ #include <vector> // Let Swift know about our test call and the type we're gonna send it. using CppFloatVec = std::vector<float>; void doTest(); ``` - In test.cpp, add: ```c++ #include <TestCommand-Swift.h> void doTest() { // Create an instance of our Swift TestClass // (which will then be torn down since it goes out of scope immediately). auto vals = std::vector<float>{1.0f}; auto obj = TestCommand::TestClass::init(vals); } ``` - In main.swift, add: ```swift public class TestClass { public init(_ values: CppFloatVec) { for v in values { print("GOT VALUE", v) } print("DONE PRINTING.") } static public func create(_ values: CppFloatVec) -> TestClass { return TestClass(values) } } // Call our C++ test code. doTest() ``` - Under build settings for the TestCommand target, set C++ and Objective-C Interoperability to “C++/Objective C++” - You should now be able to build and run the project, which should give the following output and then break on the debugger in std::vector's destructor due to a double free. ``` Hello, World! GOT VALUE 1.0 DONE PRINTING. TestCommand(39408,0x1e1271ec0) malloc: *** error for object 0x600002c70050: pointer being freed was not allocated TestCommand(39408,0x1e1271ec0) malloc: *** set a breakpoint in malloc_error_break to debug ``` - Interestingly, changing the line from `auto obj = TestCommand::TestClass::init(vals);` to `auto obj = TestCommand::TestClass::create(vals);` avoids the error.  **Environment** - Swift compiler version info: swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1) - Xcode version info: Xcode 15.0.1 Build version 15A507 - Deployment target: macOS Sonoma 14.1 (23B73)

Alex_L · October 27, 2023, 12:48am

Thanks, the two issues are manifestation of the same underlying problem with consumed parameters being passed from C++ to Swift, so I'll use one issue to track the fix for Swift 5.10

efroemling · November 14, 2023, 10:29pm

Thanks again for getting that addressed.

Just wanted to mention that I'm running into another C++ interop issue and wanted to ask if it looks like that same already-addressed bug:

In this case it's String values coming out of Swift instead of going into Swift and it's giving me a hang instead of a crash so I thought I'd ask if it looks like something different. Apologies if I'm just reporting the same bug again though.

Thanks again,
-Eric