Swift package that uses Swift.Result in a generic function crashes when used in an Xcode project (Application or Framework) on macOS 14, iOS 17, tvOS 17 but works correctly when used from Swift Test Target or previous OS versions

I have new Swift package that worked perfectly until I upgraded to macOS 14 (or run it on any of the latest major OS releases) at which point I noticed it crashes.

Notably it only fails when called from an Xcode project (when the package is used directly in an application or embedded in a framework) and only on the latest operating systems. Originally, the tests were ran using a Swift Test Package where it doesn't crash and it also doesn't crash when running on macOS 13, iOS 16, or tvOS 16.

If I replace the use of Swift.Result with an identical Kubrick.Result the crash does not happen. In the package I've created an ExecuteResult that wraps either Swift.Result or an internal copy dependent on a build condition.

I've spent considerable time trying to build a "small" reproducer to no avail...
Here is the relevant offending function with extracted protocols but it is probably easier to look at the source in GitHub here

protocol Job<Value> {...}
protocol MapJob<SourceJob: Job, NewValue> {...}
protocol CatchJob<SourceJob: Job> {...}

extension Job {
  func mapToResult() -> some Job<Result<Value, Error>> {
    return map { .success($0) }.catch { .failure($0) }
  }

  func map<NewValue: JobValue>(_ transform: @escaping (Value) async throws -> NewValue) -> some Job<NewValue> {
    return MapJob(source: self, transform: transform)
  }

  func `catch`(handler: @escaping (Error) async throws -> Value) -> some Job<Value> {
    CatchJob<Self>(source: self, handler: handler)
  }
}

To reproduce the error you can:

  1. Clone https://github.com/outfoxx/Kubrick repo
  2. Checkout BRANCH repro/result-crash
  3. Open KubricKHost/KubrickHost.xcodeproj
  4. Select iOS Simulator with iOS 17 OS.
  5. Run MapJobTests.test_MappingValuesToResults test to see crash

@Douglas_Gregor Sorry to @ you direct but I've been searching for a solution to this problem for weeks and both posted here an added a bug to the Swift repo with no response. I've seen you weigh in on stuff like this before and I'm out of ideas at even narrowing down what the actual issue is. Do you have any ideas or suggestions or maybe you can point me to another person who can help?

One more step is needed: KubrickHostTests's location is invalid and we need to manually update it from "KubrickHostTests" to "../Tests/KubrickTests"

And in my case, it is not crashing on map but on bind.

I'm not very familiar with your codebase nor Swift concurrency implementation. Maybe @ktoso will give some insights here.

We can analyze the crash by checking the assembly code.

Add a breakpoint on the following function would not give us what we want (The problem is crashing when entering this function)

public mutating func bind<SourceJob: Job<Value>>(@JobBuilder<Value> builder: () throws -> SourceJob) rethrows {
  bind(job: try builder())
}

So let's do it by hand.

Assembly Journey

Env: Xcode 15.0 + iOS 17(21A328) Simulator

Checking the second crash stack it was jumped into at +536

We rewrite the code and run it again

init() {
    var a = self.$count
    a.bind { // Add a breakpoint here
        ThrowingJob()
          .mapToResult()
    }
}

Add a breakpoint on line 3 and then get the address for <+536> (eg. 0x10d5170d0)

0x10d5170d0 <+536>: bl     0x10d69b748               ; symbol stub for: Kubrick.JobBinding.bind<τ_0_0 where τ_0_0 == τ_1_0.Value, τ_1_0: Kubrick.Job>(builder: () throws -> τ_1_0) throws -> ()
b 0x10d69b748
continue
KubrickHostTests`Kubrick.JobBinding.bind<τ_0_0 where τ_0_0 == τ_1_0.Value, τ_1_0: Kubrick.Job>(builder: () throws -> τ_1_0) throws -> ():
->  0x10d69b748 <+0>: adrp   x16, 49
    0x10d69b74c <+4>: ldr    x16, [x16, #0xb8]
    0x10d69b750 <+8>: br     x16

Add a breakpoint on br x16

continue
reg read x16
// x16 = 0x000000010048fbf8 KubrickHost`Kubrick.JobBinding.bind<Value where Value == SourceJob.Value, SourceJob: Kubrick.Job>(builder: () throws -> SourceJob) throws -> () at JobBinding.swift:63**
b 0x000000010048fbf8
continue
KubrickHost`JobBinding.bind<τ_0_0>(builder:):
->  0x10048fbf8 <+0>:   str    x20, [sp, #-0x20]!
    0x10048fbfc <+4>:   stp    x29, x30, [sp, #0x10]
    0x10048fc00 <+8>:   add    x29, sp, #0x10
    0x10048fc04 <+12>:  sub    sp, sp, #0x70
    0x10048fc08 <+16>:  stur   x1, [x29, #-0x68]
    0x10048fc0c <+20>:  stur   x2, [x29, #-0x80]
    0x10048fc10 <+24>:  stur   x3, [x29, #-0x78]
    0x10048fc14 <+28>:  stur   x4, [x29, #-0x70]
    0x10048fc18 <+32>:  mov    x9, x20
    0x10048fc1c <+36>:  ldur   x20, [x29, #-0x68]
    0x10048fc20 <+40>:  stur   x9, [x29, #-0x60]
    0x10048fc24 <+44>:  stur   xzr, [x29, #-0x28]
    0x10048fc28 <+48>:  stur   xzr, [x29, #-0x20]
    0x10048fc2c <+52>:  stur   xzr, [x29, #-0x30]
    0x10048fc30 <+56>:  mov    x8, x3
    0x10048fc34 <+60>:  stur   x8, [x29, #-0x18]
    0x10048fc38 <+64>:  ldur   x8, [x3, #-0x8]

It will then crash on 0x10048fc38 <+64>: ldur x8, [x3, #-0x8] because x3 is 0x0 - nil.

Normally this is used to get VWT for x3's type.

After this, there will be 2 ways to keep working on the issue IMO. @Kevin_Wooten

  1. Checking where x3 is from
  2. Diff the generated assembly code running in SwiftPM or iOS 16 to see why it's not crashing.

Checking where x3 is from

    0x10b687008 <+336>: adrp   x0, 455
    0x10b68700c <+340>: add    x0, x0, #0x270            ; demangling cache variable for type metadata for <<opaque return type of static Kubrick.JobBuilder.buildBlock<τ_0_0 where τ_0_0 == τ_1_0.Value, τ_1_0: Kubrick.Job>(τ_1_0) -> some>>.0
->  0x10b687010 <+344>: bl     0x10b646db8               ; __swift_instantiateConcreteTypeFromMangledName at <compiler-generated>
    0x10b687014 <+348>: stur   x0, [x29, #-0x78]
    ...
    0x10b6870c0 <+520>: ldur   x3, [x29, #-0x78]
    ...
    0x10b6870d0 <+536>: bl     0x10b80b748               ; symbol stub for: Kubrick.JobBinding.bind<Value where Value == SourceJob.Value, SourceJob: Kubrick.Job>(builder: () throws -> SourceJob) throws -> ()

The result of 0x10b646db8 is 0 (stored on x0) which is unexpected.

Diff running on SwiftPM

Again, we need some patch here to work on your branch :sweat_smile:

  dependencies: [
     ...
    .package(url: "https://github.com/outfoxx/sunday-swift", exact: "1.0.0-beta.27"),
  ],
  targets: [
    ...
    .testTarget(
        name: "KubrickTests",
        dependencies: [
        "Kubrick",
        .product(name: "Sunday", package: "sunday-swift"),
        .product(name: "SundayServer", package: "sunday-swift")
        ]
    )
  ]

Notably it only fails when called from an Xcode project (when the package is used directly in an application or embedded in a framework) and only on the latest operating systems. Originally, the tests were ran using a Swift Test Package where it doesn't crash and it also doesn't crash when running on macOS 13, iOS 16, or tvOS 16.

And actually it also crashes on my environment. So could you double check it again and update the description if needed.

Diff running on iOS 15

  1. SwiftPM testTarget + iOS 15(19F70) Simulator

SwiftPM tells me that iOS 15's libswiftCore does not include the symbol we need.

2023-11-02 21:06:05.568836+0800 xctest[62989:13665967] [Default] 未能载入软件包“KubrickTests”。 请尝试重新安装软件包。
2023-11-02 21:06:05.569053+0800 xctest[62989:13665967] [Default] (dlopen(xx/Build/Products/Debug-iphonesimulator/KubrickTests.xctest/KubrickTests, 0x0109): Symbol not found: _swift_getExtendedExistentialTypeMetadata
  Referenced from: xx/Build/Products/Debug-iphonesimulator/KubrickTests.xctest/KubrickTests
  Expected in: /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 15.5.simruntime/Contents/Resources/RuntimeRoot/usr/lib/swift/libswiftCore.dylib)
Program ended with exit code: 83
  1. Xcodeproj + iOS 15(19F70) Simulator

The same crash when running host app

**dyld[66441]: Symbol not found: _swift_getExtendedExistentialTypeMetadata**

And infinite loading when running hostTest

iOS 15.5 ships with Swift 5.6 Runtime. And the new crash may be relevant with the following SE.

I've filed #69606 to track the new bug found here.

You may consider filing another issue for your original problem - swift_instantiateConcreteTypeFromMangledName get 0x0 result.

This might have been fixed recently:

It works, for me, on iOS 16.2 and macOS 13.4. I confess I have not tried all minor versions of the 2022 series OSs. I can go back and test that easily for iOS, macOS... not so much.

Would that also fix the swift_instantiateConcreteTypeFromMangledName?

Yes, it always crashes on bind but only if you use the mapToResult() function. That's the function that brings in Swift.Result which is required for it to crash.

The OS version does not matter here. I'm talking about it is also crashing on SwiftPM env with iOS 17. You original description state the crash condition is "Xcodeproj + iOS 17". But I do not think it has anything to do with SwiftPM or Xcodeproj

No. The bug I reported as #69606 does not related to your original issue. And I have found some clue. Let's file a new issue and discuss detail there.

The OP's original issue remains unresolved. Could you help take a look or help transfer it to the right person for follow-up?

#69615

The OS version does not matter here. I'm talking about it is also crashing on SwiftPM env with iOS 17.

I've just successfully tested it again with swift test on macOS 14, while testing using KubrickHost from the Xcode project exhibits the failure. I had to edit the swift package to add back the test target (I've checked in that change).

How are you testing it on iOS directly with SPM? I didn't even know that was possible.

It's basically the same. Just make sure you have added the testTarget in Package.swift. And then navigate to the source test file and click the button. (Remember to choose iOS Simulator as destination first)

I see. Yes I knew you could do that from Xcode. I thought you were referring to using swift test to test iOS.

In any case, here's my results:

SPM Command Line (via Xcode 15.0.1) - macOS 14.0: Succeeds
Xcode 15.0.1 / 15.1 Beta 2 - macOS 14.0 - (Package.swift): Succeeds
Xcode 15.0.1 / 15.1 Beta 2 - macOS 14.0 - (KubrickHost.xcodeproject): Fails
Xcode 15.0.1 / 15.1 Beta 2 - iOS 17.0.1 / iOS 17.2 - (Package.swift): Fails
Xcode 15.0.1 / 15.1 Beta 2 - iOS 17.0.1 / iOS 17.2 - (KubrickHost.xcodeproject): Fails

1 Like

Continuing on...

Xcode 15.0.1 / 15.1 Beta 2 - iOS 16.4 - (Package.swift): Succeeds
Xcode 15.0.1 / 15.1 Beta 2 - iOS 16.4 - (KubrickHost.xcodeproj): Succeeds

1 Like

This is strange. Anyway could you also add such info to the related GitHub issue? Thanks.

@Kyle-Ye I will. Thanks for your help and time with this!

1 Like