Swift `main` built for debug: LLDB `expression` crashes in Swift context

stuchlej · August 23, 2023, 3:39pm

Hello,

I'm trying to get better understanding of expression evaluation in Swift contexts. I have built the toolchain main branches in debug mode using:

$ utils/build-script --debug --skip-early-swift-driver --skip-early-swiftsyntax --debug-lldb

The issue does not manifest, in release mode:

$ utils/build-script --release-debuginfo --skip-early-swift-driver --skip-early-swiftsyntax --lldb

Any Swift expression that can not be resolved merely by an interpreter fails to compile and crashes the LLDB.

I would like to ask, whether someone else can also reproduce the issue and ask for advice.

Reproducing the issue

Build the Toolchain in debug mode.
Create a simple Swift program, for example minimal.swift:

let x = 0
print(x)

Build it $ build/Ninja-DebugAssert/swift-linux-x86_64/bin/swiftc -g minimal.swift
Run LLDB: $ build/Ninja-DebugAssert/lldb-linux-x86_64/bin/lldb minimal
Set a breakpoint (lldb) b minimal.swift:2
Run the program (lldb) r
Execute an expression that can not be resolved only by an interpreter (lldb) e -- x
The LLDB crashes.

Thread backtrace

thread backtrace
* thread #1, name = 'lldb', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
  * frame #0: 0x00007fffe3037170 libswiftCore.so`swift::TargetMetadata<swift::InProcess>::getKind(this=0x0000000000000000) const at Metadata.h:270:38
    frame #1: 0x00007fffe303f4c9 libswiftCore.so`swift::TargetMetadata<swift::InProcess>::getTypeContextDescriptor(this=0x0000000000000000) const at Metadata.h:428:13
    frame #2: 0x00007fffe303f3f9 libswiftCore.so`swift::TargetMetadata<swift::InProcess>::getGenericArgs(this=0x0000000000000000) const at Metadata.h:462:24
    frame #3: 0x00007fffe30b196e libswiftCore.so`instantiateWitnessTable(Type=0x0000000000000000, conformance=0x00007fffe31c3a48, instantiationArgs=0x0000000000000000, fullTable=0x00007fffe3479870) at Metadata.cpp:5690:29
    frame #4: 0x00007fffe30ee35c libswiftCore.so`(anonymous namespace)::WitnessTableCacheEntry::allocate(this=0x00007fffe3479858, conformance=0x00007fffe31c3a48, instantiationArgs=0x0000000000000000) at Metadata.cpp:5791:10
    frame #5: 0x00007fffe30eafbb libswiftCore.so`std::optional<swift::TargetWitnessTable<swift::InProcess>*> swift::SimpleLockingCacheEntryBase<(anonymous namespace)::WitnessTableCacheEntry, swift::TargetWitnessTable<swift::InProcess>*>::beginAllocation<swift::TargetProtocolConformanceDescriptor<swift::InProcess> const*&, void const* const*&>(this=0x00007fffe3479858, worker=0x00007fffffff8728, args=0x00007fffffff87c0, args=0x00007fffffff87b0) at MetadataCache.h:391:16
    frame #6: 0x00007fffe30a00b9 libswiftCore.so`std::pair<(anonymous namespace)::WitnessTableCacheEntry*, swift::TargetWitnessTable<swift::InProcess>*> swift::LockingConcurrentMap<(anonymous namespace)::WitnessTableCacheEntry, swift::LockingConcurrentMapStorage<(anonymous namespace)::WitnessTableCacheEntry, (unsigned short)16>>::getOrInsert<swift::TargetMetadata<swift::InProcess> const*, swift::TargetProtocolConformanceDescriptor<swift::InProcess> const*&, void const* const*&>(this=0x00007fffe3464cd0, key=0x0000000000000000, args=0x00007fffffff87c0, args=0x00007fffffff87b0) at MetadataCache.h:276:30
    frame #7: 0x00007fffe309fc64 libswiftCore.so`swift_getWitnessTable(conformance=0x00007fffe31c3a48, type=0x0000000000000000, instantiationArgs=0x0000000000000000) at Metadata.cpp:5873:23
    frame #8: 0x00007fffe64bed74 liblldb.so.13git`lazy protocol witness table accessor for type LazySequence<ReverseBasicBlockList> and conformance LazySequence<A> at <compiler-generated>:0
    frame #9: 0x00007fffe6488da1 liblldb.so.13git`Function.reversedInstructions.getter(self=0x0000555556169968) at Function.swift:56:28
    frame #10: 0x00007fffe63e1139 liblldb.so.13git`runSimplification(function=0x0000555556169968, context=Optimizer.FunctionPassContext @ 0x00007fffffff8ac8, preserveDebugInfo=true, simplify=()) at SimplificationPasses.swift:88:50
    frame #11: 0x00007fffe640e096 liblldb.so.13git`closure #1 in variable initialization expression of ononeSimplificationPass(function=0x0000555556169968, context=Optimizer.FunctionPassContext @ 0x00007fffffff8af0) at SimplificationPasses.swift:40:7
    frame #12: 0x00007fffe63dc975 liblldb.so.13git`FunctionPass.run(bridgedCtxt=OptimizerBridging.BridgedFunctionPassCtxt @ 0x00007fffffff8b70, self=Optimizer.FunctionPass @ 0x00007fffffff8b50) at Passes.swift:29:5
    frame #13: 0x00007fffe63deaae liblldb.so.13git`closure #15 in registerSwiftPasses($0=OptimizerBridging.BridgedFunctionPassCtxt @ 0x00007fffffff8bd0) at PassRegistration.swift:80:67
    frame #14: 0x00007fffe63dead9 liblldb.so.13git`@objc closure #15 in registerSwiftPasses() at <compiler-generated>:0
    frame #15: 0x00007fffe6d1f6a6 liblldb.so.13git`runBridgedFunctionPass(runFunction=0x00007ffff7d78858, passManager=0x00007fffffff9500, f=0x0000555556169958, passName=(Data = "onone-simplification", Length = 20)) at Passes.cpp:273:3
    frame #16: 0x00007fffe6d2153d liblldb.so.13git`OnoneSimplificationPass::run(this=0x000055555615b710) at Passes.def:394:1
    frame #17: 0x00007fffe6ce60bf liblldb.so.13git`swift::SILPassManager::runPassOnFunction(this=0x00007fffffff9500, TransIdx=34, F=0x0000555556169958) at PassManager.cpp:612:10
    frame #18: 0x00007fffe6ce725a liblldb.so.13git`swift::SILPassManager::runFunctionPasses(this=0x00007fffffff9500, FromTransIdx=34, ToTransIdx=36) at PassManager.cpp:732:5
    frame #19: 0x00007fffe6ceae0a liblldb.so.13git`swift::SILPassManager::execute(this=0x00007fffffff9500) at PassManager.cpp:890:5
    frame #20: 0x00007fffe6ce4a34 liblldb.so.13git`swift::SILPassManager::executePassPipelinePlan(this=0x00007fffffff9500, Plan=0x00007fffffff9bf8) at PassManager.cpp:859:5
    frame #21: 0x00007fffe6ce4870 liblldb.so.13git`swift::ExecuteSILPipelineRequest::evaluate(this=0x00007fffffff9b58, evaluator=0x0000555556652c40, desc=SILPipelineExecutionDescriptor @ 0x00007fffffff9960) const at PassManager.cpp:360:6
    frame #22: 0x00007fffe6d34507 liblldb.so.13git`std::tuple<> swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::callDerived<0ul>(this=0x00007fffffff9b58, evaluator=0x0000555556652c40, (null)=std::index_sequence<0UL> @ 0x00007fffffff99d8) const at SimpleRequest.h:267:24
    frame #23: 0x00007fffe6d3442d liblldb.so.13git`swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(request=0x00007fffffff9b58, evaluator=0x0000555556652c40) at SimpleRequest.h:290:20
    frame #24: 0x00007fffe6d149a5 liblldb.so.13git`llvm::Expected<swift::ExecuteSILPipelineRequest::OutputType> swift::Evaluator::getResultUncached<swift::ExecuteSILPipelineRequest>(this=0x0000555556652c40, request=0x00007fffffff9b58) at Evaluator.h:368:21
    frame #25: 0x00007fffe6cf7e28 liblldb.so.13git`llvm::Expected<swift::ExecuteSILPipelineRequest::OutputType> swift::Evaluator::operator()<swift::ExecuteSILPipelineRequest, (void*)0>(this=0x0000555556652c40, request=0x00007fffffff9b58) at Evaluator.h:273:12
    frame #26: 0x00007fffe6ce4acb liblldb.so.13git`swift::executePassPipelinePlan(SM=0x0000555556159bb0, plan=0x00007fffffff9bf8, isMandatory=true, IRMod=0x0000000000000000) at PassManager.cpp:370:24
    frame #27: 0x00007fffe6d19b54 liblldb.so.13git`swift::runSILDiagnosticPasses(Module=0x0000555556159bb0) at Passes.cpp:65:3
    frame #28: 0x00007fffe5d7fd8c liblldb.so.13git`lldb_private::SwiftExpressionParser::Parse(this=0x0000555556085bc0, diagnostic_manager=0x00007fffffffbda0, first_line=4, last_line=6) at SwiftExpressionParser.cpp:1943:3
    frame #29: 0x00007fffe5d75549 liblldb.so.13git`lldb_private::SwiftUserExpression::GetTextAndSetExpressionParser(this=0x00005555566b5450, diagnostic_manager=0x00007fffffffbda0, source_code=0x5555566ae3e0, exe_ctx=0x00007fffffffc2b0, exe_scope=0x00007fff6005d990) at SwiftUserExpression.cpp:642:21
    frame #30: 0x00007fffe5d767c0 liblldb.so.13git`lldb_private::SwiftUserExpression::Parse(this=0x00005555566b5450, diagnostic_manager=0x00007fffffffbda0, exe_ctx=0x00007fffffffc2b0, execution_policy=eExecutionPolicyOnlyWhenNeeded, keep_result_in_memory=true, generate_debug_info=false) at SwiftUserExpression.cpp:747:9
    frame #31: 0x00007fffe50653cc liblldb.so.13git`lldb_private::UserExpression::Evaluate(exe_ctx=0x00007fffffffc2b0, options=0x00007fffffffc6a0, expr=(Data = "x", Length = 1), prefix=(Data = "", Length = 0), result_valobj_sp=nullptr, error=0x00007fffffffc188, fixed_expression=<parent failed to evaluate: parent is NULL>, ctx_obj=0x0000000000000000) at UserExpression.cpp:285:27
    frame #32: 0x00007fffe52ae6a7 liblldb.so.13git`lldb_private::Target::EvaluateExpression(this=0x0000555555905d70, expr=(Data = "x", Length = 1), exe_scope=0x00007fff6005d990, result_valobj_sp=nullptr, options=0x00007fffffffc6a0, fixed_expression=<parent failed to evaluate: parent is NULL>, ctx_obj=0x0000000000000000) at Target.cpp:3063:25
    frame #33: 0x00007fffe5de6bb2 liblldb.so.13git`lldb_private::CommandObjectDWIMPrint::DoExecute(this=0x0000555556526190, command=(Data = "-O  -- x", Length = 8), result=0x00007fffffffcdb0) at CommandObjectDWIMPrint.cpp:178:16
    frame #34: 0x00007fffe5101d99 liblldb.so.13git`lldb_private::CommandObjectRaw::Execute(this=0x0000555556526190, args_string="-O  -- x", result=0x00007fffffffcdb0) at CommandObject.cpp:773:17
    frame #35: 0x00007fffe50dd4e1 liblldb.so.13git`lldb_private::CommandInterpreter::HandleCommand(this=0x000055555592a490, command_line="po x", lazy_add_to_history=eLazyBoolCalculate, result=0x00007fffffffcdb0, force_repeat_command=false) at CommandInterpreter.cpp:2063:14
    frame #36: 0x00007fffe50e179d liblldb.so.13git`lldb_private::CommandInterpreter::IOHandlerInputComplete(this=0x000055555592a490, io_handler=0x00005555563a9410, line="po x") at CommandInterpreter.cpp:3143:3
    frame #37: 0x00007fffe4f8491c liblldb.so.13git`lldb_private::IOHandlerEditline::Run(this=0x00005555563a9410) at IOHandler.cpp:580:22
    frame #38: 0x00007fffe4f40061 liblldb.so.13git`lldb_private::Debugger::RunIOHandlers(this=0x0000555556432a10) at Debugger.cpp:1018:16
    frame #39: 0x00007fffe50e2b2d liblldb.so.13git`lldb_private::CommandInterpreter::RunCommandInterpreter(this=0x000055555592a490, options=0x00007fffffffd168) at CommandInterpreter.cpp:3402:16
    frame #40: 0x00007fffe4cc5b08 liblldb.so.13git`lldb::SBDebugger::RunCommandInterpreter(this=0x00007fffffffd5c0, auto_handle_events=true, spawn_thread=false) at SBDebugger.cpp:1249:42
    frame #41: 0x000055555556b062 lldb`Driver::MainLoop(this=0x00007fffffffd5a0) at Driver.cpp:643:18
    frame #42: 0x000055555556b90e lldb`main(argc=1, argv=0x00007fffffffdbc8) at Driver.cpp:823:26
    frame #43: 0x00007fffe3623a90 libc.so.6`__libc_start_call_main(main=(lldb`main at Driver.cpp:754), argc=1, argv=0x00007fffffffdbc8) at libc_start_call_main.h:58:16
    frame #44: 0x00007fffe3623b49 libc.so.6`__libc_start_main_impl(main=(lldb`main at Driver.cpp:754), argc=1, argv=0x00007fffffffdbc8, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fffffffdbb8) at libc-start.c:360:3
    frame #45: 0x00005555555680f5 lldb`_start + 37

My analysis of the issue so far

I have started by looking at the point of crash: swift::TargetMetadata<swift::InProcess>::getKind(this=0x0000000000000000) where this should obviously be non-0x0. My next move was to look where the 0x0 value originates.

The closest "human readable" piece of code is a line in swift/SwiftCompilerSources/Sources/SIL/Function.swift:56 (this will be relevant later):

  public var reversedInstructions: LazySequence<FlattenSequence<LazyMapSequence<ReverseBasicBlockList, ReverseInstructionList>>>  {
    blocks.reversed().lazy.flatMap { $0.instructions.reversed() }
  }

I have decided to summarize my finding into the diagram below, please look at the image for more context. The important assembly is in black.

I have tried to replicate this issue in a minimal example and it looks like there should be a non-0 value in demangling cache variable for type metadata for LazySequence<ReverseBasicBlockList>.
The assemly looked the same (I have compared every instruction in the execution path). The only difference was, that there was a non-0 value.

Minimal example

// File: crash_dl.swift
// Build command: swiftc -emit-module -parse-as-library -g crash_dl.swift 
protocol MySeq: Sequence {}

extension LazySequence: MySeq where Base: MySeq {}

public struct TheSeq: MySeq, IteratorProtocol {
    public typealias Element = Int

    public mutating func next() -> Int? { 0 }

    public init() {}

    public var smth: [Int] {
        lazy.prefix(2).map { $0 +  1 }
    }
}

// File: exec.swift
// Build command: swiftc -I. -L. -lcrash_dl -Xlinker -rpath=. -g exec.swift
import crash_dl

print(TheSeq().smth)

At this point, I don't know where to look. I think, that the locally built release version works, because this whole "thing" may be optimized-out. I'm wondering whether this is caused by bootstrapping. I have tried to verify this by providing "hosttools" option, but the compilation failed.

Also, since I'm just getting started, I don't have a good idea about what even is a "demangling cache" and how it all works. I have watched https://www.youtube.com/watch?v=ctS8FzqcRug but it does not go into such a detail. I think the related code might be in IRGen...

stuchlej · August 23, 2023, 3:39pm

Script generating this graph (because I can't embed svg)

#!/home/mikolas/Stažené/swift-5.8.1-RELEASE-ubuntu22.04/swift-5.8.1-RELEASE-ubuntu22.04/usr/bin/swift
// By Mikolas Stuchlik, 2023

import Foundation

enum Color: String {
  case black, gray, lightgray, white
}

enum Style: String {
  case invis, solid, dashed, dotted, bold
}

enum RankDir: String {
  case topBottom = "TB"
  case leftRight = "LR"
  case bottomTop = "BT"
  case rightLeft = "RL"
}

protocol Body {
  var name: String { get }
  
  func topLevelRender(indent: String) -> String
}

struct TableNode: Body {
  let name: String
  let label: String
}

struct Cluster: Body {
  let name: String
  let rankdir: RankDir?
  let bodies: [Body]
}

enum RankType: String {
  case same, min, source, max, sink
}

struct Rank {
  let type: RankType
  let nodes: [String]
}

struct Reference {
  let node: String
  let port: String?
}

struct Edge {
  enum Direction {
    case `default`
    case w2w
    case e2w
  }

  let src: Reference
  let dest: Reference
  let label: String
  let style: Style
  let direction: Direction
}

struct NodeOffset: Hashable {
  let name: String
  let offset: Int
}

struct Decorate {
  let offsets: ClosedRange<Int>
  let backgroundColor: Color?
  let textColor: Color?
}

struct Note {
  let port: String
  let label: String
}

enum GlobalOffsetIndex {
  private static var globalOffsetIndex: [NodeOffset: String] = [:]
  @discardableResult
  static func register(node: String, line: String, port: String) -> Int? {
    let regex = #/(?:\&lt\;|\\\<|\<)\+(?<offset>[\d]+)(?:\&gt\;|\\\>|\>)/#
    guard let result = line.firstMatch(of: regex), let line = Int(result.output.offset) else {
      return nil
    }
    register(node: node, offset: line, port: port)
    return line
  }
  static func register(node: String, offset: Int, port: String) {
    globalOffsetIndex[.init(name: node, offset: offset)] = port
  }
  static func port(for name: String, offset: Int) -> String! {
    globalOffsetIndex[.init(name: name, offset: offset)]
  }
  static func reference(for name: String, offset: Int) -> Reference! {
    port(for: name, offset: offset).flatMap { .init(node: name, port: $0) }
  }

  static func dump() -> String {
    "\(globalOffsetIndex)"
  }
}

extension Edge.Direction {
  func dot(isSource: Bool) -> String {
    switch (self, isSource) {
    case (.default, _):
      return ""
    case (.w2w, _):
      return ":w"
    case (.e2w, true):
      return ":e"
    case (.e2w, false):
      return ":w"
    }
  }
}

extension Edge {
  func dot(with id: Int) -> String {
    let srcDecorators = (src.port.flatMap { ":" + $0 } ?? "") + direction.dot(isSource: true)
    let destDecorators = (dest.port.flatMap { ":" + $0 } ?? "") + direction.dot(isSource: false)
    var result = 
#"""
  "\#(src.node)"\#(srcDecorators) -> "\#(dest.node)"\#(destDecorators) [
    id = \#(id)
    style = \#(style.rawValue)
"""#

  if !label.isEmpty {
      result += 
#"""

    label = "\#(label)"
"""#
  }

  result += 
#"""

  ];
"""#
    return result
  }
}

func asm2dot(_ asm: String, name: String, linePrefix: String = "l", decorate: [Decorate]) -> TableNode {
  let escaped = asm
    .replacingOccurrences(of: "<", with: #"&lt;"#)
    .replacingOccurrences(of: ">", with: #"&gt;"#)
    .replacingOccurrences(of: " ", with: #"&nbsp;"#)

  var components = escaped.components(separatedBy: "\n")
  var lineCounter = 0
  for index in components.indices {
    let port = linePrefix + String(lineCounter)
    let line = GlobalOffsetIndex.register(node: name, line: components[index], port: port)

    var tdAttributes: [String] = ["align='left'", "port='\(port)'"]
    var fontAttributes: [String] = []
    if let line {
      for decorator in decorate where decorator.offsets.contains(line) {
        if let bgColor = decorator.backgroundColor {
          tdAttributes.append("bgcolor='\(bgColor.rawValue)'")
        }
        if let textColor = decorator.textColor {
          fontAttributes.append("color='\(textColor.rawValue)'")
        }
      }
    }

    let source = components[index]
    let fontApplied = fontAttributes.isEmpty
      ? source
      : "<font \(fontAttributes.joined(separator: " "))>\(source)</font>"
    
    let tdApplied = "<td \(tdAttributes.joined(separator: " "))>\(fontApplied)</td>"
    components[index] = 
#"""
      <tr>\#(tdApplied)</tr>
"""#
    lineCounter += 1
  }

  let dot =
#"""
    <table border='1' cellborder='0'>
\#(components.joined(separator: "\n"))
    </table>
"""#

  return .init(name: name, label: dot)
}

extension Note {
  func table(indent: String) -> String {
    let components = label.components(separatedBy: "\n").map {
#"""
\#(indent)  <tr><td align='left'>\#($0)</td></tr>
"""#
    }

    let dot =
#"""
\#(indent)<table border='0' cellborder='0'>
\#(components.joined(separator: "\n"))
\#(indent)</table>
"""#
    return dot
  }
}

func noteNode(name: String, notes: [Note]) -> TableNode {
  let cells = notes.map { note in 
#"""
      <tr><td align='left' port='\#(note.port)'>
\#(note.table(indent: "  " + "  " + "  " + "  "))
      </td></tr>
"""#
  }

  let dot =
#"""
    <table border='0' cellborder='1'>
\#(cells.joined(separator: "\n"))
    </table>
"""#

  return .init(name: name, label: dot)
}

func cluster(name: String, rankdir: RankDir?, bodies: [Body]) -> Cluster {
  assert(name.hasPrefix("cluster_"), "Clusters are required to have prefix 'cluster_' by graphviz specification.")
  return .init(name: name, rankdir: rankdir, bodies: bodies)
}


extension TableNode {
  func topLevelRender(indent: String) -> String {
    let indentLabel = label.split(separator: "\n").map { indent + $0 }.joined(separator: "\n")
    return
#"""
\#(indent)"\#(name)" [
\#(indent)  label = <
\#(indentLabel)
\#(indent)  >
\#(indent)  shape = "plaintext"
\#(indent)];
"""#
  }
}

extension Cluster {
  func topLevelRender(indent: String) -> String {
    var result = 
#"""
\#(indent)subgraph "\#(name)" {
"""#

    if let rankdir {
      result +=
#"""

\#(indent)  rankdir=\#(rankdir.rawValue);
"""#
    }

    result +=
#"""

\#(bodies.map { $0.topLevelRender(indent: indent + "  ") }.joined(separator: "\n"))
\#(indent)};
"""#

    return result
  }
}

func embed(bodies: [Body], ranks: [Rank], edges: [Edge]) -> String {
  let dotNodes = bodies.map { $0.topLevelRender(indent: "  ") }

  let dotEdges = edges.enumerated().map { $0.element.dot(with: $0.offset) }

  let dotRanks = ranks.map { rank in
#"""
  {rank = \#(rank.type.rawValue); \#(rank.nodes.map { $0 + ";"}.joined(separator: " "))}
"""#
  }

  let graph: String = 
#"""
digraph ams {
  fontname="DejaVu Sans Mono,SF Mono,Consolas"
  nodesep=1;
  node [fontname="DejaVu Sans Mono,SF Mono,Consolas"]
  edge [fontname="DejaVu Sans Mono,SF Mono,Consolas"]
\#(dotNodes.joined(separator: "\n"))
\#(dotRanks.joined(separator: "\n"))
\#(dotEdges.joined(separator: "\n"))
}
"""#
  return graph
}





let asm1: String = 
#"""
liblldb.so.13git`Function.reversedInstructions.getter:
    0x7fffe6488cf0 <+0>:   push   rbp
    0x7fffe6488cf1 <+1>:   mov    rbp, rsp
    0x7fffe6488cf4 <+4>:   push   r13
    0x7fffe6488cf6 <+6>:   sub    rsp, 0x88
    0x7fffe6488cfd <+13>:  mov    qword ptr [rbp - 0x10], 0x0
    0x7fffe6488d05 <+21>:  mov    qword ptr [rbp - 0x10], r13
    0x7fffe6488d09 <+25>:  call   0x7fffe64729e0            ; SIL.Function.blocks.getter : SIL.BasicBlockList at Function.swift:42
    0x7fffe6488d0e <+30>:  mov    rdi, rax
    0x7fffe6488d11 <+33>:  mov    qword ptr [rbp - 0x88], rdi
    0x7fffe6488d18 <+40>:  call   0x7fffe6488e00            ; SIL.BasicBlockList.reversed() -> SIL.ReverseBasicBlockList at Function.swift:422
    0x7fffe6488d1d <+45>:  mov    rdi, qword ptr [rbp - 0x88]
    0x7fffe6488d24 <+52>:  mov    qword ptr [rbp - 0x70], rax
    0x7fffe6488d28 <+56>:  call   0x7fffe4c17010            ; symbol stub for: swift_release
    0x7fffe6488d2d <+61>:  mov    rax, qword ptr [rbp - 0x70]
    0x7fffe6488d31 <+65>:  mov    qword ptr [rbp - 0x30], rax
    0x7fffe6488d35 <+69>:  mov    rdi, qword ptr [rbp - 0x30]
    0x7fffe6488d39 <+73>:  mov    qword ptr [rbp - 0x80], rdi
    0x7fffe6488d3d <+77>:  call   0x7fffe4c20e80            ; symbol stub for: swift_retain
    0x7fffe6488d42 <+82>:  mov    rax, qword ptr [rbp - 0x80]
    0x7fffe6488d46 <+86>:  mov    qword ptr [rbp - 0x40], rax
    0x7fffe6488d4a <+90>:  call   0x7fffe6459110            ; lazy protocol witness table accessor for type SIL.ReverseBasicBlockList and conformance SIL.ReverseBasicBlockList : Swift.Sequence in SIL at <compiler-generated>
    0x7fffe6488d4f <+95>:  mov    rsi, rax
    0x7fffe6488d52 <+98>:  lea    rdi, [rip + 0x10faf417]
    0x7fffe6488d59 <+105>: lea    rax, [rbp - 0x38]
    0x7fffe6488d5d <+109>: lea    r13, [rbp - 0x40]
    0x7fffe6488d61 <+113>: mov    qword ptr [rbp - 0x78], r13
    0x7fffe6488d65 <+117>: call   0x7fffe4c1e820            ; symbol stub for: Swift.Sequence.lazy.getter : Swift.LazySequence<τ_0_0>
    0x7fffe6488d6a <+122>: mov    rdi, qword ptr [rbp - 0x78]
    0x7fffe6488d6e <+126>: call   0x7fffe6459160            ; outlined destroy of SIL.ReverseBasicBlockList at <compiler-generated>
    0x7fffe6488d73 <+131>: mov    rdi, qword ptr [rbp - 0x70]
    0x7fffe6488d77 <+135>: mov    rax, qword ptr [rbp - 0x38]
    0x7fffe6488d7b <+139>: mov    qword ptr [rbp - 0x68], rax
    0x7fffe6488d7f <+143>: call   0x7fffe4c17010            ; symbol stub for: swift_release
    0x7fffe6488d84 <+148>: mov    rax, qword ptr [rbp - 0x68]
    0x7fffe6488d88 <+152>: mov    qword ptr [rbp - 0x48], rax
    0x7fffe6488d8c <+156>: lea    rdi, [rip + 0x1184dfa5]
    0x7fffe6488d93 <+163>: call   0x7fffe63b9920            ; __swift_instantiateConcreteTypeFromMangledName at <compiler-generated>
    0x7fffe6488d98 <+168>: mov    qword ptr [rbp - 0x60], rax
    0x7fffe6488d9c <+172>: call   0x7fffe64bed40            ; lazy protocol witness table accessor for type Swift.LazySequence<SIL.ReverseBasicBlockList> and conformance Swift.LazySequence<A> : Swift.LazySequenceProtocol in Swift at <compiler-generated>
    0x7fffe6488da1 <+177>: mov    qword ptr [rbp - 0x58], rax
    0x7fffe6488da5 <+181>: call   0x7fffe6454040            ; lazy protocol witness table accessor for type SIL.ReverseInstructionList and conformance SIL.ReverseInstructionList : Swift.Sequence in SIL at <compiler-generated>
    0x7fffe6488daa <+186>: mov    rdx, qword ptr [rbp - 0x60]
    0x7fffe6488dae <+190>: mov    r8, qword ptr [rbp - 0x58]
    0x7fffe6488db2 <+194>: mov    r9, rax
    0x7fffe6488db5 <+197>: lea    rdi, [rip + 0x144]        ; closure #1 (SIL.BasicBlock) -> SIL.ReverseInstructionList in SIL.Function.reversedInstructions.getter : Swift.LazySequence<Swift.FlattenSequence<Swift.LazyMapSequence<SIL.ReverseBasicBlockList, SIL.ReverseInstructionList>>> at Function.swift:56
    0x7fffe6488dbc <+204>: lea    rcx, [rip + 0x10faea9d]
    0x7fffe6488dc3 <+211>: xor    eax, eax
    0x7fffe6488dc5 <+213>: mov    esi, eax
    0x7fffe6488dc7 <+215>: lea    rax, [rbp - 0x28]
    0x7fffe6488dcb <+219>: lea    r13, [rbp - 0x48]
    0x7fffe6488dcf <+223>: mov    qword ptr [rbp - 0x50], r13
    0x7fffe6488dd3 <+227>: call   0x7fffe4c1edf0            ; symbol stub for: Swift.LazySequenceProtocol.flatMap<τ_0_0 where τ_1_0: Swift.Sequence>((τ_0_0.Element) -> τ_1_0) -> Swift.LazySequence<Swift.FlattenSequence<Swift.LazyMapSequence<τ_0_0.Elements, τ_1_0>>>
    0x7fffe6488dd8 <+232>: mov    rdi, qword ptr [rbp - 0x50]
    0x7fffe6488ddc <+236>: call   0x7fffe64bede0            ; outlined destroy of Swift.LazySequence<SIL.ReverseBasicBlockList> at <compiler-generated>
    0x7fffe6488de1 <+241>: mov    rax, qword ptr [rbp - 0x28]
    0x7fffe6488de5 <+245>: mov    rdx, qword ptr [rbp - 0x20]
    0x7fffe6488de9 <+249>: mov    rcx, qword ptr [rbp - 0x18]
    0x7fffe6488ded <+253>: add    rsp, 0x88
    0x7fffe6488df4 <+260>: pop    r13
    0x7fffe6488df6 <+262>: pop    rbp
    0x7fffe6488df7 <+263>: ret    
"""#

let asm2: String = 
#"""
liblldb.so.13git`lazy protocol witness table accessor for type LazySequence<ReverseBasicBlockList> and conformance LazySequence<A>:
    0x7fffe64bed40 <+0>:  push   rbp
    0x7fffe64bed41 <+1>:  mov    rbp, rsp
    0x7fffe64bed44 <+4>:  sub    rsp, 0x10
    0x7fffe64bed48 <+8>:  mov    rax, qword ptr [rip + 0x118a3aa9]
    0x7fffe64bed4f <+15>: cmp    rax, 0x0
    0x7fffe64bed53 <+19>: mov    qword ptr [rbp - 0x8], rax
    0x7fffe64bed57 <+23>: jne    0x7fffe64bed85            ; <+69> at <compiler-generated>
    0x7fffe64bed59 <+25>: lea    rdi, [rip + 0x11817fd8]
    0x7fffe64bed60 <+32>: call   0x7fffe6423430            ; __swift_instantiateConcreteTypeFromMangledNameAbstract at <compiler-generated>
    0x7fffe64bed65 <+37>: mov    rsi, rax
    0x7fffe64bed68 <+40>: mov    rdi, qword ptr [rip + 0x117aadd1]
    0x7fffe64bed6f <+47>: call   0x7fffe4c20010            ; symbol stub for: swift_getWitnessTable
    0x7fffe64bed74 <+52>: mov    rcx, rax
    0x7fffe64bed77 <+55>: mov    rax, rcx
    0x7fffe64bed7a <+58>: mov    qword ptr [rip + 0x118a3a77], rcx
    0x7fffe64bed81 <+65>: mov    qword ptr [rbp - 0x8], rax
    0x7fffe64bed85 <+69>: mov    rax, qword ptr [rbp - 0x8]
    0x7fffe64bed89 <+73>: add    rsp, 0x10
    0x7fffe64bed8d <+77>: pop    rbp
    0x7fffe64bed8e <+78>: ret 
"""#

let asm3: String = 
#"""
liblldb.so.13git`__swift_instantiateConcreteTypeFromMangledNameAbstract:
    0x7fffe6423430 <+0>:   push   rbp
    0x7fffe6423431 <+1>:   mov    rbp, rsp
    0x7fffe6423434 <+4>:   sub    rsp, 0x20
    0x7fffe6423438 <+8>:   mov    qword ptr [rbp - 0x18], rdi
    0x7fffe642343c <+12>:  mov    rax, qword ptr [rdi]
    0x7fffe642343f <+15>:  mov    qword ptr [rbp - 0x10], rax
    0x7fffe6423443 <+19>:  cmp    rax, 0x0
    0x7fffe6423447 <+23>:  setl   cl
    0x7fffe642344a <+26>:  test   cl, 0x1
    0x7fffe642344d <+29>:  mov    qword ptr [rbp - 0x8], rax
    0x7fffe6423451 <+33>:  jne    0x7fffe642345d            ; <+45> at <compiler-generated>
    0x7fffe6423453 <+35>:  mov    rax, qword ptr [rbp - 0x8]
    0x7fffe6423457 <+39>:  add    rsp, 0x20
    0x7fffe642345b <+43>:  pop    rbp
    0x7fffe642345c <+44>:  ret    
    0x7fffe642345d <+45>:  mov    rsi, qword ptr [rbp - 0x18]
    0x7fffe6423461 <+49>:  mov    rax, qword ptr [rbp - 0x10]
    0x7fffe6423465 <+53>:  mov    rdx, rax
    0x7fffe6423468 <+56>:  sar    rdx, 0x20
    0x7fffe642346c <+60>:  neg    rdx
    0x7fffe642346f <+63>:  cdqe   
    0x7fffe6423471 <+65>:  add    rsi, rax
    0x7fffe6423474 <+68>:  xor    eax, eax
    0x7fffe6423476 <+70>:  mov    r8d, eax
    0x7fffe6423479 <+73>:  mov    edi, 0xff
    0x7fffe642347e <+78>:  mov    rcx, r8
    0x7fffe6423481 <+81>:  call   0x7fffe4c17210            ; symbol stub for: swift_getTypeByMangledNameInContextInMetadataState2
    0x7fffe6423486 <+86>:  mov    rcx, qword ptr [rbp - 0x18]
    0x7fffe642348a <+90>:  mov    rdx, rax
    0x7fffe642348d <+93>:  mov    rax, rdx
    0x7fffe6423490 <+96>:  mov    qword ptr [rcx], rdx
    0x7fffe6423493 <+99>:  mov    qword ptr [rbp - 0x8], rax
    0x7fffe6423497 <+103>: jmp    0x7fffe6423453            ; <+35> at <compiler-generated>
"""#

let result = embed(
  bodies: [
    asm2dot(
      asm1, 
      name: "asm1",
      decorate: [
        .init(offsets: 0...168, backgroundColor: nil, textColor: .gray),
        .init(offsets: 177...263, backgroundColor: nil, textColor: .gray),
      ]
    ),
    asm2dot(
      asm2, 
      name: "asm2",
      decorate: [
        .init(offsets: 8...8, backgroundColor: .lightgray, textColor: nil),
        .init(offsets: 25...25, backgroundColor: .lightgray, textColor: nil),
        .init(offsets: 32...32, backgroundColor: .lightgray, textColor: nil),
        .init(offsets: 47...47, backgroundColor: .lightgray, textColor: nil),
        .init(offsets: 52...78, backgroundColor: nil, textColor: .gray),
      ]
    ),
    asm2dot(
      asm3, 
      name: "asm3",
      decorate: [
        .init(offsets: 12...12, backgroundColor: .lightgray, textColor: nil),
        .init(offsets: 19...19, backgroundColor: .lightgray, textColor: nil),
        .init(offsets: 33...33, backgroundColor: .lightgray, textColor: nil),
        .init(offsets: 45...103, backgroundColor: nil, textColor: .gray),
      ]
    ),
    noteNode(
      name: "asm2_notes", 
      notes: [
        .init(
          port: "n0", 
          label:
          #"""
          Should be probably labeled:
          lazy protocol witness table cache variable for type LazySequence&lt;ReverseBasicBlockList&gt; and conformance LazySequence&lt;A&gt; : Swift.Sequence in Swift
          """#
        ),
        .init(
          port: "n1", 
          label: 
          """
          Shoudl be probably labeled:
          demangling cache variable for type metadata for LazySequence&lt;ReverseBasicBlockList&gt;
          """
        ),
        .init(
          port: "n2", 
          label: 
          #"""
          <b>(lldb)</b> reg read rdi
          rdi = 0x00007ffff7cd6d38
          <b>(lldb)</b> mem read -fp -c1 0x00007ffff7cd6d38
          0x7ffff7cd6d38: 0x0000000000000000
          """#
        ),
        .init(
          port: "n3", 
          label: 
          #"""
          1 const WitnessTable * swift::swift_getWitnessTable(
          2   const ProtocolConformanceDescriptor *conformance,
          3   const Metadata *type,
          4   const void * const *instantiationArgs
          5 )

          conformance &gt;- some value from offset +40
          type &gt;- the problematic 0x0 value returned from function call
          instantiationArgs &gt;- I could not find where rdx is set so ... garbage?
          """#
        ),
      ]
    ),
    noteNode(
      name: "asm3_notes", 
      notes: [
        .init(
          port: "n0", 
          label:
          #"""
          rax = *(rdi)
          ------------
          rdi = 0x00007ffff7cd6d38
          rax = 0x0000000000000000
          """#
        ),
        .init(
          port: "n1", 
          label:
          #"""
          rax == 0x0
          ----------
          TRUE
          """#
        ),
        .init(
          port: "n2", 
          label:
          #"""
          jump not taken, rax was equal 0x0
          """#
        ),
      ]
    ),
  ], 
  ranks: [
    .init(type: .same, nodes: ["asm2", "asm2_notes"]),
    .init(type: .same, nodes: ["asm3", "asm3_notes"]),
  ],
  edges: [
    .init(
      src: GlobalOffsetIndex.reference(for: "asm1", offset: 172), 
      dest: GlobalOffsetIndex.reference(for: "asm2", offset: 0), 
      label: "",
      style: .solid,
      direction: .w2w
    ),
    .init(
      src: GlobalOffsetIndex.reference(for: "asm2", offset: 32), 
      dest: GlobalOffsetIndex.reference(for: "asm3", offset: 0), 
      label: "rdi = 0x00007ffff7cd6d38",
      style: .solid,
      direction: .w2w
    ),
    .init(
      src: GlobalOffsetIndex.reference(for: "asm2", offset: 8), 
      dest: .init(node: "asm2_notes", port: "n0"), 
      label: "", 
      style: .dotted,
      direction: .e2w
    ),
    .init(
      src: GlobalOffsetIndex.reference(for: "asm2", offset: 25), 
      dest: .init(node: "asm2_notes", port: "n1"), 
      label: "", 
      style: .dotted,
      direction: .e2w
    ),
    .init(
      src: GlobalOffsetIndex.reference(for: "asm2", offset: 32), 
      dest: .init(node: "asm2_notes", port: "n2"), 
      label: "", 
      style: .dotted,
      direction: .e2w
    ),
    .init(
      src: GlobalOffsetIndex.reference(for: "asm3", offset: 12), 
      dest: .init(node: "asm3_notes", port: "n0"), 
      label: "", 
      style: .dotted,
      direction: .e2w
    ),
    .init(
      src: GlobalOffsetIndex.reference(for: "asm3", offset: 19), 
      dest: .init(node: "asm3_notes", port: "n1"), 
      label: "", 
      style: .dotted,
      direction: .e2w
    ),
    .init(
      src: GlobalOffsetIndex.reference(for: "asm3", offset: 33), 
      dest: .init(node: "asm3_notes", port: "n2"), 
      label: "", 
      style: .dotted,
      direction: .e2w
    ),
    .init(
      src: GlobalOffsetIndex.reference(for: "asm3", offset: 44), 
      dest: GlobalOffsetIndex.reference(for: "asm2", offset: 37), 
      label: "rax = 0x0", 
      style: .dashed,
      direction: .w2w
    ),
    .init(
      src: GlobalOffsetIndex.reference(for: "asm2", offset: 47), 
      dest: .init(node: "asm2_notes", port: "n3"), 
      label: "", 
      style: .dotted,
      direction: .e2w
    ),
  ]
)

print(result)

stuchlej · August 25, 2023, 10:30pm

I have looked into the issue some more.

It appears, that the problematic memory is located in .data section of liblldb.so.13git. I have looked at the binary:

objdump --disassemble --section=.data liblldb.so.13git | ~/swift-5.8.1-RELEASE-ubuntu22.04/usr/bin/swift-demangle | grep 'demangling cache variable for type metadata for Swift\.LazySequence' -A3
0000000013ed6d38 <demangling cache variable for type metadata for Swift.LazySequence<SIL.ReverseBasicBlockList>>:
    13ed6d38:	56 29 34 fc f4 ff ff ff d6 21 34 fc ef ff ff ff     V)4......!4.....
    13ed6d48:	42 22 34 fc f4 ff ff ff                             B"4.....

Note: I find it strange, that disassemble produces 24 bytes of data for this symbol, while symbol table says, that the symbol is 8 bytes long.

objdump -tT liblldb.so.13git | grep '0000000013ed6d38' -A3 | ~/swift-5.8.1-RELEASE-ubuntu22.04/usr/bin/swift-demangle
0000000013ed6d38 l     O .data	0000000000000008              demangling cache variable for type metadata for Swift.LazySequence<SIL.ReverseBasicBlockList>

In LLDB, I have verified, that the memory is initialized as expected and then placed a read_write watchpoint.

I have found, that the 0x0 is stored into the variable in __swift_instantiateConcreteTypeFromMangledName call few lines above in the getter. I have prepared another annotated diagram.

So, the most important piece of information I've got from this, is probably the message

failed type lookup for a�Ayӿ��G: subject type x does not conform to protocol Sequence

I'm not sure where to go from here.

I would like to ask whether someone would know, what is the meaning of the arithmetic (see diagram). Is it some sort of pointer hijack protection?

Also, none of the "mangled name" I was able to see during debugging made any sense. I have tried to interpret those strings as LLVM Bitocode 6bit characters, but either I failed to align the bits correctly, or it's some other kind of encoding.