Swift 6.1 is shipping with the ability to dump ASTs in JSON format, but there are still some known bugs around USR generation that I'm eager to fix (unlikely in 6.1 at this point, but ideally in 6.2). I'd appreciate any insight folks have on these particular issues that I've explored but hit a wall.
Context: The JSON formatted AST uses USRs exclusively to represent dumped types, because they're convenient to generate, fully represent the resolved type information, and are extremely compact. It was a non-goal to dump these as massive, deeply nested JSON objects.
Protocols with suppressed constraints
ASTMangler intentionally mangles ProtocolTypes as if they were existentials, which involves constructing an ExistentialLayout that canonicalizes the type by removing conformances to suppressable protocols. Most of the time, that's fine, but it poses a problem when dumping inheritance lists for types. ~Copyable isn't represented as its own type, but instead as Copyable with a suppressed == true flag. So the protocol actually being dumped here is just Copyable, which ASTMangler treats as any Copyable, which canonicalizes to just $sypD (Any), and we lose information.
(Inheritance from protocol compositions that contain suppressed constraints are more interesting because the suppressed == true flag isn't used there. The inheritance ~Copyable & ~Escapable is mangled as $sypRi_s_Ri0_sXPD (any Any<Self: ~Copyable, Self: ~Escapable>). Looks kind of awkward, but at least the information is there.)
Would it make sense to add a mode to ASTMangler that tells it to mangle protocol and protocol composition types precisely as they are written instead of assuming they're existentials, and use that when we generate protocol USRs in specific contexts?
USRs for types containing archetypes
ASTMangler traps if it receives a type containing a freestanding archetype, so I took the expedient route of replacing any archetypes with their interface types before generating the USRs. @hamishknight rightly points out that this isn't great and that it needs to be fixed more broadly for USR generation. For example, in this snippet:
protocol P {}
struct S: P {}
func baz<T>(_ x: T) {
func foo() -> some P { S() }
}
The result type of foo is $s3lib3bazyyxlF3fooL_QrylFQOyx_Qo_D, which demangles to <<opaque return type of foo #1 <A>() -> some in baz<A>(A) -> ()>>.0. The ... some in ... bit is weird; the protocol P is lost, although it is retrievable in from the opaque type decl's requirements later in the dump. But maybe this is a limitation of the demangler itself, because the expanded form references kind=OpaqueReturnTypeParent, index=5628794040, which can't really be used after the fact.
Parameter pack crashes
I've stumbled on some cases with parameter packs where code that otherwise compiles successfully crashes when generating USRs. For example, the following snippet crashes when generating the USR for the interface type of g:
struct Pack<each T> {
func f(_ t: repeat each T) {
repeat g(each t)
}
func g<U>(_ t: U) {}
}
Expand for crash message and stack trace
Cannot build interface type for term each τ_0_0
Invalid generic parameter: each τ_0_0
Valid generic parameters: τ_1_0
...
3. While evaluating request AbstractGenericSignatureRequest(NULL, {τ_1_0}, {Pack{repeat each τ_0_0} : Copyable, Pack{repeat each τ_0_0} : Escapable, τ_1_0 : Copyable, τ_1_0 : Escapable}, 0)
...
6 swift-frontend 0x000000010621ac48 std::__1::optional<swift::Type> llvm::function_ref<std::__1::optional<swift::Type> (swift::TypeBase*, swift::TypePosition)>::callback_fn<swift::rewriting::PropertyMap::getTypeFromSubstitutionSchema(swift::Type, llvm::ArrayRef<swift::rewriting::Term>, llvm::ArrayRef<swift::GenericTypeParamType*>, swift::rewriting::MutableTerm const&) const::$_0>(long, swift::TypeBase*, swift::TypePosition) (.cold.3) + 0
7 swift-frontend 0x0000000101f1e2a4 swift::rewriting::MutableTerm::MutableTerm(swift::rewriting::Symbol const*, swift::rewriting::Symbol const*) + 0
8 swift-frontend 0x0000000101f1d648 getTypeForSymbolRange(swift::rewriting::Symbol const*, swift::rewriting::Symbol const*, llvm::ArrayRef<swift::GenericTypeParamType*>, swift::rewriting::PropertyMap const&) + 460
Reading from stdin via: /var/folders/0v/1tmqzwl17lgcckn6pp05wftw006b5z/T/code-stdin-dXi
9 swift-frontend 0x0000000101f1d468 swift::rewriting::PropertyMap::getTypeForTerm(swift::rewriting::Term, llvm::ArrayRef<swift::GenericTypeParamType*>) const + 80
10 swift-frontend 0x0000000101f2ee30 swift::rewriting::RequirementMachine::buildRequirementsFromRules(llvm::ArrayRef<unsigned int>, llvm::ArrayRef<unsigned int>, llvm::ArrayRef<swift::GenericTypeParamType*>, bool, std::__1::vector<swift::Requirement, std::__1::allocator<swift::Requirement>>&, std::__1::vector<swift::ProtocolTypeAlias, std::__1::allocator<swift::ProtocolTypeAlias>>&) const + 1180
11 swift-frontend 0x0000000101f46b4c swift::rewriting::RequirementMachine::computeMinimalGenericSignature(bool) + 248
12 swift-frontend 0x0000000101f476dc swift::AbstractGenericSignatureRequest::evaluate(swift::Evaluator&, swift::GenericSignatureImpl const*, llvm::SmallVector<swift::GenericTypeParamType*, 2u>, llvm::SmallVector<swift::Requirement, 2u>, bool) const + 2684
13 swift-frontend 0x0000000101a2c738 llvm::PointerIntPair<swift::GenericSignature, 3u, swift::optionset::OptionSet<swift::GenericSignatureErrorFlags, unsigned int>, llvm::PointerLikeTypeTraits<swift::GenericSignature>, llvm::PointerIntPairInfo<swift::GenericSignature, 3u, llvm::PointerLikeTypeTraits<swift::GenericSignature>>> swift::SimpleRequest<swift::AbstractGenericSignatureRequest, llvm::PointerIntPair<swift::GenericSignature, 3u, swift::optionset::OptionSet<swift::GenericSignatureErrorFlags, unsigned int>, llvm::PointerLikeTypeTraits<swift::GenericSignature>, llvm::PointerIntPairInfo<swift::GenericSignature, 3u, llvm::PointerLikeTypeTraits<swift::GenericSignature>>> (swift::GenericSignatureImpl const*, llvm::SmallVector<swift::GenericTypeParamType*, 2u>, llvm::SmallVector<swift::Requirement, 2u>, bool), (swift::RequestFlags)2>::callDerived<0ul, 1ul, 2ul, 3ul>(swift::Evaluator&, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul>) const + 148
14 swift-frontend 0x0000000101e7bf50 swift::AbstractGenericSignatureRequest::OutputType swift::Evaluator::getResultUncached<swift::AbstractGenericSignatureRequest, swift::AbstractGenericSignatureRequest::OutputType swift::evaluateOrDefault<swift::AbstractGenericSignatureRequest>(swift::Evaluator&, swift::AbstractGenericSignatureRequest, swift::AbstractGenericSignatureRequest::OutputType)::'lambda'()>(swift::AbstractGenericSignatureRequest const&, swift::AbstractGenericSignatureRequest::OutputType swift::evaluateOrDefault<swift::AbstractGenericSignatureRequest>(swift::Evaluator&, swift::AbstractGenericSignatureRequest, swift::AbstractGenericSignatureRequest::OutputType)::'lambda'()) + 192
15 swift-frontend 0x0000000101e7bd78 swift::AbstractGenericSignatureRequest::OutputType swift::Evaluator::getResultCached<swift::AbstractGenericSignatureRequest, swift::AbstractGenericSignatureRequest::OutputType swift::evaluateOrDefault<swift::AbstractGenericSignatureRequest>(swift::Evaluator&, swift::AbstractGenericSignatureRequest, swift::AbstractGenericSignatureRequest::OutputType)::'lambda'(), (void*)0>(swift::AbstractGenericSignatureRequest const&, swift::AbstractGenericSignatureRequest::OutputType swift::evaluateOrDefault<swift::AbstractGenericSignatureRequest>(swift::Evaluator&, swift::AbstractGenericSignatureRequest, swift::AbstractGenericSignatureRequest::OutputType)::'lambda'()) + 548
16 swift-frontend 0x0000000101e7a44c swift::buildGenericSignature(swift::ASTContext&, swift::GenericSignature, llvm::SmallVector<swift::GenericTypeParamType*, 2u>, llvm::SmallVector<swift::Requirement, 2u>, bool) + 136
17 swift-frontend 0x0000000101fb6bb0 swift::Type::subst(swift::InFlightSubstitution&) const + 1212
18 swift-frontend 0x0000000101fb6094 swift::Type::subst(llvm::function_ref<swift::Type (swift::SubstitutableType*)>, llvm::function_ref<swift::ProtocolConformanceRef (swift::CanType, swift::Type, swift::ProtocolDecl*)>, swift::SubstOptions) const + 404
19 swift-frontend 0x0000000101c9ab04 swift::Mangle::ASTMangler::mangleTypeAsUSR(swift::Type) + 104
...
This may be related to the issue above where we replace archetypes in the type with their interface types, but I'm a bit out of my element here when it comes to packs, so I'm not sure. I'm definitely motivated to fix this one though, because having to work around compiler crashes in otherwise correct code isn't great for our semantic analysis tooling story.