Synthesizing extensions in ASTGen to wrap other decls

I thought I'd tilt at implementing a feature I've wanted for a while—the ability to write nested types directly like this:

struct Outer {}
struct Outer.Inner {}

where the second line would be the same as writing extension Outer { struct Inner {} } today.

Ignoring the C++ parser and AST for the time being and focusing on the new parser and ASTGen, "surely this will be straightforward!" I tricked myself into believing. I could cheat by just synthesizing an ExtensionDecl and putting the nominal type decl inside it.

Unfortunately I'm hitting an assertion in my implementation:

Assertion failed: (!member->NextDecl && "Already added to a container"), function addMemberSilently, file DeclContext.cpp, line 1014.

Full backtrace
1.	Swift version 6.2-dev (LLVM 52a70e588b5dd9e, Swift c25e421fb54f448)
2.	Compiling with effective version 4.1.50
3.	While evaluating request ExtendedNominalRequest(extension of Outer)
4.	While evaluating request UnqualifiedLookupRequest(looking up 'Outer' from 0x13743ec08 FileUnit file="/Users/allevato/Developer/Swift/swift/test/decl/nested/implicit_toplevel_nesting.swift" with options { AllowProtocolMembers, TypeLookup })
5.	While evaluating request LookupInModuleRequest(0x13743ec08 FileUnit file="/Users/allevato/Developer/Swift/swift/test/decl/nested/implicit_toplevel_nesting.swift", 'Outer', UnqualifiedLookup, TypesOnly, 0x13743ec08 FileUnit file="/Users/allevato/Developer/Swift/swift/test/decl/nested/implicit_toplevel_nesting.swift", { NL_RemoveNonVisible, NL_RemoveOverridden })
 #0 0x00000001081b499c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x105b4099c)
 #1 0x00000001081b2f58 llvm::sys::RunSignalHandlers() (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x105b3ef58)
 #2 0x00000001081b4fe0 SignalHandler(int) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x105b40fe0)
 #3 0x0000000182b0a584 (/usr/lib/system/libsystem_platform.dylib+0x18047a584)
 #4 0x0000000182ad9c20 (/usr/lib/system/libsystem_pthread.dylib+0x180449c20)
 #5 0x00000001829e6a30 (/usr/lib/system/libsystem_c.dylib+0x180356a30)
 #6 0x00000001829e5d20 (/usr/lib/system/libsystem_c.dylib+0x180355d20)
 #7 0x00000001082bd6bc swift::IterableDeclContext::addMemberSilently(swift::Decl*, swift::Decl*, bool) const::$_0::operator()(swift::Decl*, swift::Decl*) const (.cold.3) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x105c496bc)
 #8 0x0000000103ef63dc swift::IterableDeclContext::addMemberSilently(swift::Decl*, swift::Decl*, bool) const (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1018823dc)
 #9 0x0000000103ef6248 swift::IterableDeclContext::addMember(swift::Decl*, swift::Decl*, bool) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101882248)
#10 0x0000000103ef5fc4 swift::IterableDeclContext::loadAllMembers() const (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101881fc4)
#11 0x0000000103ef5ed4 swift::IterableDeclContext::getMembers() const (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101881ed4)
#12 0x0000000103f723f4 void swift::SourceLookupCache::addToUnqualifiedLookupCache<llvm::ArrayRef<swift::ASTNode>>(llvm::ArrayRef<swift::ASTNode>, bool, bool) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1018fe3f4)
#13 0x0000000103f730e4 swift::SourceLookupCache::SourceLookupCache(swift::ModuleDecl const&) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1018ff0e4)
#14 0x0000000103f7528c swift::ModuleDecl::lookupValue(swift::DeclName, swift::NLKind, swift::optionset::OptionSet<swift::ModuleLookupFlags, unsigned int>, llvm::SmallVectorImpl<swift::ValueDecl*>&) const (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x10190128c)
#15 0x0000000103fa7548 (anonymous namespace)::ModuleNameLookup<(anonymous namespace)::LookupByName>::lookupInModule(llvm::SmallVectorImpl<swift::ValueDecl*>&, swift::DeclContext const*, swift::ImportPath::Access, swift::DeclContext const*, swift::NLOptions) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101933548)
#16 0x0000000103fa73f0 swift::LookupInModuleRequest::evaluate(swift::Evaluator&, swift::DeclContext const*, swift::DeclName, swift::NLKind, swift::namelookup::ResolutionKind, swift::DeclContext const*, swift::NLOptions) const (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1019333f0)
#17 0x0000000103fa847c swift::LookupInModuleRequest::OutputType swift::Evaluator::getResultUncached<swift::LookupInModuleRequest, swift::LookupInModuleRequest::OutputType swift::evaluateOrDefault<swift::LookupInModuleRequest>(swift::Evaluator&, swift::LookupInModuleRequest, swift::LookupInModuleRequest::OutputType)::'lambda'()>(swift::LookupInModuleRequest const&, swift::LookupInModuleRequest::OutputType swift::evaluateOrDefault<swift::LookupInModuleRequest>(swift::Evaluator&, swift::LookupInModuleRequest, swift::LookupInModuleRequest::OutputType)::'lambda'()) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x10193447c)
#18 0x0000000103fa7b18 swift::LookupInModuleRequest::OutputType swift::evaluateOrDefault<swift::LookupInModuleRequest>(swift::Evaluator&, swift::LookupInModuleRequest, swift::LookupInModuleRequest::OutputType) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101933b18)
#19 0x0000000103fa7a20 swift::namelookup::lookupInModule(swift::DeclContext const*, swift::DeclName, llvm::SmallVectorImpl<swift::ValueDecl*>&, swift::NLKind, swift::namelookup::ResolutionKind, swift::DeclContext const*, swift::SourceLoc, swift::NLOptions) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101933a20)
#20 0x00000001040a2ef4 swift::UnqualifiedLookupRequest::evaluate(swift::Evaluator&, swift::UnqualifiedLookupDescriptor) const (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101a2eef4)
#21 0x0000000103fc7b20 swift::SimpleRequest<swift::UnqualifiedLookupRequest, swift::LookupResult (swift::UnqualifiedLookupDescriptor), (swift::RequestFlags)33>::evaluateRequest(swift::UnqualifiedLookupRequest const&, swift::Evaluator&) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101953b20)
#22 0x000000010383224c swift::UnqualifiedLookupRequest::OutputType swift::Evaluator::getResultUncached<swift::UnqualifiedLookupRequest, swift::UnqualifiedLookupRequest::OutputType swift::evaluateOrDefault<swift::UnqualifiedLookupRequest>(swift::Evaluator&, swift::UnqualifiedLookupRequest, swift::UnqualifiedLookupRequest::OutputType)::'lambda'()>(swift::UnqualifiedLookupRequest const&, swift::UnqualifiedLookupRequest::OutputType swift::evaluateOrDefault<swift::UnqualifiedLookupRequest>(swift::Evaluator&, swift::UnqualifiedLookupRequest, swift::UnqualifiedLookupRequest::OutputType)::'lambda'()) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1011be24c)
#23 0x0000000103832110 swift::UnqualifiedLookupRequest::OutputType swift::evaluateOrDefault<swift::UnqualifiedLookupRequest>(swift::Evaluator&, swift::UnqualifiedLookupRequest, swift::UnqualifiedLookupRequest::OutputType) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1011be110)
#24 0x0000000103fb6c58 directReferencesForUnqualifiedTypeLookup(swift::DeclNameRef, swift::SourceLoc, swift::DeclContext*, (anonymous namespace)::LookupOuterResults, swift::optionset::OptionSet<DirectlyReferencedTypeLookupFlags, unsigned int>) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101942c58)
#25 0x0000000103fadb64 directReferencesForTypeRepr(swift::Evaluator&, swift::ASTContext&, swift::TypeRepr*, swift::DeclContext*, swift::optionset::OptionSet<DirectlyReferencedTypeLookupFlags, unsigned int>) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x101939b64)
#26 0x0000000103fb57a0 swift::ExtendedNominalRequest::evaluate(swift::Evaluator&, swift::ExtensionDecl*) const (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1019417a0)
#27 0x0000000103e6e180 swift::ExtendedNominalRequest::OutputType swift::Evaluator::getResultUncached<swift::ExtendedNominalRequest, swift::ExtendedNominalRequest::OutputType swift::evaluateOrDefault<swift::ExtendedNominalRequest>(swift::Evaluator&, swift::ExtendedNominalRequest, swift::ExtendedNominalRequest::OutputType)::'lambda'()>(swift::ExtendedNominalRequest const&, swift::ExtendedNominalRequest::OutputType swift::evaluateOrDefault<swift::ExtendedNominalRequest>(swift::Evaluator&, swift::ExtendedNominalRequest, swift::ExtendedNominalRequest::OutputType)::'lambda'()) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1017fa180)
#28 0x0000000103e6e078 swift::ExtendedNominalRequest::OutputType swift::Evaluator::getResultCached<swift::ExtendedNominalRequest, swift::ExtendedNominalRequest::OutputType swift::evaluateOrDefault<swift::ExtendedNominalRequest>(swift::Evaluator&, swift::ExtendedNominalRequest, swift::ExtendedNominalRequest::OutputType)::'lambda'(), (void*)0>(swift::ExtendedNominalRequest const&, swift::ExtendedNominalRequest::OutputType swift::evaluateOrDefault<swift::ExtendedNominalRequest>(swift::Evaluator&, swift::ExtendedNominalRequest, swift::ExtendedNominalRequest::OutputType)::'lambda'()) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1017fa078)
#29 0x0000000103e415c8 swift::ExtensionDecl::computeExtendedNominal() const (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x1017cd5c8)
#30 0x0000000103b911c8 swift::bindExtensions(swift::ModuleDecl&)::$_1::operator()(swift::Decl*) const (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x10151d1c8)
#31 0x0000000103b91018 swift::bindExtensions(swift::ModuleDecl&) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x10151d018)
#32 0x0000000102b7ad04 swift::CompilerInstance::performParseAndResolveImportsOnly() (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x100506d04)
#33 0x0000000102b7ad58 swift::CompilerInstance::performSema() (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x100506d58)
#34 0x00000001028eb760 withSemanticAnalysis(swift::CompilerInstance&, swift::FrontendObserver*, llvm::function_ref<bool (swift::CompilerInstance&)>, bool) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x100277760)
#35 0x00000001028e13d0 performCompile(swift::CompilerInstance&, int&, swift::FrontendObserver*) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x10026d3d0)
#36 0x00000001028e0c24 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x10026cc24)
#37 0x00000001026aabe4 swift::mainEntry(int, char const**) (/Users/allevato/Developer/Swift/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend+0x100036be4)
#38 0x000000018274f1541

I've probably done (or forgotten to do) something silly, but I'm a bit stuck. Is there somewhere obvious I should be looking?

Another thing I was unclear about is, what purpose does the Fingerprint that's associated with the parsed members serve? Since there's no ExtensionDeclSyntax in the syntax tree to generate one from, I tried both of these approaches:

  • Generating one from the nominal type decl's node, but I wasn't sure if it being the same as the struct's fingerprint would be a problem.
  • Initializing it to (0, 0), but likewise I wasn't sure if that was forbidden or not.

Oops, forgot to actually link to the implementation so far. Might be helpful.

4 Likes

Since I built the type, let me answer just this part. A Fingerprint summarizes the syntax of a declaration for the incremental build. It used to be computed as an “interface hash” of all the members of a declaration context minus their bodies with the idea being that a change in the fingerprint was a signal then that a declaration has changed. Swift, of course, is not so simple to compile incrementally and syntactic changes alone do not cut it for correctly invalidating incremental build state. Nevertheless, though we have more complex machinery at work, it is still important that the Fingerprint change when the declaration changes. At a minimum, so the incremental build knows when the module changes.

2 Likes

Looking at the code, looks like the issue is passing back an array of BridgedNominalTypeDecl? to setParsedMembers, which is expecting an array of BridgedDecl, this should fix it:

diff --git a/lib/ASTGen/Sources/ASTGen/Decls.swift b/lib/ASTGen/Sources/ASTGen/Decls.swift
index a3ab00aa713..f3697b1aa0f 100644
--- a/lib/ASTGen/Sources/ASTGen/Decls.swift
+++ b/lib/ASTGen/Sources/ASTGen/Decls.swift
@@ -232,7 +232,8 @@ extension ASTGenVisitor {
     )
 
     let members = self.withDeclContext(extensionDecl.asDeclContext) {
-      [makeNominalType()]
+      guard let nom = makeNominalType() else { return [] }
+      return [nom.asDecl]
     }
     let fp = self.generateFingerprint(declGroup: node)
     extensionDecl.setParsedMembers(

The use of the untyped BridgedArrayRef for bridging arrays is a pretty easy to misuse footgun, I'm planning on introducing a typed wrapper at some point ([DNM] Experiment: Try introduce a typed `ArrayRef` bridging wrapper by hamishknight · Pull Request #76642 · swiftlang/swift · GitHub)

1 Like

That did the trick, thank you! :tada: I panicked briefly when I still didn't see anything in -dump-ast, but then I remembered that it suppresses top-level implicit AST nodes.

This code now compiles and correctly prints main.S.C:

struct S {}
class S.C {}
func f(_ x: S.C) {
    print(String(reflecting: type(of: x)))
}
f(S.C())

(Now I guess I have to make it work in the C++ parser, too...)

4 Likes