C++ lambda captures, references, and lifetimes question

Development Compiler
1

I recently had to debug a strange issue with references while working on placeholder types, and I was hoping that someone would be able to shed a bit more light on what was going wrong so that I can deepen my understanding. The high-level background info is as follows:

  • When doing type resolution, we pass in a handler function to help resolve placeholder types.
    • If we're in a context where we can generate type variables, we pass in a function that generates a type variable for each placeholder type.
    • Otherwise, we resolve to a PlaceholderType that can be turned into a type variable later.
    • (Also, some contexts outright reject placeholder types).

On to specifics. I believe the problem starts in PatternTypeRequest::evaluate, where we make a call to TypeResolution::forContextual, possibly passing in a hanlder which generates PlaceholderTypes whenever we encounter a PlaceholderTypeRepr (edited for brevity, points of interest marked):

Type PatternTypeRequest::evaluate(Evaluator &evaluator,
                                  ContextualPattern pattern) const {
  [...]
  auto &Context = dc->getASTContext(); // <- (1)
  switch (P->getKind()) {
  [...]
  case PatternKind::Typed: {
    OpenUnboundGenericTypeFn unboundTyOpener = nullptr;
    HandlePlaceholderTypeReprFn placeholderHandler = nullptr;
    if (pattern.allowsInference()) {
      unboundTyOpener = [](auto unboundTy) {
         return unboundTy;
       };
       placeholderHandler = [&](auto placeholderRepr) {
         return PlaceholderType::get(Context, placeholderRepr); // <- (3)
       };
     }
     return validateTypedPattern( // <- (2)
         cast<TypedPattern>(P),
         TypeResolution::forContextual(dc, options, unboundTyOpener,
                                       placeholderHandler));
  }

Once type resolution actually hits a PlaceholderTypeRepr, we call through to PlaceholderType::get:

Type PlaceholderType::get(ASTContext &ctx, Originator originator) {
  assert(originator);
  return new (ctx, AllocationArena::Permanent) // <- (4)
      PlaceholderType(ctx, originator, RecursiveTypeProperties::HasPlaceholder);
}

Which calls into ASTContext::Allocate:

void *Allocate(unsigned long bytes, unsigned alignment,
               AllocationArena arena = AllocationArena::Permanent) const {
  if (bytes == 0)
    return nullptr;

  if (LangOpts.UseMalloc) // <- Segfault here (5)
    return AlignedAlloc(bytes, alignment);

  if (arena == AllocationArena::Permanent && Stats)
    Stats->getFrontendCounters().NumASTBytesAllocated += bytes;
  return getAllocator(arena).Allocate(bytes, alignment);
}

As noted, we get a segfault at (5) above. From my investigation I found that:

  • lldb was completely unable to inspect this from within the ASTContext::Allocate frame.
  • I was getting different values for &ctx in PlaceholderType::get at (4) and &Context in PatternTypeRequest::evaluate at (2).

Given that ctx should come directly from Context via the PlaceholderType::get call at (3), I figured that something was getting mucked up with the reference to Context captured by the placeholderHandler lambda, so I refactored things to pass the ASTContext down the call stack here rather than capture it from the surrounding context.

This seems to have resolved the issue, so something is clearly off about my mental model for C++ references, C++ lifetimes, C++ lambda captures, llvm::function_ref, or all of the above. :sweat_smile: My reasoning for why the previous logic should have been okay was roughly the following:

  • Context in PatternTypeRequest::evaluate is declared at the top level of the function, so it should be valid until after we return from the function.
  • placeholderHandler's capture-by-reference default should result in Context referring to the same ASTContext from within the lambda as outside the lambda.

It's also worth noting that this issue only cropped up in release builds, which made it even more fun to debug. :grinning_face_with_smiling_eyes:

If anyone could help be correct my mental model I would be immensely grateful!

ETA: references I've found online to similar situations (e.g., this StackOverflow question), seem almost relevant, except that in both that question and the linked potential duplicate, the lambda which captures the reference-by-reference very obviously outlives the local reference it captures. I don't think this is the case here, unless, of course, I'm misunderstanding something about C++ lifetimes.

ETA: If anyone is interested in reproducing, you can check out this commit, build a release build (with ninja) and then run the test/Sema/placeholder_type.swift test.

1 Like