Thoughts on DeclContexts

"What is a DeclContext? Can you eat it?"

Most parts of the Swift compiler AST are pretty straightforward: you've got your Decls (declarations), your Types (types), your TypeReprs (sometimes how you write the type is important), and a smattering of other helpers. But you've also got DeclContexts, which seem to show up in all sorts of places. What is a DeclContext, anyway?

When I was asked this question by a colleague, I couldn't immediately give a good answer. That's because DeclContext is serving (at least) two purposes: a "definition context" that uniques and disambiguates definitions, and a "use context" that determines what can be used from a particular point in code. The trouble is I'm not sure it's particularly good at either of them.

Note: I know "context" is mostly a meaning-free word here. I kept it in my descriptions because I couldn't think of a better word where several things (definitions or uses) could have the same "context", but they're not exactly "contained" in that context.

As a "definition context"

The number one thing that DeclContexts have is a parent pointer, which goes all the way up to the module. (A module DeclContext has no parent.) This hierachy provides uniqueness for declarations, which is used to construct mangled names. Mangled names, remember, are unique identifiers that can be used to refer to declarations in a string-ish way.

...except that DeclContexts alone don't provide enough information to make a mangled name: within a function or other local context, you can have more than one local variable named x as long as they're in different code blocks, and so there's an additional uniquing identifier on a per-DeclContext basis called the "local discriminator".

Still, DeclContexts are arranged in a lexically-nested hierarchy whose root is the containing module, and every Decl and ProtocolConformance has a DeclContext. Structurally, that's pretty much all they are, plus some identifying information.

Some DeclContexts allow you to iterate over all the Decls in them; these are the IterableDeclContexts. In practice these are struct, enum, class, and protocol declarations, plus extensions. Files are also a kind of DeclContext (FileUnit) and also allow iteration over their declarations, but don't (currently?) use the IterableDeclContext abstraction; maybe they should.

Finally, DeclContexts are where new generic parameters and requirements can be introduced.

As a "use context"

A number of APIs throughout Swift's Sema and AST libraries take a DeclContext as a token representing where something (a name, a declaration, a conformance) is being used. This is used to check a number of things:

  • what declarations are visible (usually filtered on a particular name)
  • whether or not a particular declaration can be accessed (i.e. access control checking)
  • whether a variable can be mutated (also access control checking, but also setting a let property's initial value in a constructor)

and probably more that I'm forgetting. The trouble is it's not particularly good at those either:

  • The compiler doesn't make a DeclContext for every BraceStmt inside a function, so there's additional logic to deal with local variables by source location. We're hoping to replace that with something called ASTScopes, but isn't planned to be just plain DeclContexts.

  • Access control rules used to follow the DeclContext hierarchy, but with SE-0169 that's no longer strictly the case with private. (There is a nice abstraction for that now, AccessScope, and it's still based on DeclContexts.)

  • The constructor exception for variable mutation also depends on whether the base is self, which isn't captured by the lexical use site.

  • Uses in a function's signature and a function's body aren't distinguished (they both have the function as a DeclContext), even though those two generally have very different conditions on what can and can't be used.

  • This concept doesn't extend to SIL, but we have reasons to perform similar queries in SIL, like "can I assume this code will only run on macOS 10.12 and later?".

So again, a DeclContext is used for a bunch of things, but it may not be the best tool for the job. But it's a convenient one, because all declarations have one, and various parts of the AST are kinds of DeclContext.

As a proxy for the containing file / module

Sometimes a part of the compiler just needs to know which module a declaration is in, or which source file something is being used from---say, to determine whether there's a @testable import in play. Since the DeclContext's parent chain eventually ends in a module, sometimes a DeclContext is used as a convenient proxy for either the containing file or the containing module. It's also used as a common type for "module or file", such as during the SIL passes to determine if it's sound to do whole-module optimizations.

I consider this a convenience use and not so interesting for a larger discussion.

Call to action!...?

For all their flaws, I think DeclContexts are doing okay as "definition contexts", and for their lexically-nested hierarchy. What I think I'd like to see is for their "lookup context" uses to get more specific: AccessScopes, ASTScopes, AvailabilityContexts, etc. We might be able to unify some of that into a "UseSite" concept, but then again maybe not. All of this can still be implemented using DeclContexts, by the way, like AccessScope is today, but by not using the same type we both signify that the intended use is different and give ourselves more freedom to change the behavior in the future.

10 Likes

(There is no urgency to this. I just wanted to write it up after having the same conversation twice.)

2 Likes

So this begs the question: why is BraceStmt not a DeclContext? After all, Swift already has DeclContexts that aren't Decls, so a Stmt being a DeclContext doesn't seem that weird.

I dunno. I guess it could be. Might actually slow down the compiler, though (more parent pointers to hop through), and also leads to more churn in local names when you restructure code. But it does seem like it would simplify things, so maybe it's worth measuring! cc @David_Ungar2

Nice write-up, Jordan. It would be nice to clean this area up sometime. I'm working on an implementation of scopes which is relevant to this area, as you know. It's a bit of a slog, though.

You'd also have to preserve this BraceStmt structure when serializing local types. It might not be worth the additional complexity.

In addition to BraceStmt, other statements also introduce new scopes -- IfStmt, ForEachStmt, GuardStmt and others. GuardStmt is tricky because the scope lasts until the end of the lexical block; its not nested inside the guard itself. Modeling this sort of behavior is precisely what the ASTScope data structures attempt.

1 Like