Description:
After working with @Douglas_Gregor on SwiftLexicalLookup since GSoC 2024, the development of the library has arrived at the point where the API is stable, and it’s possible to run the validation of the current implementation for a great amount of integration tests inside the compiler using the automatic validation tool recently added to the compiler. Another milestone was running successful validation for the entire Swift Standard Library compilation process recently.
The library now closely resembles ASTScope behavior with minor deviations rarely popping out, usually due to imperfections in the validation algorithm or intended differences between SwiftLexicalLookup and ASTScope. The library should be mature enough to start implementing functionality around it and, through bug reports, community feedback, and involvement, ensure its continuous improvement.
Link to PR:
Swift Interface:
Initially, I think it would be a good idea to not expose the underlying scope structure as it still might change substantially. That’s why the PR makes public the following: ImplicitDecl, LookupName, LookInMembersScopeSyntax , and LookupResult that form the query result data structure, an extension to SyntaxProtocol that forms an entry point for the API and LookupConfig enables clients to tweak the exact behavior of the query.
/// An entity that is implicitly declared based on the syntactic structure of the program.
public enum ImplicitDecl {
/// `self` keyword representing object instance.
/// Could be associated with type declaration, extension,
/// or closure captures. Introduced at function edge.
case `self`(DeclSyntaxProtocol)
/// `Self` keyword representing object type.
/// Could be associated with type declaration or extension.
case `Self`(DeclSyntaxProtocol)
/// `error` value caught by a `catch`
/// block that does not specify a catch pattern.
case error(CatchClauseSyntax)
/// `newValue` available by default inside `set` and `willSet`.
case newValue(AccessorDeclSyntax)
/// `oldValue` available by default inside `didSet`.
case oldValue(AccessorDeclSyntax)
// ...
}
public enum LookupName {
/// Identifier associated with the name.
/// Could be an identifier of a variable, function or closure parameter and more.
case identifier(IdentifiableSyntax, accessibleAfter: AbsolutePosition?)
/// Declaration associated with the name.
/// Could be class, struct, actor, protocol, function and more.
case declaration(NamedDeclSyntax)
/// Name introduced implicitly by certain syntax nodes.
case implicit(ImplicitDecl)
/// Dollar identifier introduced by a closure without parameters.
case dollarIdentifier(ClosureExprSyntax, strRepresentation: String)
/// Represents equivalent names grouped together.
/// - Important: The array should be non-empty.
///
/// ### Example:
/// ```swift
/// switch X {
/// case .a(let x), .b(let x):
/// print(x) // <-- lookup here
/// }
/// ```
/// For lookup at the given position, the result
/// contains only one name, that represents both `let x` declarations.
case equivalentNames([LookupName])
// ...
}
/// Represents result from a specific scope.
public enum LookupResult {
/// Scope and the names that matched lookup.
case fromScope(SyntaxProtocol, withNames: [LookupName])
/// Indicates where to perform member lookup.
case lookInMembers(LookInMembersScopeSyntax)
/// Indicates to lookup generic parameters of extended type.
///
/// ### Example
/// ```swift
/// extension Foo {
/// func bar() {
/// let a = A() // <-- lookup here
/// }
/// }
/// ```
/// For a lookup started at the marked position, `lookInGenericParametersOfExtendedType`
/// will be included as one of the results prompting the client
/// to lookup the generic parameters of of the extended `Foo` type.
case lookInGenericParametersOfExtendedType(ExtensionDeclSyntax)
/// Indicates this closure expression could introduce dollar identifiers.
///
/// ### Example
/// ```swift
/// func foo() {
/// let a = {
/// $0 // <-- lookup here
/// }
/// }
/// ```
/// When looking up for any identifier at the indicated position,
/// the result will include `mightIntroduceDollarIdentifiers`
/// result kind. If it's performed for a dollar identifier, `LookupName.dollarIdentifier`
/// with the appropriate identifier will be used in the
/// result associated with the closure expression inside `a`.
case mightIntroduceDollarIdentifiers(ClosureExprSyntax)
// ...
}
extension SyntaxProtocol {
/// Returns all names that `for` refers to at this syntax node.
/// Optional configuration can be passed as `config` to customize the lookup behavior.
///
/// - Returns: An array of `LookupResult` for `identifier` at this syntax node,
/// ordered by visibility. If `identifier` is set to `nil`, returns all available names ordered by visibility.
/// The order is from the innermost to the outermost scope,
/// and within each result, names are ordered by their introduction
/// in the source code.
///
/// Example usage:
/// ```swift
/// class C {
/// var a = 42
///
/// func a(a: Int) {
/// a // <--- lookup here
///
/// let a = 0
/// }
///
/// func a() {
/// // ...
/// }
/// }
/// ```
/// When calling this function on the declaration reference `a` within its name,
/// the function returns the parameter first, then the identifier of the variable
/// declaration, followed by the first function name, and then the second function name,
/// in this exact order. The constant declaration within the function body is omitted
/// due to the ordering rules that prioritize visibility within the function body.
public func lookup(
_ identifier: Identifier?,
with config: LookupConfig = LookupConfig()
) -> [LookupResult] {
scope?.lookup(identifier, at: self.position, with: config) ?? []
}
}
public protocol LookInMembersScopeSyntax: SyntaxProtocol {
/// Position used for member lookup.
var lookupMembersPosition: AbsolutePosition { get }
}
public struct LookupConfig {
/// Specifies whether lookup should finish in the closest sequential scope.
///
/// ### Example
/// ```swift
/// class X {
/// let a = 42
///
/// func (a: Int) {
/// let a = 123
///
/// a // <-- lookup here
/// }
/// }
/// ```
/// When looking up at the specified position with `finishInSequentialScope`
/// set to `false`, lookup will return declaration from inside function body,
/// function parameter and the `a` declaration from `class X` member block.
/// If `finishInSequentialScope` would be set to `false`, the only name
/// returned by lookup would be the `a` declaration from inside function body.
public var finishInSequentialScope: Bool
public var configuredRegions: ConfiguredRegions?
/// Creates a new lookup configuration.
///
/// - `finishInSequentialScope` - specifies whether lookup should finish
/// in the closest sequential scope. `false` by default.
public init(
finishInSequentialScope: Bool = false,
configuredRegions: ConfiguredRegions? = nil
) {
self.finishInSequentialScope = finishInSequentialScope
self.configuredRegions = configuredRegions
}
}
I’d also like to start here a discussion on IdentifiableSyntax that’s part of SwiftLexicalLookup and is also included as a public protocol in the PR. Swift-syntax already has NamedDeclSyntax which can be used to refer to nominal type declarations, but I’m not aware of a counterpart for variables, function, closure, generic parameters etc. IdentifiableSyntax is that missing common protocol for those syntax nodes:
public protocol IdentifiableSyntax: SyntaxProtocol {
var identifier: TokenSyntax { get }
}
extension IdentifierPatternSyntax: IdentifiableSyntax {}
extension ClosureParameterSyntax: IdentifiableSyntax {
public var identifier: TokenSyntax {
secondName ?? firstName
}
}
extension FunctionParameterSyntax: IdentifiableSyntax {
public var identifier: TokenSyntax {
secondName ?? firstName
}
}
extension ClosureShorthandParameterSyntax: IdentifiableSyntax {
public var identifier: TokenSyntax {
name
}
}
// ...
I think this protocol, after a bit of refining/refactoring, could be worth moving to SwiftSyntax. Having a central unified protocol representation for those nodes could be valuable not only for SwiftLexicalLookup, but other clients as well.
Commonality:
The library provides a long list of potential clients, starting with the compiler, through IDEs, and even potentially developers seeking to develop new, more sophisticated macros.
Discoverability:
Any already existing syntax node can be used as an entry point for the API. The method also contains extensive documentation, making it easy to start with.
Not trivially composable:
The only counterpart for SwiftLexicalLookup is ASTScope, which is part of the compiler and not accessible to outside clients. A detached implementation of unqualified lookup would be especially beneficial once the compiler adopts it. Making it the canonical implementation would mean the query is guaranteed to be correct, and so could be all the clients using it.