Hello, I'm new to the swift compiler codebase, and I have been walking through the lexer code lib/Lexer.cpp. I noticed that things such as numbers operators and identifiers get tokenized in the format:
...
case ch:
lexChar(ch);
...
However, I couldn't find where exactly keywords such as var, func etc get tokenized within Lexer.cpp in the format shown above. Does this happen a method further down the line? Or are the identifier tokens transformed from their initial capture? Any help is greatly appreciated.
Keywords are returned from Lexer::lexIdentifier(). They are
initially lexed as identifiers; then they are passed to Lexer::kindOfIdentifier(), which classifies them as either keywords or ordinary identifiers. The implementation of this method is actually based on the contents of a file called TokenKinds.def, which is ultimately generated from a list defined in Python(!) code at utils/gyb_syntax_support/Token.py.