Where do keywords get tokenized

Hello, I'm new to the swift compiler codebase, and I have been walking through the lexer code lib/Lexer.cpp. I noticed that things such as numbers operators and identifiers get tokenized in the format:

...
case ch:
   lexChar(ch);
...

However, I couldn't find where exactly keywords such as var, func etc get tokenized within Lexer.cpp in the format shown above. Does this happen a method further down the line? Or are the identifier tokens transformed from their initial capture? Any help is greatly appreciated.

Keywords are returned from Lexer::lexIdentifier(). They are
initially lexed as identifiers; then they are passed to Lexer::kindOfIdentifier(), which classifies them as either keywords or ordinary identifiers. The implementation of this method is actually based on the contents of a file called TokenKinds.def, which is ultimately generated from a list defined in Python(!) code at utils/gyb_syntax_support/Token.py.

Hope this helps!

2 Likes

Thank you! That was indeed very helpful.

Terms of Service

Privacy Policy

Cookie Policy