[GSoC 2019] Integration of libSyntax tree with rest of complier pipeline

Development Compiler
1

Hi! I'm Abdullah, a sophomore student at National University of Science and Technology, Pakistan. I learned iOS Development in 1st grade of my college and worked on different projects in C, C++ and Java in my university years. I am one of the Apple fan and would be glad to work for swift. I have gone through all the projects and found "Integration of libSyntax tree with compiler pipeline" interesting. I have gone through it a few times but I having ambiguity in initial steps.

My understanding regarding "Derive the AST nodes from the libSyntax tree" is:

  1. I need to modify the parser to generate libSyntax tree instead of AST.
  2. Add a compiler option that provides libSyntax tree JSON file as an input for the compiler.

"Having the parser generate only a libSyntax tree":

  1. A serialised libSyntax tree JSON will be deserialized using JSON deserializer (which is already present) then It will generate a libSyntax tree which I have to implement for an AST.

It would be appreciated if anyone could help me to start the project.

2

Hi Abdullah, Thank you for being interested in this project!

Currently, the parser primarily generates AST (lib/AST), and libSyntax tree as "by-product". like:

(.swift file) --> [Parser] --(AST)------------> [Sema] -> ...
                          \   and
                           `-(libSyntax tree)-> [SourceKit, etc.]

In this project, ultimately, we want make them like:

(.swift file) -> [Parser] -(libSyntax tree)---> [Converter (new)] -(AST)-> [Sema] ...
                                           \ or
                                            `-> [SourceKit, etc.]

The parser primarily and only generates libSyntax tree, and the converter converts libSyntax tree to AST when necessary.

So you don't necessarily need to use JSON serializer/deserializer for this project. But it might be useful for debugging or testing purpose.

3

Hi @rintaro, Thanks for your reply.

So basically what you are saying is, lib/parser should be edited to generate only libSyntax tree and then a converter should convert libSytax tree to AST. Right?

4

Yes, that's right.
Also, we want to implement them incrementally, instead of prepare and merge them all at once. For example, start with type parser: parse type as libSyntax TypeSyntax node, then convert it to TypeRepr AST node. After that, proceed to Expr nodes, and so on.

5

@rintaro I'm also interested in taking up this project, and from the discussion, it seems that the only task of the project is writing a convertor. However, in the project ideas page, we have incremental reparsing as a possible future extension. Is it because we need a derivation for the program in question for incremental reparsing, which is (somewhat) provided by a libSyntax tree?

6

@harshnarang8 Sorry I couldn't get back to you sooner.

And modifying Parser. I think this is challenging and time consuming task because we cannot break the current behavior of the parser.

Yeah, incremental reparsing has already been (somewhat) implemented. At least, we have to keep this functionality in this project. I think that shouldn't be a difficult task though.

7

Hi @rintaro,

I was not able to contribute because of my mid-term exams but now I am free. I have successfully build swift compiler and now looking for contributions. As, you mentioned in the previous chat, the good point to start is StartBugs but there is no bug related to this project. I am going through the parser code and now wanted to contribute. Kindly, explain the procedure a little bit.