Is there an intermediate representation with types? Could we make one?

Development Compiler
1

I wanted to see if there was any interest in the community for an intermediate representation that included type information for expressions in a Swift program. For instance, something that would say that the 1.0 below has been typechecked as a Float:

func f(x: Float) { }

f(1.0) // Is 1.0 a Float?  A Double? Something else that's ExpressibleByFloatLiteral?

I mention this because it's a problem I still haven't quite been able to solve. I wondered if anyone else had needed this before, or if the current tools had been enough for everyone.

Some background: I just published a project called Gryphon, a Swift to Kotlin translator. When I was designing it I wanted to avoid writing a whole parser to handle the input Swift files, so instead I tried to see if the Swift compiler had any intermediate representations I could use. Something like SourceKit would have been ideal, but unfortunately it didn't have the type information I needed. I could also have gone deeper and used SIL, but that was way too low level.

In the end, I ended up having to use the AST dump, which was far from ideal. It's an output that's only made for debugging the compiler, so it's pretty unstable and could disappear at any moment. I had to add extra tests just to make sure that adding support for Swift 5.2's AST dump didn't break support for Swift 5.1's AST dump. In short, I'd much rather be using something that's officially supported if possible.

Now, this was back in 2017, so maybe now there's something else I could use that I've been overlooking. If that's the case, please let me know.

Otherwise, do you all think this is something you could use, or something the community could benefit from?

2 Likes
2

A similar question got raised on a PR review of mine, where I asked if there were any facilities available for testing how the types of different sub-expressions are inferred. There's definitely places in the tests where the output of -dump-ast is used directly to check the types of checked expressions, e.g., test/Sema/type_eraser.swift, but I'm not sure if that's the preferred way to check something like this.

3

Does SourceKit's CollectExpressionType request provide enough information for you (e.g. swift/basic.swift at main · apple/swift · GitHub)?
It gives the source range (byte offset to start and end) and type of every expression in a file, including literal expressions. A combination of https://github.com/apple/swift-syntax (which gives you a syntax tree with source locations/ranges) and SourceKit is probably the best we have at the moment.

2 Likes
4

Looks interesting. How can I test the SourceKit request?

I tried to find the sourcekitd-test executable but couldn't, do I need to build the Swift compiler for it?

I also tried to get it with sourcekitten request --yaml using a yaml file inspired by the docs, but it throws an error I can't really understand:

key.request: source.request.expression.type
key.sourcefile: "/path/to/main.swift"
key.compilerargs:
  - "/path/to/main.swift"
key.expectedtypes:
  - "CustomStringConvertible"

The operation couldn’t be completed. (SourceKittenFramework.Request.Error error 2.)
5

That error message really isn't very helpful :frowning_face:. sourcekitd-test is just a test utility that isn't distributed with the toolchain like the actual framework. I've never used SourceKitten but I'd expect its just passing the yaml through to sourcekitd. If that's the case, most of the yaml being passed to sourcekitd in its tests looks a little different though. Maybe try the below (including the braces):

{
  key.request: source.request.expression.type,
  key.compilerargs: [
    "/path/to/main.swift"
  ],
  key.sourcefile: "/path/to/main.swift"
}
6

No luck either, I got the same error. If the framework is in the toolchain, then is it easy to use? Could I create a simple program that makes the request or something?

7

If this is on macOS, you will need at least the SDK in the compiler arguments

key.compilerargs: [
  "/path/to/main.swift",
  "-sdk",
  "<SDK>"
]

Where SDK would be the result of xcrun --show-sdk-path --sdk macosx.

8

Thanks, that works! I managed to get the type list for a simple example. Unfortunately, the output for the f(1.0) example in the original post comes back empty:

{
  "key.expression_type_list" : [

  ]
}

Assuming that I'm using it right and this is it, does anyone have an idea how hard it would be to "improve" SourceKit to provide type information for every expression?

9

I think that's because f(1.0) is an error in the original example – it's missing the 'x' argument label.

10

You're right! I fixed this locally after posting, then forgot about the fix when I went to try again. Sorry about that :sweat_smile:

Thanks for the help, everybody. I'll start looking into using this combination for real then, and I'll come back if I find anything that still needs answering.

2 Likes
11

Oh, one more thing: are SourceKit and SwiftSyntax also available on Linux? I noticed the need to add a macOS SDK to the call there, which got me a little concerned.

12

Yes, they are.

1 Like