Demo of my Swift Interpreter that uses SwiftSyntax

Hey guys, I've been working on a Swift Interpreter for a while, and the results are starting to look good. It works by parsing the Swift code using SwiftSyntax then then traversing the AST tree. Here's a demo of it working!

Note: right now the repo is private. If you'd like to help or use it, message me and I can get you access.

29 Likes

Are you still working on this? I was curious to try it.

1 Like

I’m about to start an open source rewrite after discovering a good hack for initing generics. I’ll update the thread when it’s up.

4 Likes

Yes, I realize this is taking forever. (Who knew writing an interpreter would be hard? :wink:) Anyways, here's two updates on the rewrite.

1. My generics hack

Here's the repo. The most important feature is the ability to construct any generic you want at runtime. i.e. the following types can be constructed at runtime without being explicitly there at compile-time.

// compile time struct
struct MyStruct<T, U> {}
// ... later one at runtime the following can be constructed
MyStruct<Int, Double>
MyStruct<MyClass, SomeOtherType>

This generics hack is what is going to allow the interpreter to construct PATs, generic types, and call generic functions even when specific types of the values which the interpreter will create and pass around are not known at compile time. The demo video of the old interpreter in my OP has no real notion of types, and so it relied on type-eraser to handle generics.

2. A Swift bytecode compiler in the works

The rewrite is a bytecode interpreter, not just a simple AST-walker. Here's a screenshot of it going from source to bytecode.

There's still a lot to implement, but I will release it open source once I get it running simple Swift programs. No promises on when that will be.

11 Likes

This is honestly the coolest project.

Just a small update. I'm taking a break from this project for a little bit. I've learned a lot about compilers and interpreters in the last year though. Really fun. I'll update this thread if/when I return to the project.

7 Likes

This is a pretty cool project. Do you have any plans to release the current sources in case other contributors are able to / want to take a crack at it?

I would like to but the truth is that I’m unhappy with the current state. I’d rather the first public commit be something I’m sure others could meaningfully contribute to.

Maybe in the meantime the implementation could be discussed. There is a big design decision I’m struggling to make right now with the type system. That problem was actually part of the reason I needed to take a break.

2 Likes

Good news. I'm on track to open sourcing my rewrite later this year. Keep on eye on this thread :)

9 Likes

This is super inspiring! Hope that the project is going well.

3 Likes

@Nicholas agreed! a Swift interpreter --> Swift as a scripting language --> :star_struck:

@sudoio hope you are doing well :grinning:

1 Like

Wow, that's awesome project!

Here's a very crude version of an alternative "low-tech" approach (running "/usr/bin/swift" from within the app)
import SwiftUI

// MARK: sandbox is disabled
// tried com.apple.security.temporary-exception.files.absolute-path.read-only /usr/bin/swift
// but that didn't work for some reason

// TODO: proper error checking and remove force unwrapping

struct ContentView: View {
    @State var source = #"print("Hello, World!")"#
    @State var output = ""
    @State var outcome = ""
    @State var executing = false
    
    var body: some View {
        VStack(alignment: .leading) {
            HStack {
                Spacer()
                Button("run", action: run)
                    .disabled(executing)
                    .keyboardShortcut(.defaultAction)
            }
            TextEditor(text: $source)
                .monospaced()
                .font(.title2)
            TextEditor(text: $output)
                .disabled(true)
                .font(.title2)
                .foregroundColor(.secondary)
            Text(outcome)
                .font(.title2)
                .foregroundColor(.red)
        }
        .padding()
    }
    
    func run() {
        let fileUrl = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("foo.swift")
        let outputUrl = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("output.txt")
        try! source.data(using: .utf8)!.write(to: fileUrl)
        try! "".data(using: .utf8)!.write(to: outputUrl)
        let task = try! NSUserUnixTask(url: URL(fileURLWithPath: "/usr/bin/swift"))
        let file = try! FileHandle(forWritingTo: outputUrl)
        task.standardOutput = file
        task.standardError = file
        output = ""
        outcome = ""
        executing = true
        task.execute(withArguments: [fileUrl.path]) { error in
            let data = try! Data(contentsOf: outputUrl)
            output = String(data: data, encoding: .utf8)!
            outcome = error?.localizedDescription ?? ""
            executing = false
            try? file.close()
        }
    }
}

// smart quotes workaround
extension NSTextView {
    open override var frame: CGRect {
        didSet {
            isAutomaticQuoteSubstitutionEnabled = false
        }
    }
}

@main struct macUApp: App {
    var body: some Scene {
        WindowGroup { ContentView() }
    }
}

Obviously this depends on "/usr/bin/swift" being available on the computer. Note that this was very quickly cooked and the code is littered with unsafe optional unwrapping and lacks proper error handling.

I had to disable sandbox as I wasn't able to talk macOS into using a provided temporary exception absolute file path readonly access. With sandbox enabled the app will refuse with: "The file "swift" couldn't be opened because you don't have permission to view it."

Example output

SwiftConsole

2 Likes

Just wanted to add that this is really awesome! I've been looking for a nice embeddable language that has similar advantages to Swift. It would be awesome to see this grow!

1 Like

Hey everyone, I wanted to update this dead thread and just say that I’m still working on the Swift interpreter. It’s going very well, and I’ll be sharing results sometime this year. If anyone wants an early peek, send me a message.

6 Likes

I hope that the interpreter supports non-Apple platforms and does not require Swift toolchain installed (which includes Microsoft Build Tools on Windows).

1 Like

It’s implemented in Swift using SPM. I’ve managed to get it working in WASM so it should be possible to build it for other platforms

8 Likes

I have lots of good news and some bad news:

The good news:

  • It works on Android and Web too.
  • Advanced Swift features, like result builders, noncopyables, and param packs are working
  • Generics work (all gets rewritten to type erased boxes i.e. <T: Protocol> become any Protocol)
  • Bindings/interop with native Swift is working well
  • I almost have a demo which can render real SwiftUI via the interop/binding system. There's a few more limitations with the interop system I have right now preventing me from representing TupleView<...> which is blocking a lot of other types like VStack, ScrollView, etc.
  • Async/await is working. No real concurrency is allowed, but you can interrupt the VM to run main thread work if needed.
  • Over 1k passing e2e tests in place
  • Diagnostic e2e test are also in place. They ensure the compiler emits useful errors when encountering malformed Swift code (it used to just crash).
  • You can try an old demo here (last updated in March, based on e2e tests I had back in March. Some don't work on the web) https://sl-web-editor.vercel.app/

And the bad news: I used a lot of Claude code to push it in the last few months. So I feel uncomfortable with a lot of the code. I probably need to just sit down and reread a lot of the codebase to remove the bloat.

3 Likes

This is not bad news. Use the tools that help you get to your goals, it sounds like you are responsible and get to double-checking it, so you're good! Breathe. Great progress.

2 Likes

Don't feel bad for using the tools you needed to ship a product. Especially one many people have asked for, for a long time.

And as long as you have tests, you can refactor.

Can you share the repo?

2 Likes

Thanks for the support everyone. :heart: It's good to know some of you are excited to use it!

I'm starting the open source process of SwiftLite now!

Here's my strategy / guidelines:

  1. Publish as SPM repositories, either under my personal GitHub or on a new GitHub organization.
  2. Use a permissive license. Likely MIT.
  3. Code may be AI generated, but all commits and each LOC will be human reviewed before release.
  4. Ensure all external dependencies of SwiftLite are open sourced first.
  5. The main SwiftLite repository will be open-sourced incrementally.

Here's all the external dependencies SwiftLite currently uses. There's only one left to open source.

Repository Description How SwiftLite uses it Maintainer Open Source?
swift-syntax Apple's Swift parsing / AST models - Parsing
- AST representation
- Operator folding
- Parse-time diagnostics
- Native binding scanning
Apple :white_check_mark:
swift-collections Swift collection data structures UniqueArray and RigidArray value types used by the SwiftLiteVM Apple :white_check_mark:
NamedClosure Named closure macro and wrapper #namedClosure is used to make debugging easier in the type resolution system Joe Hinkle :white_check_mark:
SwiftTaggedPointer Packs type tags + pointer + metadata into 64-bit values - Used by the VM to represent any SLValue (SwiftLite Value) at runtime at 64 bits
- Support for non-Apple platforms is needed
- Non-Apple platforms used 128 bits to represent SLValue without tagged pointers
Joe Hinkle :white_check_mark:
swift-argument-parser CLI argument parsing Powers all CLI entry points Apple :white_check_mark:
TOMLKit TOML parser/encoder E2E test results are saved to disk as .toml files LebJe :white_check_mark:
AllocatorKit Swift manual memory allocation abstraction - Allows SwiftLite to use manual allocators
- Used by tests to measure memory usage by VM
- Used by tests to assert there's no memory leaks
Joe Hinkle :white_check_mark:
SwiftSourceSig Source file signature/lint tool - SwiftSourceSigLintBuildTool plugin verifies generated files aren't tampered with
- SwiftSourceSig module provides generated-file header string for codegen
Joe Hinkle :white_check_mark:
SwiftFileBuilder Programmatic Swift code generation DSL Used to programmatically create Swift files, such as the disassembler. Joe Hinkle -

Feel free to provide feedback for any of these repos! I should have SwiftFileBuilder up sometime this month.

If anyone feels comfortable with using low level unsafe Swift APIs, I'd definitely appreciate feedback on AllocatorKit and SwiftTaggedPointer. Both are used extensively by the VM.

I don't have a clear plan for how I will incrementally open source the main repository. The main issue is that the majority of the code was originally handwritten, but last few months were nearly all claude code driven. I need a strategy to avoid publishing unreviewed AI code. I'll share more info when I'm able to.

6 Likes