[GSoC 2019] Code Formatting Functionality for SourceKit-LSP

You need to do git submodule update --init before the swift build command.

1 Like

There's the SourceKit-LSP but there's also the swift-format part. As Ben mentioned earlier that tool only supports reformatting the whole file, so part of the GSoC project would also be to add capability in the tool and its library API to reformat a particular range. SourceKit-LSP will use it as a library.
To get started look into getting accustomed with both SourceKit-LSP and swift-format, build and run them, and see how they work.

Thank you for your advice.
I read some documents and understood how it works.
However, I got a problem.

I downloaded swift-format and built it, but I cannot use command swift-format -v on terminal.

Do I still need to do something to run it?

Thank you.

Could you clarify what is the exact output you are seeing ?

I pulled format branch and run git submodule update --init and swift build.
Then I run the swift-format -v on the same directly.
The error message says -bash: swift-format: command not found

I'd recommend to familiarize yourself with how to use swiftpm, see this quick guide on swift.org.

Thank you very much.
Ive only used swift on Xcode, so the website is really helpful.
I can run it now.

Can I have small issues or something for the swift format?

On a side note, swift-format uses an old version of swift-syntax (Swift 4.2) so part of the job is to upgrade swift-format to the to the Swift snapshot used by the sourcekit-lsp) or wait until swift-format maintainers will do that.

Hi everyone,

I was going to start a new thread, but I noticed in a couple of posts that @akyrtzi was recommending people to post in this thread instead.

My name is Nick, and I'm an undergraduate senior at the University of Virginia's College at Wise where I'm majoring in Software Engineering.

I've always wanted to contribute to open source software, but have found that I get overwhelmed when trying to make sense of the larger projects that are out there (like Swift). I believe that working closely with an experienced mentor in a real-world project would be an amazing learning experience and would allow me to give back to a language that I fondly call my favorite.

I'm going to fork swift-format and begin looking through the code to get an idea of how it works. Does anyone have any tips or techniques that they find helpful when analyzing a project they're unfamiliar with?

  • There is some good information in the Documentation directory of this repo :smile:
  • It's often useful to look at new code with a specific goal in mind. The classic example is to choose a bug that seems straightforward and try to fix it. It looks like there aren't very many open issues on this repo, so another idea would be to choose something specific about the formatting it does, and try to figure out where that happens in the code and understand it.
  • To get a high-level overview, you could start in the main function and try to follow along at a high level with what the flow of the program is - either by reading the code or in a debugger.

@allevato any other suggestions?

Hi Nick (a fellow southwest Virginian! I did my undergrad and grad school at Virginia Tech).

For integration with SourceKit-LSP, the main entry point you're going to be interested in is the SwiftFormat module, which provides a Swift API for invoking the formatter instead of spawning an external command-line process. The README discusses that in a little bit more detail (mainly just linking to the doc comments in the source).

So, you can add a dependency to swift-format in the Package.swift for SourceKit-LSP and then import those modules wherever you need them.

Right now the API is designed around either formatting code based on a file URL, or on a syntax tree that's already in memory. That should be enough (or almost enough) to do whole-file formatting. One area that we haven't jumped into is range-based (partial file) formatting, so there's room for improvement there.

@krzyzanowskim is right that swift-format is currently targeting Swift 4.2 compilers (since we wanted it to work for folks writing shipping code with the latest GM release available right now). We'll need to migrate it to support Swift 5 (some of the syntax nodes have changed); feel free to take a stab at it if you like. The changes aren't too significant, but we'll need to host them in a different branch for the time being until the code moves to a more permanent location.

2 Likes

My name is Salman Zafar 6th semester student at Iqra University Islamabad, Pakistan (department of Computer Science)

I'm interested in SourceKit-LSP and i would like to participate.

I am using swift for over a year in iOS development and i am familiar with swift-package-manager

As @blangmuir and @akyrtzi mention that the tool swift-format format the whole file, but in gsoc we are going to format the particular line or range of text, am i right ?

I was also getting the CCommonMark error and how i solved the error

if you look into swift-format package.swift file

  targets: [
    .target(
      name: "CCommonMark",
      exclude: [
        "cmark/api_test",
        // We must exclude main.c or SwiftPM will treat this target as an executable target instead
        // of a library, and we won't be able to import it from the CommonMark Swift module.
        "cmark/src/main.c",
        "cmark/test",
      ]

They are using the cmark so i downloaded the cmark and added the downloaded file to ./swift-format/Sources/CCommonMark/cmark

Then swift build
Then swift package generate-xcodeproj to generate xcode file
Then CMD+R

and in debug folder you will find the swift-format file

./swift-format -v to display the version

or if you go to main.swift file there is a function main (at the end of the file) taking argument of string array

exit(main(["swift-format", "-v"])) this will also display the current version

CCommonMark is a C library, so it can't be added to the SPM. Instead, the devs have it listed as a submodule of the project. After cloning swift-format, you can initialize and download CCommonMark by running these two commands from the project directory:

git submodule init
git submodule update

@allevato I've woke up to snow and temperatures in the teens the last couple of mornings. Good ole southwest Virginia for you. I have a couple of friends/family going to Virginia Tech at the moment. It's a beautiful campus and Blacksburg is a great city!

I've been looking through the SwiftFormat module of swift-format, and I believe I have a decent grasp of what is going on.

Essentially, we need to overload the format method of SwiftFormatter to accept a Range parameter that would need properties for the beginning line/character and ending line/character.

The first approach that comes to mind is to grab the code from the given range, create a temporary file containing the code, parse that into a syntax tree, and then pass it to format<Output: TextOutputStream>(syntax: Syntax, assumingFileURL url: URL, to outputStream: inout Output). I was trying to come up with a way of doing it without creating a temporary file, but it seems as if most of the methods I would be calling require a URL. Am I on the right track with this line of thinking?

On another note, I was attempting to use the format-diff.sh bash script to see swift-format in action, but I'm getting error: unknown option -p; use --help to list available options when I invoke the script. The bash script is trying to pass a -p option to swift-format even though it doesn't accept one.

The SwiftSyntax folks are better equipped to provide more details here (@akyrtzi). Since swift-format uses the 4.2-compatible APIs, it works by invoking swiftc under the hood so you would need to create a temporary in that case. More recent versions have other incremental parsing functionality that might be useful here to get you a partial Syntax tree that you can pass in instead, but I'm not sure what the state of those APIs is.

Oops, thanks for pointing that out! The SwiftFormatPrettyPrint phase of the formatter was originally optional until it was fully baked, and that was enabled by passing -p, but now it's enabled by default and that flag was removed. I'll update the script.

I'll also update the README to make sure this is more obvious.

1 Like

You don't need to parse again in order to get the syntax tree of a given range, you already have this tree, it is contained within the syntax tree of the whole file.

To clarify more, if you extract the source code of the range, the given range may not result in valid source code. You should assume you have the whole tree of the whole file and then asked to only re-format a range from it.

2 Likes

i have taken the specific syntax node and pass only that specific node instead of the whole tree

public init(context: Context, node: Syntax, printTokenStream: Bool) {

but how can i get the specific node in more complex tree, it only work with this swift file

swift file

print(      "hello world"       )
print("syntax tree")

var hello =             "hello world"

Swift-Formatter

print("Enter the line: ", terminator: "")
    if let line = Int(readLine() ?? "") {
        if let node = getNode(transformedSyntax, line) {
            prettyPrint(context: context, node: node, debugOptions: debugOptions)
        }
    }

    func getNode( _ syntax: Syntax ,_ line: Int) -> Syntax? {

    guard let syntaxChild = syntax.child(at: 0) else { return nil }
    guard let child = syntaxChild.child(at: line) else { return nil }

    return child
    }

    func prettyPrint(context: Context, node: Syntax, debugOptions: DebugOptions) {

    let printer = PrettyPrinter(
        context: context,
        node: node,
        printTokenStream: debugOptions.contains(.dumpTokenStream))

        print(printer.prettyPrint())   
    }

child(at:) accepts the index for the node child to return (e.g. for a function call argument list, '2' will return the third argument), it doesn't accept source line numbers.
You would find the sub-nodes of a range by walking the tree and comparing the source positions of the nodes in relation to the given source range.

A good way to explore the SwiftSyntax tree of some source code is to run

swift -frontend -emit-syntax /path/to/file.swift | python -m json.tool

or use this SwiftSyntax AST Explorer site.

1 Like

A post was split to a new topic: Getting SwiftSyntax subtree using source range

A post was split to a new topic: Range formatting