Question/Help: Guidance on debugging sourcekit

Development Source Tooling
1

I should start by saying I'm new to the swift compiler and it's related tooling. After reading the sourcekitd stress tester blog post, I decided to run it on a couple different projects. I ended up finding several errors/crashes. One of which I was able to create a very simple test case for, I filed it here.

I'd been looking to get involved for a while now, and since this seemed like a simple bug I thought I'd take a stab at it. I'm stuck now on how to actually debug this.

How do I make sk-stress-test use my build of SourceKitService? and how can I run SourceKitService under a debugger (preferably xcode) so that I can set breakpoints and trace where the error is coming from?

Maybe SourceKitService is the wrong process for me to be trying to debug, if that's the case then let me know! I found where the diagnostic I'm hitting is being emitted, so first step is I'd like to set a breakpoint there.

Thanks in advance for the guidance!

2

Thanks a lot for running the stress tester and filing that issue, Colton!

Rather than trying to debug the issue via sk-stress-test, it's simpler to use the existing SourceKit testing utilities. The main one is sourcekitd-test, which lets you make any type of sourcekitd request, but for refactoring functionality it's usually more convenient to use the swift-refactor utility. You'll find it adjacent to the swiftc binary in your build directory. You can see how it's used in the existing tests for local rename under swift/test/refactoring/rename at main · apple/swift · GitHub. For the reproducing Test.swift file you uploaded you'd run it like below to reproduce the failure:

$ /path/to/swift-build-directory/bin/swift-refactor -rename -source-filename /tmp/Test.swift -pos=11:16
<unknown>:0: error: value decl ''init()'' has no declaration location

Once you've got an invocation of swift-refactor that reproduces the issue you can edit the 'swift-refactor' scheme in the Swift Xcode project generated via utils/build-script -x to pass the same arguments. From there you can build and run that scheme to debug the issue in Xcode.

In this case the swift::ide::findLocalRenameRanges function in Refactoring.cpp is where you want to start investigating. Based on the error message it looks like we're treating a local rename initiated at the start of the DefaultConfig() call as renaming the initializer, when it should actually be a rename of the DefaultConfig type itself. We probably need to do something similar to what's happening in the CursorInfo request here: swift/SwiftSourceDocInfo.cpp at main · apple/swift · GitHub

I hope that helps and thanks again for looking into this!

1 Like
3

Thank you so much for the help!

swift-refactor worked great, I was able to reproduce the issue in xcode and even create new lit tests. Your last paragraph and link was especially helpful!

I created a PR: [refactoring] Fix error when renaming nested type at constructor - init() has no declaration location by cltnschlosser · Pull Request #26016 · apple/swift · GitHub

Going to dig into sourcekitd-test now and see if I can reproduce my next issue.

Thanks again for the great response!

4

Hey @Nathan_Hawes (or other helpful person you may be reading this),

So, I'm looking at my next issue and I'm having some trouble with sourcekitd-test. Hopefully I'm just missing something dumb.

This is my test file:

import MessageUI.MFMailComposeViewController

func test() {
  final class MailDelegate: NSObject, MFMailComposeViewControllerDelegate {
    func mailComposeController(_ controller: MFMailComposeViewController,
                               didFinishWith result: MFMailComposeResult,
                               error: Error?) {}
  }
}

Running sk-stress-test for this file results in the following:

/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2019-07-03-a.xctoolchain/usr/bin/sk-stress-test -r CursorInfo -m none /Users/cschlosser/workspace/sourcekit-errors/lib/Navigator/Test.swift swiftc -sdk /Applications/Xcode-11.Beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.0.sdk -target x86_64-apple-ios11.0-simulator /Users/cschlosser/workspace/sourcekit-errors/lib/Navigator/Test.swift

Failure detected: SourceKit returned an error response
  request: SemanticRefactoring (Local Rename) in /Users/cschlosser/workspace/sourcekit-errors/lib/Navigator/Test.swift at offset 145 with args: -sdk /Applications/Xcode-11.Beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.0.sdk -target x86_64-apple-ios11.0-simulator /Users/cschlosser/workspace/sourcekit-errors/lib/Navigator/Test.swift
  response: error response (Request Failed): <unknown>:0: error: value decl ''mailComposeController(_:didFinishWith:error:)'' has no declaration location
-- begin file content --------
<unmodified>
-- end file content ----------

I'm running sourcekitd-test from xcode with the following arguments:

-req=cursor -offset=145 -cursor-action /Users/cschlosser/workspace/sourcekit-errors/lib/Navigator/Test.swift -- -sdk /Applications/Xcode-11.Beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.0.sdk -target x86_64-apple-ios11.0-simulator /Users/cschlosser/workspace/sourcekit-errors/lib/Navigator/Test.swift

It ends up failing and exiting with this error: did not find primary SourceFile and yes, I did make sure that the file does actually exist :slight_smile:
It's also really slow, but that's probably just using a debug build.

Also in case it's helpful this the the sourcekit request that sourcekitd-test prints out:

{
  key.request: source.request.cursorinfo,
  key.compilerargs: [
    "-sdk",
    "/Applications/Xcode-11.Beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.0.sdk",
    "-target",
    "x86_64-apple-ios11.0-simulator",
    "/Users/cschlosser/workspace/sourcekit-errors/lib/Navigator/Test.swift"
  ],
  key.offset: 145,
  key.sourcefile: "/Users/cschlosser/workspace/sourcekit-errors/lib/Navigator/Test.swift",
  key.retrieve_refactor_actions: 1
}

Thanks again!

5

It worked for me (using a path on my machine). Could you try adding -req=track-compiles == before the cursor info request and see if you get any errors from that? Here's what I get from that:

{
  key.notification: source.notification.compile-will-start,
  key.filepath: "/tmp/t.swift",
  key.compileid: "1",
  key.compilerargs-string: "-sdk /Xcodes/Y/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.0.sdk -target x86_64-apple-ios11.0-simulator /tmp/t.swift"
}
{
  key.notification: source.notification.compile-did-finish,
  key.diagnostics: [
  ],
  key.compileid: "1"
}
6

I think it could actually be a misleading error message. I hit this too, but the actual problem was that I hadn't built my local Swift for iOS and this is an iOS test case.
Colton, if you didn't have --ios in your build-script invocation already, does adding it help?

Also one thing I forgot to mention about sourcekitd-test is that it talks to SourceKitService over XPC (out of process) and because of that Xcode's debugger can be a bit flaky about attaching to it. If you have trouble getting it to stop on your breakpoints, I find it's usually able to attach if I set a breakpoint on the sourcekitd-test side at the start of the skt_main function in sourcekitd-test.cpp so that it pauses there for a while before continuing execution. If that still doesn't work you can edit the sourcekitd-test scheme to set the environment variables mentioned at the bottom of swift/README.txt at main · apple/swift · GitHub so that it uses SourceKit in-process instead.

1 Like
7

I didn't have --ios, but adding it causes this error:

xcodebuild: error: The project 'Swift.xcodeproj' does not contain a target named 'swift-test-stdlib-iphonesimulator-i386'.
swift/utils/build-script: fatal error: command terminated with a non-zero exit status 65, aborting

No errors from within xcode, but it still doesn't work, it takes forever and then I get the did not find primary SourceFile.
I'm using this build command: swift/utils/build-script --debug --xcode --ios

@blangmuir I added -req=track-compiles == and it is failing to compile: failed to load module 'Swift', but I imagine that's due to the iOS issue still.

Going to try --ios-all now, since it's failing on an iOS test target :man_shrugging: --ios-all is unavailable.

8

@blangmuir @Nathan_Hawes Looks like using the --ios flag isn't supported for xcode. swift/build-presets.ini at main · apple/swift · GitHub

Any tips on how to debug this? Or do I need to try something else that doesn't depend on an iOS lib?

9

Exactly.

Ah, that's unfortunate. You could build with ninja to get support for --ios.