(This post is about providing context about what I'm trying to achieve with PR #693)
The workflow of an IDE/editor interacting with SourceKit-LSP for semantic functionality, like code-completion, should be as follows:
Once the user opens a file in the IDE/editor:
The build system should send the compiler arguments for the file to SKLSP ASAP. SKLSP will use the compiler arguments to create a compiler AST and offer semantic functionality ASAP.
At the same time, SKLSP will ask the build system to do a "prepare" operation for that file, so that build artifacts & dependencies for that file can be created/updated. Once the operation completes SKLSP will react accordingly, potentially updating its state, like discarding the existing AST and creating a new more accurate one.
Note that this workflow optimizes for low latency, with eventual accuracy, than high latency. Depending on the state of the project (is it built before, what is the file depending on, etc.) step #1 may offer satisfactory functionality for the user without needing to wait for #2. Even if #1 has missing pieces, #2 will bring the functionality up to high accuracy.
PR #693 fits within this workflow like this: once #2 completes I would like a way for SKLSP to know whether any of the build artifacts & dependencies have been updated since step #1, so that SKLSP can discard its current state and AST and create a new one, or whether the build artifacts & dependencies are still up-to-date for #1, meaning SKLSP does not need to unnecessarily discard its state and AST.
To determine this it is not precise for the build system to just report whether it did something when running #2 operation, because:
If a swift source file has been touched, swift compiler will be invoked as part of #2 to update the swift module, but if the module interface did not change then the module file won't get updated.
This would be a case where the build system did invoke a task but the dependencies were not actually updated.
#2 may get pre-empted by another "prepare" operation for a different file, cancelled, and then restarted. The second operation that pre-empted it may happen to bring the build artifacts & dependencies of the first one up-to-date because they showed up in its dependencies graph. Because of this, when the first operation restarts it will perform no task.
This would be a case where the build system did not invoke any task, but the dependencies were actually updated since #1.
To have a precise mechanism to determine the state of build artifacts & dependencies, I opened PR #693 to offer a way for an llbuild client to directly inspect the dependency nodes of the node that it built. This offers the flexibility to implement the inference for SKLSP purposes.
Thanks, that does help... purely from this description though I am still missing a high-level picture of what is going on. Is this all happening within a single build invocation, or is this looking at the build results postmortem via the database (bear in mind I haven't read the PRs yet).
To make sure it is clear, #1 (get the compiler arguments) is unrelated to an llbuild invocation. #2 (do the "prepare" build operation and inspect the dependency node results) is indeed happening within a single build invocation.
I'm not sure walking the results is the right approach here. That sounds to me like SourceKit-LSP will be duplicating some of the build state tracking, especially given the notes in the PR about timestamps, etc. In my mind, the better approach would be to have SKLSP state-update actions in the actual build graph, such that they depend directly on the results of relevant upstream tasks. Modeled this way would ensure that SKLSP would more naturally get the benefit of improvements in dependency/state tracking in the core build, such as content based hashing, etc.
To clarify, there is no build state tracking that SKLSP will be involved with, the intention is for the build system to do the "build node" operation and, as part of that one operation, report what is the timestamp of the most recent dependency node. SKLSP will not concern itself will any more built state details than this.
Yes, this is what the "build node" action is going to do. The node will represent the state-update action (if I understand correctly what you are suggesting).
I'm not sure I'm following, could you clarify how I can achieve the following:
When I say modeled as an action in the build graph, I mean something like a CustomCommand that llbuild would directly call to trigger the SKLSP update when appropriate. That way SKLSP shouldn't even have to care about the timestamp.
I think I see what you mean. Let me know if this makes sense:
There is a CustomCommand that when invoked it triggers the notification that SKLSP needs to update (a notification as part of the build invocation). In order to handle the following:
The CustomCommand appears in the build graph with a dynamic rule to check timestamps of dependency nodes. The rule is dynamic because the timestamp can change between llbuild invocations for the same build description.
Let me know if this sounds reasonable or whether you have another suggestion.
The custom command(s) should request as inputs the files/build commands that it needs. i.e. if it cares about a .c file and the command that builds that .c file, it would request those as inputs. llbuild will take care of checking the time stamps and signatures of the those and run the build action when appropriate. So yes, if they are all in the build description, a given 'prepare' operation may trigger multiple updates to happen, and subsequent such 'prepare's might be null builds.
To clarify, a null build does not mean that a notification to SKLSP is not necessary, in fact a notification may still be required even if nothing happened (as I mentioned in the "#2 may get pre-empted by another "prepare" operation..." paragraph).
In any case, I'll see if I can achieve my goals with a custom command in a way that doesn't need llbuild changes.
