Conditional Diagnostic Verification

Hey everyone!

Previously when using the diagnostic verifier to test Swift code, one was limited to the same set of expected-* lines in each file even if one had multiple RUN lines:

func test() {
   // All RUN lines that use -verify need to ensure this warning occurs.
   let y = ... // expected-warning {{unused variable 'y'}}
}

This prevented one from testing different modes in the same file with multiple run lines. So for instance, given the following example:

// RUN: %target-swift-frontend %s -verify -emit-sil -o /dev/null
// RUN: %target-swift-frontend %s -verify -DFIRST -emit-sil -o /dev/null

func test() {
  let x = 5 // expected-warning {{}}

#if FIRST
  let y = 5 // expected-warning {{}}
#endif
}

there was no way that one could pattern match something within FIRST without also causing the first RUN line to fail. This would result in one needing to duplicate the test file or use FileCheck both of which are unfortunate.

In this commit, I fixed this issue by teaching the Diagnostic Verifier how to handle conditional diagnostic verification. The way that this works is that:

  1. All invocations with -verify will always check expected-{error,warning,note,remark} checks. These are checks that must pass for all runs of the diagnostic verifier on the file.

  2. One can pass an additional parameter to swift-frontend called -verify-additional-prefix $PREFIX that causes the diagnostic verifier to also check verifier strings of the form expected-$PREFIX{error, warning,node,remark}.

So for example, given the above example, I could conditionalize the diagnostic for y by changing the test as follows:

// RUN: %target-swift-frontend %s -verify -emit-sil -o /dev/null
// RUN: %target-swift-frontend %s -verify -verify-additional-prefix first- -DFIRST -emit-sil -o /dev/null

func test() {
  let x = 5 // expected-warning {{}}

#if FIRST
  let y = 5 // expected-first-warning {{}}
#endif
}

Now the test will compile successfully in all modes since when compiling without the -verify-additonal-prefix option, the diagnostic verifier ignores the expected check for y.

One can also add multiple such prefixes and mix/match the prefixes to create more complex pattern matching. For instance in the following test, we have four RUN lines where the first just checks expected-, the second checks expected-/expected-first-, the third checks expected-/expected-second-, and the fourth checks all three of expected-/expected-first-/expected-second-:

// RUN: %target-swift-frontend %s -verify -emit-sil -o /dev/null
// RUN: %target-swift-frontend %s -verify \
// RUN:    -verify-additional-prefix first- -DFIRST -emit-sil -o /dev/null
// RUN: %target-swift-frontend %s -verify -verify-additional-prefix second- \
// RUN:    -DSECOND -emit-sil -o /dev/null
// RUN: %target-swift-frontend %s -verify \
// RUN:    -verify-additional-prefix first- -verify-additional-prefix second- \
// RUN:    -DFIRST -DSECOND -emit-sil -o /dev/null

func test() {
  let x = 5 // expected-warning {{}}

#if FIRST
  let y = 5 // expected-first-warning {{}}
#endif

#if SECOND
  let z = 5 // expected-second-warning {{}}
#endif
}

allowing for one to have one file that tests all of the relevant diagnostics.

3 Likes