Compile time tests


(José Manuel Sánchez) #1

I don’t know if this has come before (I tried searching but it’s not easy with the mailing lists system); sorry in advance if that’s the case.

I’ve been using more generics lately, and I’ve come to a point where it would be really nice to check if my types are correctly specified so that a wrong combination isn’t possible because the compiler will not allow it. The problem is, I can’t do this using unit testing, because if everything is correct, the code won’t compile, and then the tests will fail. I’m guessing there’s currently no way to do this? I did some googling and I only found something like this for boost (compile-fail):

http://www.boost.org/build/doc/html/bbv2/builtins/testing.html

Regards
José Manuel


(Slava Pestov) #2

We have an undocumented -verify flag you can use for this. Here’s some code that doesn’t type check because the second call to g() is passing a type that does not conform to protocol P:

protocol P {}
protocol Q {}

struct R : P {}
struct S : Q {}

func g<T : P>(_: T) {}

g(R())
g(S()) // expected-error{{does not conform}}

···

——

When you run this file through swiftc and pass the magic incarnation ‘-typecheck -Xfrontend -verify’, the compiler exits with a code of 0 indicating success, because a diagnostic was produced on the line with the ‘expected-error’ directive, and no other errors were produced:

Slavas-MBP:swift slava$ ../build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swiftc -typecheck -Xfrontend -verify generics.swift
Slavas-MBP:swift slava$ echo $?
0

You’ll need to come up with some way to wire the above up into your test harness, because as you correctly observed we never allow ill-typed code to *execute*.

It goes without saying that anything involving -Xfrontend is undocumented/unsupported/can change at any time. And of course the phrasing of the diagnostics can and will change too, which would also cause your test to fail. But we use this functionality extensively in our own test suite — it’s how we ensure that diagnostics don’t regress and invalid code is rejected in a clean way instead of crashing.

Slava

On Mar 31, 2017, at 12:44 AM, José Manuel Sánchez via swift-users <swift-users@swift.org> wrote:

I don’t know if this has come before (I tried searching but it’s not easy with the mailing lists system); sorry in advance if that’s the case.

I’ve been using more generics lately, and I’ve come to a point where it would be really nice to check if my types are correctly specified so that a wrong combination isn’t possible because the compiler will not allow it. The problem is, I can’t do this using unit testing, because if everything is correct, the code won’t compile, and then the tests will fail. I’m guessing there’s currently no way to do this? I did some googling and I only found something like this for boost (compile-fail):

http://www.boost.org/build/doc/html/bbv2/builtins/testing.html

Regards
José Manuel
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(José Manuel Sánchez) #3

Excellent! I hadn’t thought that you would need this to test the compiler itself. Would be nice to have some supported and documented way to do this at some point, but it works for me.

Regards
José Manuel

···

El 31 mar 2017, a las 11:41, Slava Pestov <spestov@apple.com> escribió:

We have an undocumented -verify flag you can use for this. Here’s some code that doesn’t type check because the second call to g() is passing a type that does not conform to protocol P:

protocol P {}
protocol Q {}

struct R : P {}
struct S : Q {}

func g<T : P>(_: T) {}

g(R())
g(S()) // expected-error{{does not conform}}

——

When you run this file through swiftc and pass the magic incarnation ‘-typecheck -Xfrontend -verify’, the compiler exits with a code of 0 indicating success, because a diagnostic was produced on the line with the ‘expected-error’ directive, and no other errors were produced:

Slavas-MBP:swift slava$ ../build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swiftc -typecheck -Xfrontend -verify generics.swift
Slavas-MBP:swift slava$ echo $?
0

You’ll need to come up with some way to wire the above up into your test harness, because as you correctly observed we never allow ill-typed code to *execute*.

It goes without saying that anything involving -Xfrontend is undocumented/unsupported/can change at any time. And of course the phrasing of the diagnostics can and will change too, which would also cause your test to fail. But we use this functionality extensively in our own test suite — it’s how we ensure that diagnostics don’t regress and invalid code is rejected in a clean way instead of crashing.

Slava

On Mar 31, 2017, at 12:44 AM, José Manuel Sánchez via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

I don’t know if this has come before (I tried searching but it’s not easy with the mailing lists system); sorry in advance if that’s the case.

I’ve been using more generics lately, and I’ve come to a point where it would be really nice to check if my types are correctly specified so that a wrong combination isn’t possible because the compiler will not allow it. The problem is, I can’t do this using unit testing, because if everything is correct, the code won’t compile, and then the tests will fail. I’m guessing there’s currently no way to do this? I did some googling and I only found something like this for boost (compile-fail):

http://www.boost.org/build/doc/html/bbv2/builtins/testing.html

Regards
José Manuel
_______________________________________________
swift-users mailing list
swift-users@swift.org <mailto:swift-users@swift.org>
https://lists.swift.org/mailman/listinfo/swift-users