Future of Conditional Compilation
Over the past couple of years, there have been a few attempts to make incremental improvements to conditional compilation directives, like allowing them inside array/dictionary literals or around catch clauses. The core team response to the first proposal does a good job of explaining why these improvements have never moved forward, even though there is near-universal support for the features in theory: the current implementation of #if
creates complexity throughout the compiler that scales with the number of contexts a conditional compilation directive is permitted to appear in. Having worked on the catch clause implementation myself, and taken a look at the array literal one, I think it's become clear at this point that in order to support use cases like conditionally compiled catch clauses, literals, attributes, etc. which have been requested by the community, a new approach is required. This post is intended to begin exploring what that new approach should look like.
It's useful to begin by surveying some notable approaches taken by others with regard to #if
-like features:
Rust
The Rust docs contain a short, but very helpful chapter on conditional compilation. In Rust, the primary mechanism to conditionally compile code is through the cfg
attribute. Translated to Swift, this might look something like:
@if(os: Linux) struct OnlyCompiledOnLinux {}
However, Rust also allows the use of attributes to annotate arbitrary code blocks, statements, parameters, etc. which makes the feature much more flexible. The language also provides a cfg!
macro which is expanded by the compiler to true
or false
and can be used in an arbitrary expression context. Rust, like Swift, requires that excluded code is parseable.
I think this design is probably not the right one for Swift, even ignoring the source compatibility implications. It's unclear whether Swift should support attributes on code blocks, statements, etc., and applied to our current modeling of attributes, probably wouldn't solve the problem of AST complexity. I do think it illustrates an interesting point though, that it's valuable to consider conditional compilation of declarations and statements as a closely related but separate problem to intra-declaration(e.g. attributes), intra-statement(e.g. switch and do-catch), and intra-expression (e.g. array/dictionary literals) conditionals. I'd argue that #if
as it stands today supports the former cases very well, and it's the latter cases we should direct our focus to.
C/C++/C#
C -family languages rely on the preprocessor for conditional compilation, so the compiler never has to worry about representing compiled-out blocks in the AST. This allows for the use of #ifdef
et. al. pretty much anywhere (even where it can result in very confusing code). C does not require that excluded code is parseable.
It's been suggested in the past that Swift could use some kind of "integrated preprocessing" approach in between the lexing and parsing stages. This would allow conditional inclusion of individual tokens. Integrating this step into the compiler is also key: it would avoid some problems C faces due to the fact that the C preprocessor and compiler tokenize code slightly differently. This is similar to the approach taken by C#
. Downsides if Swift were to take this approach include:
- Compiled-out code would no longer be parsed. This would make cross-platform code somewhat harder to maintain, and would hurt the quality of tools like formatters that rely on the syntax tree.
- There are some open questions as to how such a change would interact with module interface printing.
Other possible techniques
While researching this topic I came across a few papers (Refactoring C with conditional compilation | IEEE Conference Publication | IEEE Xplore, https://cs.nyu.edu/rgrimm/papers/pldi12.pdf) on refactoring C in the presence of #ifdef blocks which contain some interesting techniques that might apply to this problem. Most operate by hoisting conditional compilation directives. For example, the following program:
let arr = [
1,
#if SOMETHING
2,
#else
3,
#endif
4
]
Would be transformed into this program:
#if SOMETHING
let arr = [
1,
2,
4
]
#else
let arr = [
1,
3,
4
]
#endif
Such a preprocessing step transforms a program which uses the C/C++ conditional compilation model into one which is supported by the current Swift model by hoisting conditional compilation directives to the innermost point at which they can be parsed by the current grammar. This means they could still be checked by the parser/used by tooling. The downside to this approach of course, is that actually implementing such a program transformation is very difficult, and there's not a lot of prior art to draw on. Many possible approaches involve "forking" the parser state, which would also have a nontrivial performance impact.
What to do about #warning and #error
It's worth noting that the #warning
and #error
directives have many of the same AST representation and usability problems as #if
. Historically, it seems like most users haven't found this to be an issue in practice. Nevertheless, in the interests of consistency and maintainability, any redesign of conditional compilation should probably apply to #warning
and #error
as well.
Bringing it all together
I think we've seen enough proposals related to conditional compilation to indicate most of the community is interested in making some kind of change to the status quo. It also seems pretty clear to me that any solution we pick is going to involve some kind of compromise, whether it's tooling, code readability, or something else entirely. In my opinion, the most important considerations are going to be:
- Source compatibility
- Intra- declaration/statement/expression conditional compilation
- Tooling support for understanding and manipulating excluded code blocks
- Implementation maintenance burden
- How hard it is to expand the model to hypothetical future language features
And so far the languages I've looked at fall into one of two broad categories
- Preprocessor based conditional compilation: C, C++, C#, etc.
- Grammar-integrated conditional compilation: Swift today, Rust, D, etc.
As I see it that leaves us with three options when determining the future of conditional compilation in Swift:
- Keep the existing grammar-integrated model, accepting its current limitations. This would likely mean it would never support every syntactic construct, only those with clear and compelling use cases.
- Move to an integrated-preprocessor model, allowing conditional compilation directives to appear almost anywhere. This would likely be a source compatible change, and it resolves the issue of intra-declaration/statement/expression conditionals. It would likely have a negative impact on source tooling.
- Pursue some kind of hybrid model, perhaps involving conditional hoisting or some other program transformation. This isn't particularly well-defined and would require more investigation. It might be able to mitigate some of the downsides of the integrated preprocessor approach, but would come at the cost of significant technical complexity.
With all that said, I'm very interested in hearing everyone's thoughts on this. Do you know of any languages which take an approach different from the ones described here? Is there any other criteria we should be considering? What direction do you think we should go in from here? I'm leaning towards the second option at the moment (integrated preprocessor), but I haven't made up my mind yet.