The “constexpr” facility from C++ allows users to define constants and functions that are determined and usable at compile-time, for compile-time constructs but still usable at run-time. The facility is a key step for value-based generic parameters (and fixed-size arrays if you don’t want to be stuck with integer literals for bounds). Can figuring out Swift’s story here be part of Swift 5?
—
Daryle Walker
Mac, Internet, and Video Game Junkie
darylew AT mac DOT com
I’ve thought about this a lot lately. I think, given llvm’s built-in MCJIT engine and interpreter, it would be relatively easy to skip the under-developed constexpr that C++ settled on and go straight for a more elaborate compile-time facilities, which would also provide extremely powerful meta programming features (think C preprocessor macros, except context-aware, more powerful, convenient and safe). D (http://dlang.org) has this feature, which is really great, but it doesn’t save the otherwise sloppy and lackluster language.
In a nutshell, marking something as compiletime (or some more fitting keyword for the purpose) would do two things:
Any runtime-defined facilities (code to be executed, values of function parameters, assertion and precondition checks) would be attempted to be run/acquired at compiletime.
If any of those facilities are not possible to compute at compiletime, a compiler error would occur.
Besides being useful in generic value parameters, this could also be used to define a much more elaborate reflection system in Swift (which would now be much less of a magic).
If in addition to this, select parts of Swift’s frontend would be exposed to the language in the form of compile-time mutable values (like types, variables, functions, attributes), then users could use these compiletime facilities to do what Swift’s gyb.py (https://github.com/apple/swift/blob/master/utils/gyb.py\) currently does (and if you’re like me, you find yourself using it a lot, especially for libraries with a complete integration with some other library).
This is a huge undertaking, but if we split it in two parts: "executing at compiletime" and "exposing swift’s frontend”, this could surely be done by Swift 5. From all the ideas that I’ve had or seen on this mailing list, in my opinion this is the most ground-braking, powerful and useful, so I’d be willing to work day and night to help implement this.
On Jul 30, 2017, at 2:02 AM, Daryle Walker via swift-evolution <swift-evolution@swift.org> wrote:
The “constexpr” facility from C++ allows users to define constants and functions that are determined and usable at compile-time, for compile-time constructs but still usable at run-time. The facility is a key step for value-based generic parameters (and fixed-size arrays if you don’t want to be stuck with integer literals for bounds). Can figuring out Swift’s story here be part of Swift 5?
—
Daryle Walker
Mac, Internet, and Video Game Junkie
darylew AT mac DOT com_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
Note that there's no particular reason that value-based generic parameters, including fixed-size arrays, actually need to be constant expressions in Swift.
John.
On Jul 29, 2017, at 7:01 PM, Daryle Walker via swift-evolution <swift-evolution@swift.org> wrote:
The “constexpr” facility from C++ allows users to define constants and functions that are determined and usable at compile-time, for compile-time constructs but still usable at run-time. The facility is a key step for value-based generic parameters (and fixed-size arrays if you don’t want to be stuck with integer literals for bounds). Can figuring out Swift’s story here be part of Swift 5?
Hello,
[snip]
This is a huge undertaking, but if we split it in two parts: "executing at compiletime" and "exposing swift’s frontend”, this could surely be done by Swift 5. From all the ideas that I’ve had or seen on this mailing list, in my opinion this is the most ground-braking, powerful and useful, so I’d be willing to work day and night to help implement this.
As you say, I agree that these are separate concepts (compile-time evaluation vs. meta-programming), but the first is surely going to be helpful for the second.
As far as I can tell, compile-time evaluation would probably lose most / all of the stdlib. I’d hate to replicate parts of the standard library in a restricted subset of the language so that they are available in `constexpr` mode (which is what 80% of C++’s template meta-programming libraries do). I’m also not quite sure how to make that feature work with cross-compilation when host and target differ in architecture.
As for the meta-programming / macro part, a recent C++ proposal tries to extend the language in that direction: https://herbsutter.files.wordpress.com/2017/07/p0707r1.pdf .
Ignoring the syntax, at the very least it might reduce the need for gyb in places. That said, because Swift’s and C++’s instantiation model for generic code is so different, there’s going to be some blockers there…
So while I think they’re both great features to have, I assume the core team will have higher priorities on their schedule, as I don’t see how these features wouldn’t eat up most resources for at least (!) one Swift release-cycle.
Daniel.
On 30. Jul 2017, at 08:52, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org> wrote:
more elaborate compile-time facilities, which would also provide extremely powerful meta programming features
That's an interesting twist — but whenever you put a "meta" somewhere, things tend to get complicated, and people end up with different associations to the topic… ;-)
I took a brief look at the C++ document, but it seemed still to much macro-like to me.
My take on the topic would be he ability to express common programming tasks (declaring a class, overriding a method…) in the language itself.
Imagine
public class ProxyViewController: UIView {}
Could be written as
let subclass = createClass(classname: "ProxyViewController", superclass: UIViewController, accessLevel: .public)
Quite stupid at first sight, and basically the exact opposite of syntactic sugar ("syntactic salt" already has a meaning… so I'd call it "syntactic pepper" ;-).
But now imagine that:
for (method, implementation) in UIViewController.methods where method.accessLevel == .open {
subclass.methods[method] = { parameters in
print("Subclass method \(method) called with \(parameters)")
return implementation(parameters)
}
}
Not that stupid anymore, isn't it?
I think this would be way cooler than poking around with variants of search & replace…
- Tino
(to get syntax colouring, I wrote ~30 lines of Swift that turn the straw man example into valid code… it's fun, maybe I play with it a little bit more ;-)
The parameters for a fixed-size array type determine the type's size/stride, so how could the bounds not be needed during compile-time? The compiler can't layout objects otherwise.
Or do you mean that the bounds are integer literals? (That's what I have in the design document now.)
Sent from my iPhone
On Jul 30, 2017, at 8:51 PM, John McCall <rjmccall@apple.com> wrote:
On Jul 29, 2017, at 7:01 PM, Daryle Walker via swift-evolution <swift-evolution@swift.org> wrote:
The “constexpr” facility from C++ allows users to define constants and functions that are determined and usable at compile-time, for compile-time constructs but still usable at run-time. The facility is a key step for value-based generic parameters (and fixed-size arrays if you don’t want to be stuck with integer literals for bounds). Can figuring out Swift’s story here be part of Swift 5?Note that there's no particular reason that value-based generic parameters, including fixed-size arrays, actually need to be constant expressions in Swift.
John.
Hello,
[snip]
This is a huge undertaking, but if we split it in two parts: "executing at compiletime" and "exposing swift’s frontend”, this could surely be done by Swift 5. From all the ideas that I’ve had or seen on this mailing list, in my opinion this is the most ground-braking, powerful and useful, so I’d be willing to work day and night to help implement this.
As you say, I agree that these are separate concepts (compile-time evaluation vs. meta-programming), but the first is surely going to be helpful for the second.
That’s exactly what I had in mind: compiletime execution comes first, meta-programming gets built on top of it.
As far as I can tell, compile-time evaluation would probably lose most / all of the stdlib. I’d hate to replicate parts of the standard library in a restricted subset of the language so that they are available in `constexpr` mode (which is what 80% of C++’s template meta-programming libraries do). I’m also not quite sure how to make that feature work with cross-compilation when host and target differ in architecture.
I disagree. Most of the standard library is in protocols and small generic wrapper structures, which are prime candidates for compile-time evaluation. Even if Foundation has to be dropped (which is actually not that big of a deal), I think adapting the standard library to be less architecturally dependent to make it accessible in compiletime wouldn’t be such a big problem considering that it is already largely driven by protocols.
I think the architectural differences are negligible largely due to LLVM IR, which is portable (even byte ordering is taken into account).
As for the meta-programming / macro part, a recent C++ proposal tries to extend the language in that direction: https://herbsutter.files.wordpress.com/2017/07/p0707r1.pdf .
Ignoring the syntax, at the very least it might reduce the need for gyb in places. That said, because Swift’s and C++’s instantiation model for generic code is so different, there’s going to be some blockers there…
C++ relies on its template system as a compile-time code generation mechanism to achieve this, that’s why it can get away with little-to-no metaprogramming concepts. The major drawback is that it’s a lot like the C preprocessor: it’s a dumb search-replace shtick that has to be micromanaged to work correctly. Since Swift’s generics are a whole different kind of fruit, they are more closely related to C++’s functions that take a polymorphic pointer, rather then a template function, so Swift’s generics are not going to be of any help in this. My idea was to introduce a first-class toolset to Swift that allows one to essentially generate arbitrary Swift code at compile-time, but rather than generating a string and mixing it into the source, it would be much more AST-aware and safe. This could be done by executing a function at compiletime that returns an instance of Meta.TypeDefinition, which would be declared as compiletime struct (meaning, that the type and its instances cannot exist at runtime). The import keyword could be reused to not only statically import modules, but to import instances of Meta.TypeDefinition.
So while I think they’re both great features to have, I assume the core team will have higher priorities on their schedule, as I don’t see how these features wouldn’t eat up most resources for at least (!) one Swift release-cycle.
That’s just it: the core team is already prioritizing a lot of features that would greatly benefit from this. In fact, if this would be implemented, a large portion of the swift compiler could be moved into the standard library, because it wouldn’t be magic any more. This would single-handedly reduce the implementation time and effort of most proposals by a very large margin.
On Jul 30, 2017, at 1:03 PM, Daniel Vollmer via swift-evolution <swift-evolution@swift.org> wrote:
On 30. Jul 2017, at 08:52, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org> wrote:
Daniel.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
Tino Heth:
If you read my reply to Daniel Vollmer, you’ll find that we’re thinking about the exact same thing. Your code snippers show my vision of compiletime beautifully 
.
Now what I really want at this point is to here the opinions of the core team on this topic.
Swift Core Team:
Have you guys thought of this? Do you think this is a good idea to put on the table or do you have different plans?
On Jul 30, 2017, at 7:56 PM, Tino Heth <2th@gmx.de> wrote:
more elaborate compile-time facilities, which would also provide extremely powerful meta programming features
That's an interesting twist — but whenever you put a "meta" somewhere, things tend to get complicated, and people end up with different associations to the topic… ;-)
I took a brief look at the C++ document, but it seemed still to much macro-like to me.My take on the topic would be he ability to express common programming tasks (declaring a class, overriding a method…) in the language itself.
Imagine
public class ProxyViewController: UIView {}
Could be written as
let subclass = createClass(classname: "ProxyViewController", superclass: UIViewController, accessLevel: .public)Quite stupid at first sight, and basically the exact opposite of syntactic sugar ("syntactic salt" already has a meaning… so I'd call it "syntactic pepper" ;-).
But now imagine that:for (method, implementation) in UIViewController.methods where method.accessLevel == .open {
subclass.methods[method] = { parameters in
print("Subclass method \(method) called with \(parameters)")
return implementation(parameters)
}
}Not that stupid anymore, isn't it?
I think this would be way cooler than poking around with variants of search & replace…- Tino
(to get syntax colouring, I wrote ~30 lines of Swift that turn the straw man example into valid code… it's fun, maybe I play with it a little bit more ;-)
The parameters for a fixed-size array type determine the type's size/stride, so how could the bounds not be needed during compile-time? The compiler can't layout objects otherwise.
Swift is not C; it is perfectly capable of laying out objects at run time. It already has to do that for generic types and types with resilient members. That does, of course, have performance consequences, and those performance consequences might be unacceptable to you; but the fact that we can handle it means that we don't ultimately require a semantic concept of a constant expression, except inasmuch as we want to allow users to explicitly request guarantees about static layout.
Value equality would still affect the type-checker, but I think we could pretty easily just say that all bound expressions are assumed to potentially resolve unequally unless they are literals or references to the same 'let' constant.
The only hard constraint is that types need to be consistent, but that just means that we need to have a model in which bound expressions are evaluated exactly once at runtime (and of course typically folded at compile time).
John.
On Jul 30, 2017, at 11:43 PM, Daryle Walker <darylew@mac.com> wrote:
Or do you mean that the bounds are integer literals? (That's what I have in the design document now.)
Sent from my iPhone
On Jul 30, 2017, at 8:51 PM, John McCall <rjmccall@apple.com> wrote:
On Jul 29, 2017, at 7:01 PM, Daryle Walker via swift-evolution <swift-evolution@swift.org> wrote:
The “constexpr” facility from C++ allows users to define constants and functions that are determined and usable at compile-time, for compile-time constructs but still usable at run-time. The facility is a key step for value-based generic parameters (and fixed-size arrays if you don’t want to be stuck with integer literals for bounds). Can figuring out Swift’s story here be part of Swift 5?Note that there's no particular reason that value-based generic parameters, including fixed-size arrays, actually need to be constant expressions in Swift.
John.
I assume that the planned “hygienic macro” facility, if it has access to metadata, or “annotations", could be used for these sorts of meta tasks, .
This is slightly off-topic, but I hope that there is a way for macros to be fully expanded and debuggable (and even formatted). I use Cog with Swift as a kind of "macro-system on steroids” that provides these features. I would love to see a Cog-like system included as a standard feature of the language in the future. It could be a component of annotations, macros, or both.
I would like to see a source code generation pre-pass added for annotations, and hopefully macros when they arrive, so that developers can see, debug, and code against the expanded code. I realize that may be difficult or even impossible for some macros.
GOG uses Python as its macro language, which is certainly compatible with Apple’s toolchain. Using a real programming language for annotations, is extremely powerful.
The danger of adding a macro system too early, is that it can be used as a “cheat” to implement functionality that should be part of the base language and discourage language development. I trust that the core team will not let this happen.
See Cog:
https://nedbatchelder.com/code/cog/
A fully integrated Cog-like facility this should have language support to cleanup the (pretty ugly) delimiters and eventually tool support to selectively toggle expansion, as we can today with code blocks, for example, in many tools.
I don’t mean to derail this discussion, but it seems that an annotation or macro system would be appropriate for this kind of feature.
- Chris
On Jul 30, 2017, at 11:03 AM, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org> wrote:
Tino Heth:
If you read my reply to Daniel Vollmer, you’ll find that we’re thinking about the exact same thing. Your code snippers show my vision of compiletime beautifully
Now what I really want at this point is to here the opinions of the core team on this topic.Swift Core Team:
Have you guys thought of this? Do you think this is a good idea to put on the table or do you have different plans?On Jul 30, 2017, at 7:56 PM, Tino Heth <2th@gmx.de <mailto:2th@gmx.de>> wrote:
more elaborate compile-time facilities, which would also provide extremely powerful meta programming features
That's an interesting twist — but whenever you put a "meta" somewhere, things tend to get complicated, and people end up with different associations to the topic… ;-)
I took a brief look at the C++ document, but it seemed still to much macro-like to me.My take on the topic would be he ability to express common programming tasks (declaring a class, overriding a method…) in the language itself.
Imagine
public class ProxyViewController: UIView {}
Could be written as
let subclass = createClass(classname: "ProxyViewController", superclass: UIViewController, accessLevel: .public)Quite stupid at first sight, and basically the exact opposite of syntactic sugar ("syntactic salt" already has a meaning… so I'd call it "syntactic pepper" ;-).
But now imagine that:for (method, implementation) in UIViewController.methods where method.accessLevel == .open {
subclass.methods[method] = { parameters in
print("Subclass method \(method) called with \(parameters)")
return implementation(parameters)
}
}Not that stupid anymore, isn't it?
I think this would be way cooler than poking around with variants of search & replace…- Tino
(to get syntax colouring, I wrote ~30 lines of Swift that turn the straw man example into valid code… it's fun, maybe I play with it a little bit more ;-)
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
For completeness: I forgot to mention that I have been meaning to check-out Sourcery, which is possibly a more “Swifty” tool. I use Cog’s file-scope extensively, and it is unclear to me if that is a feature of Sourcery
OK, now I am sure that someone is going to tell me that I am off-topic :)
- Chris K
On Jul 30, 2017, at 11:54 AM, Christopher Kornher <ckornher@me.com> wrote:
I assume that the planned “hygienic macro” facility, if it has access to metadata, or “annotations", could be used for these sorts of meta tasks, .
This is slightly off-topic, but I hope that there is a way for macros to be fully expanded and debuggable (and even formatted). I use Cog with Swift as a kind of "macro-system on steroids” that provides these features. I would love to see a Cog-like system included as a standard feature of the language in the future. It could be a component of annotations, macros, or both.
I would like to see a source code generation pre-pass added for annotations, and hopefully macros when they arrive, so that developers can see, debug, and code against the expanded code. I realize that may be difficult or even impossible for some macros.
GOG uses Python as its macro language, which is certainly compatible with Apple’s toolchain. Using a real programming language for annotations, is extremely powerful.
The danger of adding a macro system too early, is that it can be used as a “cheat” to implement functionality that should be part of the base language and discourage language development. I trust that the core team will not let this happen.
See Cog:
Cog | Ned BatchelderA fully integrated Cog-like facility this should have language support to cleanup the (pretty ugly) delimiters and eventually tool support to selectively toggle expansion, as we can today with code blocks, for example, in many tools.
I don’t mean to derail this discussion, but it seems that an annotation or macro system would be appropriate for this kind of feature.
- Chris
On Jul 30, 2017, at 11:03 AM, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Tino Heth:
If you read my reply to Daniel Vollmer, you’ll find that we’re thinking about the exact same thing. Your code snippers show my vision of compiletime beautifully
Now what I really want at this point is to here the opinions of the core team on this topic.Swift Core Team:
Have you guys thought of this? Do you think this is a good idea to put on the table or do you have different plans?On Jul 30, 2017, at 7:56 PM, Tino Heth <2th@gmx.de <mailto:2th@gmx.de>> wrote:
more elaborate compile-time facilities, which would also provide extremely powerful meta programming features
That's an interesting twist — but whenever you put a "meta" somewhere, things tend to get complicated, and people end up with different associations to the topic… ;-)
I took a brief look at the C++ document, but it seemed still to much macro-like to me.My take on the topic would be he ability to express common programming tasks (declaring a class, overriding a method…) in the language itself.
Imagine
public class ProxyViewController: UIView {}
Could be written as
let subclass = createClass(classname: "ProxyViewController", superclass: UIViewController, accessLevel: .public)Quite stupid at first sight, and basically the exact opposite of syntactic sugar ("syntactic salt" already has a meaning… so I'd call it "syntactic pepper" ;-).
But now imagine that:for (method, implementation) in UIViewController.methods where method.accessLevel == .open {
subclass.methods[method] = { parameters in
print("Subclass method \(method) called with \(parameters)")
return implementation(parameters)
}
}Not that stupid anymore, isn't it?
I think this would be way cooler than poking around with variants of search & replace…- Tino
(to get syntax colouring, I wrote ~30 lines of Swift that turn the straw man example into valid code… it's fun, maybe I play with it a little bit more ;-)
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
Tino Heth:
If you read my reply to Daniel Vollmer, you’ll find that we’re thinking about the exact same thing. Your code snippers show my vision of compiletime beautifully
Now what I really want at this point is to here the opinions of the core team on this topic.Swift Core Team:
Have you guys thought of this? Do you think this is a good idea to put on the table or do you have different plans?
I don’t speak for the core team as a whole, but here are my personal opinions. Note that this is off the cuff, without a lot of deliberate consideration, so YMMV:
1) I don’t think it makes sense to add constexpr before there is a motivating use-case for it. I agree that it could be useful with fixed sized arrays and with constant parameters to generics, but we don’t have either of those yet. If it were up to me, the place to start would be figuring out constant generic parameters, since the answer there could inform fixed size arrays.
2) I agree that there is heavy overlap with constexpr and macros, and that it would be (probably showstoppingly) unfortunate to require duplication of large swaths for the stdlib to make it work with constexprs. Adding “constexpr” to a few operator implementations would be reasonable though, and may be a way to formalize “transparent”.
3) I’m a fan of eventually adding a very general, but still hygienic, macro system at some point, and assume that the # sigil will be used to for all things macros. For example, I’ve come to see the old “property behaviors” proposal as being better reimagined as “property macros”, which suggest syntax like “var #resettable foo : T” or something like that.
4) The macro system should eventually support unstructured compile time expansion, along the lines of what people do with Gyb and Sourcery with all the tradeoffs they bring.
I tend to prefer that exploration in this space happen in order of points above, and would prefer not to get to a generalized macro system for a couple of years - just to give Swift more time to mature and develop without it. Other languages have gotten macro systems too early in their development, and become too reliant on it. Perhaps if C didn’t have one it would have ended up with a proper import mechanism. Perhaps if Rust didn’t have one, some of its decisions would have been different. etc.
-Chris
On Jul 30, 2017, at 10:03 AM, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org> wrote:
On Jul 30, 2017, at 7:56 PM, Tino Heth <2th@gmx.de <mailto:2th@gmx.de>> wrote:
more elaborate compile-time facilities, which would also provide extremely powerful meta programming features
That's an interesting twist — but whenever you put a "meta" somewhere, things tend to get complicated, and people end up with different associations to the topic… ;-)
I took a brief look at the C++ document, but it seemed still to much macro-like to me.My take on the topic would be he ability to express common programming tasks (declaring a class, overriding a method…) in the language itself.
Imagine
public class ProxyViewController: UIView {}
Could be written as
let subclass = createClass(classname: "ProxyViewController", superclass: UIViewController, accessLevel: .public)Quite stupid at first sight, and basically the exact opposite of syntactic sugar ("syntactic salt" already has a meaning… so I'd call it "syntactic pepper" ;-).
But now imagine that:for (method, implementation) in UIViewController.methods where method.accessLevel == .open {
subclass.methods[method] = { parameters in
print("Subclass method \(method) called with \(parameters)")
return implementation(parameters)
}
}Not that stupid anymore, isn't it?
I think this would be way cooler than poking around with variants of search & replace…- Tino
(to get syntax colouring, I wrote ~30 lines of Swift that turn the straw man example into valid code… it's fun, maybe I play with it a little bit more ;-)
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
Is there any reason that constexpr couldn’t be inferred? Wherever you need something to be compile time constant — value-based generics, rawValue enum cases, etc — the compiler could make sure that the value was known at compile time, and if not then raise an error. Basically, assume everything is constexpr unless it can be shown to not (necessarily) be the case.
On Jul 30, 2017, at 5:34 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
On Jul 30, 2017, at 10:03 AM, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org> wrote:
Tino Heth:
If you read my reply to Daniel Vollmer, you’ll find that we’re thinking about the exact same thing. Your code snippers show my vision of compiletime beautifully
Now what I really want at this point is to here the opinions of the core team on this topic.Swift Core Team:
Have you guys thought of this? Do you think this is a good idea to put on the table or do you have different plans?I don’t speak for the core team as a whole, but here are my personal opinions. Note that this is off the cuff, without a lot of deliberate consideration, so YMMV:
1) I don’t think it makes sense to add constexpr before there is a motivating use-case for it. I agree that it could be useful with fixed sized arrays and with constant parameters to generics, but we don’t have either of those yet. If it were up to me, the place to start would be figuring out constant generic parameters, since the answer there could inform fixed size arrays.
2) I agree that there is heavy overlap with constexpr and macros, and that it would be (probably showstoppingly) unfortunate to require duplication of large swaths for the stdlib to make it work with constexprs. Adding “constexpr” to a few operator implementations would be reasonable though, and may be a way to formalize “transparent”.
3) I’m a fan of eventually adding a very general, but still hygienic, macro system at some point, and assume that the # sigil will be used to for all things macros. For example, I’ve come to see the old “property behaviors” proposal as being better reimagined as “property macros”, which suggest syntax like “var #resettable foo : T” or something like that.
4) The macro system should eventually support unstructured compile time expansion, along the lines of what people do with Gyb and Sourcery with all the tradeoffs they bring.
I tend to prefer that exploration in this space happen in order of points above, and would prefer not to get to a generalized macro system for a couple of years - just to give Swift more time to mature and develop without it. Other languages have gotten macro systems too early in their development, and become too reliant on it. Perhaps if C didn’t have one it would have ended up with a proper import mechanism. Perhaps if Rust didn’t have one, some of its decisions would have been different. etc.
-Chris
On Jul 30, 2017, at 7:56 PM, Tino Heth <2th@gmx.de> wrote:
more elaborate compile-time facilities, which would also provide extremely powerful meta programming features
That's an interesting twist — but whenever you put a "meta" somewhere, things tend to get complicated, and people end up with different associations to the topic… ;-)
I took a brief look at the C++ document, but it seemed still to much macro-like to me.My take on the topic would be he ability to express common programming tasks (declaring a class, overriding a method…) in the language itself.
Imagine
public class ProxyViewController: UIView {}
Could be written as
let subclass = createClass(classname: "ProxyViewController", superclass: UIViewController, accessLevel: .public)Quite stupid at first sight, and basically the exact opposite of syntactic sugar ("syntactic salt" already has a meaning… so I'd call it "syntactic pepper" ;-).
But now imagine that:for (method, implementation) in UIViewController.methods where method.accessLevel == .open {
subclass.methods[method] = { parameters in
print("Subclass method \(method) called with \(parameters)")
return implementation(parameters)
}
}Not that stupid anymore, isn't it?
I think this would be way cooler than poking around with variants of search & replace…- Tino
(to get syntax colouring, I wrote ~30 lines of Swift that turn the straw man example into valid code… it's fun, maybe I play with it a little bit more ;-)
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
I don’t think a Cog-like or Gyb-like macro system would be good. That’s essentially a preprocessor and we all know how that worked out for C. The only a code-generation feature can exist in a safe language like Swift is by making it a first-class citizen as described previously. I’m not sure whether or not the core team would consider such a thing, bit I am very certain that a text-based macro system is something they’d never consider.
On Jul 30, 2017, at 8:54 PM, Christopher Kornher <ckornher@me.com> wrote:
I assume that the planned “hygienic macro” facility, if it has access to metadata, or “annotations", could be used for these sorts of meta tasks, .
This is slightly off-topic, but I hope that there is a way for macros to be fully expanded and debuggable (and even formatted). I use Cog with Swift as a kind of "macro-system on steroids” that provides these features. I would love to see a Cog-like system included as a standard feature of the language in the future. It could be a component of annotations, macros, or both.
I would like to see a source code generation pre-pass added for annotations, and hopefully macros when they arrive, so that developers can see, debug, and code against the expanded code. I realize that may be difficult or even impossible for some macros.
GOG uses Python as its macro language, which is certainly compatible with Apple’s toolchain. Using a real programming language for annotations, is extremely powerful.
The danger of adding a macro system too early, is that it can be used as a “cheat” to implement functionality that should be part of the base language and discourage language development. I trust that the core team will not let this happen.
See Cog:
Cog | Ned BatchelderA fully integrated Cog-like facility this should have language support to cleanup the (pretty ugly) delimiters and eventually tool support to selectively toggle expansion, as we can today with code blocks, for example, in many tools.
I don’t mean to derail this discussion, but it seems that an annotation or macro system would be appropriate for this kind of feature.
- Chris
On Jul 30, 2017, at 11:03 AM, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Tino Heth:
If you read my reply to Daniel Vollmer, you’ll find that we’re thinking about the exact same thing. Your code snippers show my vision of compiletime beautifully
Now what I really want at this point is to here the opinions of the core team on this topic.Swift Core Team:
Have you guys thought of this? Do you think this is a good idea to put on the table or do you have different plans?On Jul 30, 2017, at 7:56 PM, Tino Heth <2th@gmx.de <mailto:2th@gmx.de>> wrote:
more elaborate compile-time facilities, which would also provide extremely powerful meta programming features
That's an interesting twist — but whenever you put a "meta" somewhere, things tend to get complicated, and people end up with different associations to the topic… ;-)
I took a brief look at the C++ document, but it seemed still to much macro-like to me.My take on the topic would be he ability to express common programming tasks (declaring a class, overriding a method…) in the language itself.
Imagine
public class ProxyViewController: UIView {}
Could be written as
let subclass = createClass(classname: "ProxyViewController", superclass: UIViewController, accessLevel: .public)Quite stupid at first sight, and basically the exact opposite of syntactic sugar ("syntactic salt" already has a meaning… so I'd call it "syntactic pepper" ;-).
But now imagine that:for (method, implementation) in UIViewController.methods where method.accessLevel == .open {
subclass.methods[method] = { parameters in
print("Subclass method \(method) called with \(parameters)")
return implementation(parameters)
}
}Not that stupid anymore, isn't it?
I think this would be way cooler than poking around with variants of search & replace…- Tino
(to get syntax colouring, I wrote ~30 lines of Swift that turn the straw man example into valid code… it's fun, maybe I play with it a little bit more ;-)
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
If you read my reply to Daniel Vollmer, you’ll find that we’re thinking about the exact same thing.
Good to hear — read that post later, but I guess repeating doesn't hurt here: People still talk about macros... ;-)
So while I think they’re both great features to have, I assume the core team will have higher priorities on their schedule, as I don’t see how these features wouldn’t eat up most resources for at least (!) one Swift release-cycle.
That’s just it: the core team is already prioritizing a lot of features that would greatly benefit from this. In fact, if this would be implemented, a large portion of the swift compiler could be moved into the standard library, because it wouldn’t be magic any more. This would single-handedly reduce the implementation time and effort of most proposals by a very large margin.
… and I like specific examples ;-), so out of my head, four features that people want to have and that could be added easily with "Meta-Swift":
- Forwarding of protocol conformance (Kotlin, for example, has this: When a member conforms to a protocol, you don't have to write a bunch of methods that just say "let my member do this")
- init with reduced boilerplate
- Subtyping for non-class types, including a "newtype" option
- Property behaviours
Imho the reflection capabilities that are needed to make the feature fly are the big tasks — using this information shouldn't be that hard, as it's just a explicit way of telling the compiler what to do.
Assuming that reflection will be improved anyways, it might be a very cheap feature with huge benefit.
It seems to me that this applies just the same to type generic parameters. Are there any reason that you can't instantiate a generic type at runtime, or is it just because the need is not as evident as it would/could be with non-type generic parameters?
Le 30 juil. 2017 à 21:10, John McCall via swift-evolution <swift-evolution@swift.org> a écrit :
On Jul 30, 2017, at 11:43 PM, Daryle Walker <darylew@mac.com> wrote:
The parameters for a fixed-size array type determine the type's size/stride, so how could the bounds not be needed during compile-time? The compiler can't layout objects otherwise.Swift is not C; it is perfectly capable of laying out objects at run time. It already has to do that for generic types and types with resilient members. That does, of course, have performance consequences, and those performance consequences might be unacceptable to you; but the fact that we can handle it means that we don't ultimately require a semantic concept of a constant expression, except inasmuch as we want to allow users to explicitly request guarantees about static layout.
Value equality would still affect the type-checker, but I think we could pretty easily just say that all bound expressions are assumed to potentially resolve unequally unless they are literals or references to the same 'let' constant.
The only hard constraint is that types need to be consistent, but that just means that we need to have a model in which bound expressions are evaluated exactly once at runtime (and of course typically folded at compile time).
John.
Or do you mean that the bounds are integer literals? (That's what I have in the design document now.)
Sent from my iPhone
On Jul 30, 2017, at 8:51 PM, John McCall <rjmccall@apple.com> wrote:
On Jul 29, 2017, at 7:01 PM, Daryle Walker via swift-evolution <swift-evolution@swift.org> wrote:
The “constexpr” facility from C++ allows users to define constants and functions that are determined and usable at compile-time, for compile-time constructs but still usable at run-time. The facility is a key step for value-based generic parameters (and fixed-size arrays if you don’t want to be stuck with integer literals for bounds). Can figuring out Swift’s story here be part of Swift 5?Note that there's no particular reason that value-based generic parameters, including fixed-size arrays, actually need to be constant expressions in Swift.
John.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
The parameters for a fixed-size array type determine the type's size/stride, so how could the bounds not be needed during compile-time? The compiler can't layout objects otherwise.
Swift is not C; it is perfectly capable of laying out objects at run time. It already has to do that for generic types and types with resilient members. That does, of course, have performance consequences, and those performance consequences might be unacceptable to you; but the fact that we can handle it means that we don't ultimately require a semantic concept of a constant expression, except inasmuch as we want to allow users to explicitly request guarantees about static layout.
Doesn't this defeat the purpose of generic value parameters? We might as well use a regular parameter if there's no compile-time evaluation involved. In that case, fixed-sized arrays will be useless, because they'll be normal arrays with resizing disabled. As far as I know, the pinnacle of uses for fixed-size arrays is having a compile-time pre-allocated space of the necessary size (either literally at compile-time if that's a static variable, or added to the pre-computed offset of the stack pointer in case of a local variable).
Value equality would still affect the type-checker, but I think we could pretty easily just say that all bound expressions are assumed to potentially resolve unequally unless they are literals or references to the same 'let' constant.
Shouldn't the type-checker use the Equatable protocol conformance to test for equality? Moreover, as far as I know, Equatable is not recognized by the compiler in any way, so it's just a regular protocol. What would make it special? Some types would implement operator == to compare themselves to other types, that's beyond the scope of Equatable. What about those? And how are custom operator implementations going to serve this purpose at compile-time? Or will it just ignore the semantics of the type and reduce it to a sequence of bits? Or maybe only a few hand-picked types will be supported?
The seemingly simple generic value parameter concept gets vastly complicated and/or poorly designed without an elaborate compile-time execution system... Unless I'm missing an obvious way out.
The only hard constraint is that types need to be consistent, but that just means that we need to have a model in which bound expressions are evaluated exactly once at runtime (and of course typically folded at compile time).
What exactly would it take to be able to execute select piece of code at compile-time? Taking the AST, converting it to LLVM IR and feeding it to the MCJIT engine seems to be easy enough. But I'm pretty sure it's more tricky than that. Is there a special assumption or two made about the code that prevents this from happening?
On Jul 31, 2017, at 7:10 AM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:
On Jul 30, 2017, at 11:43 PM, Daryle Walker <darylew@mac.com> wrote:
John.
Or do you mean that the bounds are integer literals? (That's what I have in the design document now.)
Sent from my iPhone
On Jul 30, 2017, at 8:51 PM, John McCall <rjmccall@apple.com> wrote:
On Jul 29, 2017, at 7:01 PM, Daryle Walker via swift-evolution <swift-evolution@swift.org> wrote:
The “constexpr” facility from C++ allows users to define constants and functions that are determined and usable at compile-time, for compile-time constructs but still usable at run-time. The facility is a key step for value-based generic parameters (and fixed-size arrays if you don’t want to be stuck with integer literals for bounds). Can figuring out Swift’s story here be part of Swift 5?Note that there's no particular reason that value-based generic parameters, including fixed-size arrays, actually need to be constant expressions in Swift.
John.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
Why this long due myth subsist, LLVM IR is not portable. You can produce portable IR by limiting the scope of the input language, but it does not even provides a way to express sizeof(long) in a architecture independent way. And I don’t even mention the type layouts.
Le 30 juil. 2017 à 12:23, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org> a écrit :
On Jul 30, 2017, at 1:03 PM, Daniel Vollmer via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Hello,
On 30. Jul 2017, at 08:52, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
[snip]
This is a huge undertaking, but if we split it in two parts: "executing at compiletime" and "exposing swift’s frontend”, this could surely be done by Swift 5. From all the ideas that I’ve had or seen on this mailing list, in my opinion this is the most ground-braking, powerful and useful, so I’d be willing to work day and night to help implement this.
As you say, I agree that these are separate concepts (compile-time evaluation vs. meta-programming), but the first is surely going to be helpful for the second.
That’s exactly what I had in mind: compiletime execution comes first, meta-programming gets built on top of it.
As far as I can tell, compile-time evaluation would probably lose most / all of the stdlib. I’d hate to replicate parts of the standard library in a restricted subset of the language so that they are available in `constexpr` mode (which is what 80% of C++’s template meta-programming libraries do). I’m also not quite sure how to make that feature work with cross-compilation when host and target differ in architecture.
I disagree. Most of the standard library is in protocols and small generic wrapper structures, which are prime candidates for compile-time evaluation. Even if Foundation has to be dropped (which is actually not that big of a deal), I think adapting the standard library to be less architecturally dependent to make it accessible in compiletime wouldn’t be such a big problem considering that it is already largely driven by protocols.
I think the architectural differences are negligible largely due to LLVM IR, which is portable (even byte ordering is taken into account).
I am not sure that macros and annotations would be good, either. It may be impossible to avoid them, given Swift’s goal of “world domination”. Both features are very useful in certain applications.
We could continue to use custom tools like Cog and Sourcery when we need to, or standardize and better integrate their functionality. I honestly am not sure which is a better solution.
"Hygienic Macros” and “Annotations” have both been discussed. Both of these features would bring power and complexity to the language, to be sure. I just wanted to voice my opinion that these features, if and when they arrive, should be transparent, i.e., the generated code should be visible and sensibly formatted so that it can be examined and debugged. This would be a hard problem, and I am not sure if it is even possible for complex macro & annotation combinations…
C macros are not hygienic and not “transparent”. C preprocessor output, for example, is largely unusable.
The uses of macros and annotation overlap. Introducing macros and annotations at the same time would provide a chance to sensibly balance the features of each and provide a coherent set of guidelines for developers.
- Chris K
On Jul 30, 2017, at 12:03 PM, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org> wrote:
I don’t think a Cog-like or Gyb-like macro system would be good. That’s essentially a preprocessor and we all know how that worked out for C. The only a code-generation feature can exist in a safe language like Swift is by making it a first-class citizen as described previously. I’m not sure whether or not the core team would consider such a thing, bit I am very certain that a text-based macro system is something they’d never consider.
On Jul 30, 2017, at 8:54 PM, Christopher Kornher <ckornher@me.com <mailto:ckornher@me.com>> wrote:
I assume that the planned “hygienic macro” facility, if it has access to metadata, or “annotations", could be used for these sorts of meta tasks, .
This is slightly off-topic, but I hope that there is a way for macros to be fully expanded and debuggable (and even formatted). I use Cog with Swift as a kind of "macro-system on steroids” that provides these features. I would love to see a Cog-like system included as a standard feature of the language in the future. It could be a component of annotations, macros, or both.
I would like to see a source code generation pre-pass added for annotations, and hopefully macros when they arrive, so that developers can see, debug, and code against the expanded code. I realize that may be difficult or even impossible for some macros.
GOG uses Python as its macro language, which is certainly compatible with Apple’s toolchain. Using a real programming language for annotations, is extremely powerful.
The danger of adding a macro system too early, is that it can be used as a “cheat” to implement functionality that should be part of the base language and discourage language development. I trust that the core team will not let this happen.
See Cog:
Cog | Ned BatchelderA fully integrated Cog-like facility this should have language support to cleanup the (pretty ugly) delimiters and eventually tool support to selectively toggle expansion, as we can today with code blocks, for example, in many tools.
I don’t mean to derail this discussion, but it seems that an annotation or macro system would be appropriate for this kind of feature.
- Chris
On Jul 30, 2017, at 11:03 AM, Gor Gyolchanyan via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Tino Heth:
If you read my reply to Daniel Vollmer, you’ll find that we’re thinking about the exact same thing. Your code snippers show my vision of compiletime beautifully
Now what I really want at this point is to here the opinions of the core team on this topic.Swift Core Team:
Have you guys thought of this? Do you think this is a good idea to put on the table or do you have different plans?On Jul 30, 2017, at 7:56 PM, Tino Heth <2th@gmx.de <mailto:2th@gmx.de>> wrote:
more elaborate compile-time facilities, which would also provide extremely powerful meta programming features
That's an interesting twist — but whenever you put a "meta" somewhere, things tend to get complicated, and people end up with different associations to the topic… ;-)
I took a brief look at the C++ document, but it seemed still to much macro-like to me.My take on the topic would be he ability to express common programming tasks (declaring a class, overriding a method…) in the language itself.
Imagine
public class ProxyViewController: UIView {}
Could be written as
let subclass = createClass(classname: "ProxyViewController", superclass: UIViewController, accessLevel: .public)Quite stupid at first sight, and basically the exact opposite of syntactic sugar ("syntactic salt" already has a meaning… so I'd call it "syntactic pepper" ;-).
But now imagine that:for (method, implementation) in UIViewController.methods where method.accessLevel == .open {
subclass.methods[method] = { parameters in
print("Subclass method \(method) called with \(parameters)")
return implementation(parameters)
}
}Not that stupid anymore, isn't it?
I think this would be way cooler than poking around with variants of search & replace…- Tino
(to get syntax colouring, I wrote ~30 lines of Swift that turn the straw man example into valid code… it's fun, maybe I play with it a little bit more ;-)
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
If you read my reply to Daniel Vollmer, you’ll find that we’re thinking about the exact same thing.
Good to hear — read that post later, but I guess repeating doesn't hurt here: People still talk about macros... ;-)
So while I think they’re both great features to have, I assume the core team will have higher priorities on their schedule, as I don’t see how these features wouldn’t eat up most resources for at least (!) one Swift release-cycle.
That’s just it: the core team is already prioritizing a lot of features that would greatly benefit from this. In fact, if this would be implemented, a large portion of the swift compiler could be moved into the standard library, because it wouldn’t be magic any more. This would single-handedly reduce the implementation time and effort of most proposals by a very large margin.
… and I like specific examples ;-), so out of my head, four features that people want to have and that could be added easily with "Meta-Swift":
- Forwarding of protocol conformance (Kotlin, for example, has this: When a member conforms to a protocol, you don't have to write a bunch of methods that just say "let my member do this")
- init with reduced boilerplate
- Subtyping for non-class types, including a "newtype" option
- Property behaviours
The first and third items in that list seem similar to the “strong type-alias” idea I just updated (at <https://gist.github.com/CTMacUser/c493f775075e946efdcfd85d38473291>\). Is it close to what you’re talking about? I did the update because I recently looked up product and sum types again, and saw things called subtypes and quotient types. I realized that, with a little tweaking, strong-type-aliases could what would be needed to model sub- and quotient types.
On Jul 30, 2017, at 5:34 PM, Tino Heth <2th@gmx.de> wrote:
Imho the reflection capabilities that are needed to make the feature fly are the big tasks — using this information shouldn't be that hard, as it's just a explicit way of telling the compiler what to do.
Assuming that reflection will be improved anyways, it might be a very cheap feature with huge benefit.
—
Daryle Walker
Mac, Internet, and Video Game Junkie
darylew AT mac DOT com