I know that folks who have been working with C++ templates from Swift have found that need to introduce a bit of boilerplate for each instantiation, and were hoping that macros could help. @ktoso might have some insight here.
Doug
Could you elaborate a bit? At least for function templates I don't see what macros get us that generic functions don't (maybe specialization/performance, but that can already be done with attributes).
More generally, I think most of the boilerplate and attributes are on the C++ side, not the Swift side.
What I'm hitting and hoping to use macros for is on the Swift side, but more because of needing to generate boilerplate "bridges" between concurrency models. I.e. some C++ code using their own concurrency things and needing to call into Swift async functions; so I want to generate a small (and very repetetive) "bridge from C++ futures, to async function" method declarations. Not really much to do with C++ templates.
Usually it looks like this:
@_expose(Cxx)
actor Something {
func doIt() -> X { X() }
nonisolated public func doIt(promise: PromiseX) {
Task { promise.completeWith(await self.doIt) }
}
}
There is also annoying boilerplate on the C++ side because templates... since we can't use Promise<X> and have to do this:
using PromiseX = Promise<X>;
in order for Swift to be able to use this specialized template... I don't know if macros could help get rid of this annoyance, probably not.
We were thinking about âuniversal referencesâ over in Val land, and examples like this one seemed particularly difficult. How are we going to represent g's function call operator to Val? If we had macros, and were to import all C++ templates into Val as macros with macro expansion defined to instantiate the template, then at least in concrete code, uses of g would do what is clearly the right thing, and the template would be represented to Val not as a foreign kind of entity (imported C++ template) but as a Val macro that, at least from the outside, obeys the usual language rules.
Getting g to do what a C++ programmer expects from inside a Val generic, where the argument to g is (dependent on) a generic parameter, is a harder problem that would require monomorphization of Val generics and delayed macro expansion. So I'm not saying mapping templates to macros is a slam dunk, but it's worth investigating
Anyway, it seems like all the same reasoning applies to Swift. I guess what I'm saying is that templates are in many ways more like macros than they are like Swift generics, and, if you're going to have macros, trying to squeeze them into the mold of generics might be a losing proposition.
(As discussed in previous threads, I'll use the term monomorphization rather than specialization here to prevent ambiguity even though specialization is often the more common term when discussing C++ templates.)
I may be missing what you are proposing here, but I generally disagree. While I do like to (mostly facetiously) question the difference between templates and C++ macros, I don't see how Swift macros will be helpful in the import of templates.
There are a number of problems when importing templates into Swift, and I think monomorphization is near the bottom of the list. More importantly, Swift already has the infrastructure in place to monomorphize any imported C++ template (or Swift generic for that matter). The problem here is a more fundamental one: the places where templates cannot be monomorphized arise at module boundaries where we decide not to include templates definitions in the module (a mostly artificial limitation that would be relatively easy to overcome), and potentially deep in generic code where, again, we impose an artificial limitation to prevent substitution failures (and other unfortunate, template-related diagnostics) from leaking into Swift.
To put this a different way, macros work at a syntactic level (hence swift syntax), the issues we run into with importing templates are almost exclusively semantic. I could see creating a macro to assist in explicit monomorphization of templates (applying a substitution map to a call), but that is not a problem that needs solving. Macros are helpful when you need custom syntax but cannot modify the compiler. C++ interop is part of the compiler, so we don't need to define a macro for our monomorphization syntax (we can just modify the compiler). Swift already has a tool for constructing and applying substitution maps, and we can leverage this existing infrastructure which will result a more native experience anyway.
The semantic challenges with importing C++ templates are real, though, and we've discussed them at great length over the forums and in the C++ interop work group. If you see a way for macros to assist in type checking, etc. I'd be very interested to hear.
1 Like
To address this concrete suggestion, Swift macros still need to be type checked, so all the problems with type checking that arise when wrapping a C++ template in a Swift generic apply here as well (at least as far as I understand, but maybe I'm mistaken).
I think this is the crux of it. Once we have macros, it would be desirable to first try implementing the full feature ( in our case c++ template interop) using the macro system and only if the macro system doesnât not support what we need then it goes to the compiler. Today c++ interop has to live in the compiler but tomorrow maybe it can be moved to the macro system ( and if not then maybe it would be good to find out what feature work the macro system would need to make it possible).
How would you implement any part of C++ interop using macros (or even a future incarnation of macros)?
1 Like
I guess for me it would be something like CXX â system library interface for Rust // Lib.rs
The important thing to me is the interop was implemented as a package which means that the language would have the facilities for end users to create these types of interop (for any language) at the package level using any incarnation of macros (or anything else for that matter).
That is a completely different approach to interop. That package forces users to declare all the APIs they want, and it just handles mapping calling conventions and providing definitions. The meat of Swift and C++ interop is automatically providing these declarations, so APIs (mostly) "just show up."
Why is it important to you that interop is a package? What's preventing end users from using macros or whatever facilities to interop with other languages?
I should say, using macros to define foreign APIs in Swift is actually a pretty interesting idea. I could see macros being a useful tool here for interoperating with other languages.
But our goals for C++ interop necessitate a tighter integration with both the Swift and C++ compilers, so macros aren't of much help. I don't think it's a goal of macros to support this case either (and I'm not really sure how they could, even if it were desirable).
1 Like
Being a package folks do not have to learn the compiler to contribute which should allow more folks to participate. In the case of c, objective-c it makes sense that we put that into the compiler (there is not other place to put it) and even now for objective-c++ it would also make sense to follow the same pattern since these are meant for Apple pipelines.
My comments are more about c++ with non-clang compilers. Personally most of my work related c++ projects are only set up to compile with Microsoftâs compiler. If I wanted to use Swift for interop then I would have to add clang support to my code. If I find an issue with the way the compiler imports my c++ then I would have to wait for a fix in the compiler or fix it myself (yak shaving).
C++ interop is a great differentiator for swift but if it only works great for Darwin clang c++ projects then I think we will miss a great opportunity.
As someone who generally avoids macros like these but would like compile-time meta-programming, let me give you the perspective of the macro-averse (largely because I avoid domains that require boilerplate to begin with)
I want to explain a bit what I meant by this. In my experience, there are at least four common uses of compile-time meta-programming:
- Compile-time Code Generation (CTCG): the syntactic approach in this gist is clearly aimed primarily at this usecase, largely aimed at reducing boilerplate.
- Compile-time Manipulation (CTM): examples would be conditional compilation through
#ifstatements or type aliases, or perhaps some day allowing the programmer to set data alignment to compile-time computed values. - Compile-time Code Execution (CTCE): This is the least important, as you can always manually run the same code and output the results to build inputs, but when needed for the first two usecases above, it makes them so much easier to use.
- Generics: Really just a combination of the first three, but so important that it was implemented first with its own DSL.
The syntactic macro approach linked here may be sufficiently powerful that you could implement all of these with it, but it is primarily suited for CTCG, ie you probably wouldn't want to re-implement generics with it. I am primarily interested in CTM and some CTCE to help with it, which I don't think this approach is really geared towards. Not knocking this approach, as I see the need for it for CTCG, but I would like to see further work on the other two usecases in time.
3 Likes
First of all I'd like to say, that I am delighted to see work has begun
on adding macros to Swift.
Back to the discussion:
One over-looked aspect (in this discussion) is that macros allows
the users of the language to experiment with new language features.
This means:
1. more people can weigh in on design decisions
2. a larger design space can be explored
3. it's possible to try two or more implementations
of the same concept in order to contrast and compare
As a case in point, recently if-and-switch-expressions were discussed:
https://forums.swift.org/t/pitch-if-and-switch-expressions/61149
Having macros available would allow the authors ors of such a proposal
to write a working prototype without involving the compiler team at all.
For a user of a if-and-switch expressions it is irrelevant whether
they are implemented as macros or as core forms in the compiler.
In an ideal macro system invoking a macro shouldn't be marked
by a special sigil. A sigil might be needed for non-expression
macros, but it would be a good thing to allow macro calls
(for macros that expand into expressions) to have the same
syntax as standard function calls.
An if-expression like if(x=2,"yes","no") is self-explanatory,
even if it uses function syntax.
Since the macros are used only in situations where it is not possible
to use functions, in practise it is quite natural [look at languages such as
as Racket and Scheme, where macros calls and function calls
have the same syntax].
The discussion has been light on concrete examples on what macros
can be used for. In general macros are used in situations,
where normal functions aren't applicable.
The most common types of extensions are:
- new binding forms
- forms changing the order of evaluation
- forms that analyse program elements at compile time
ad 1.
A prime example is pattern matching.
The macro-less language Python just got pattern matching after 20 years.
A nifty binding construct is found in the MetaPost.
Given variables,say, x and y one can declare a linear dependence, say:
x + y = 300 and 0.1*(x+y)=20
When the variables are referenced, the equations involving the variables
are solved, and the results are used as the value of the reference.
Note: Can macros change variable references?
ad 2.
Standard control constructs like switch, case, cond, when, unless, etc.
Comprehensions for user data structures.
ad 3.
Check static properties that are outside the type system.
An important consideration is to provide users with tools that allows robust macros.
I can recommend the following papers that explains how the macro system works
in Racket. The method used in "Binding as Sets of Scopes" [1] has been used
in SweetJS ( https://www.sweetjs.org/ ) to add macros to JavaScript.
From a user perspective Racket allows users to write language
constructs that seamlessly fit into the language. Users of a macro doesn't
need to know whether a construct is "builtin" or defined using a macro.
Finally MacroPy [2] is a project that adds macros to Python. It could
be interesting in the sense that it shows one approach of adding
macros to a macro-less language.
[1] "Binding as Sets of Scopes"
Matthew Flatt
[2] "Submodules in Racket: You Want it When, Again?"
Matthew Flatt
[3] SweetJS https://www.sweetjs.org/
[4] MacroPy macropy3 · PyPI
1 Like
What about an uninitialized let?
let profileContainer : /* some type (possibly provided by the macro module) */
AXIDView(profileContainer)
This will allow a kind of opaque management type among several related macros.
I'm starting to become more skeptical if macros is a good idea. Macros are in general more difficult to understand for the programmer as there is a disconnect between what is being generated and what is scaffolding code. There are probably ways to do this more user friendly but still they are more difficult to follow.
If we look at existing examples like Nim, I find that macros in Nim are horrible and puts C++ to shame when it comes to user unfriendliness. AST macros in Nim are extremely verbose and you need to write a lot to do simple things. You also have to understand what every AST node does. Nim also have another syntax more similar to a procedural macro but they are also quite difficult to understand. You can do anything with Nim macros but the cost is very high.
The other end of this scale is D where they have incorporated metaprogramming as functions and generics. Implementation wise it looks more like C++ but without the crazy ML functional syntax. I find metaprogramming in D much more intuitive.
Then I ask you this question, is variadic generics a better gateway drug to metaprogramming than macros? Between macros and variadic generics, I would say that the latter one is much more desirable for me.
4 Likes
I suspect this is a dumb question, but could the existing REPL be used for this?
I'm not sure I'd say it failed "spectacularly", or even really that it failed at all. IIUC, it's a feature people still want, but has taken a backseat to more pressing concerns.
When Swift first came out, I was firmly in the "wait, no macros?!?" camp. These days, I'm much closer to "well, maybe we really can cover all their use cases with other language features", but I'm not sure I know enough to definitively answer that question. In particular, I hadn't really been thinking about the third kind of macro -- the kind where arbitrary code gets to manipulate the AST and such -- the last time I was thinking about whether macros are still "necessary". I think they've been called "procedural macros" (at least in this thread), but regardless it seems to me that they're essentially a way to bypass the language's syntax and get right at the heart of what can be done with the language's semantics. That seems extraordinarily powerful to me and I find myself being somewhat unable to look away.
I believe I have an immediate use case for variadic generics. This is not the case with macros, but that's at least partially because I haven't really thought about what I'd want to do with them (aside from code gen, of course).
2 Likes
Although this looks way better than macros in C, I think there is another way of doing metaprogramming that has more appeal. Sadly, Swift also still lacks really good reflection capabilities, but imagine those would be there, and you could use the same methods which are used to inspect entities at runtime to modify them at compile time.
Because there are many missing prerequisites, I don't expect I can show an example without some inconsistencies, but I'm thinking of something like this:
#compiletime do { // execute this block automatically during compilation
for each type in #module.types where type.protocols.contains(Equatable.self) && !Equatable.requirementsFullfilled(by: type) {
let other = Function.Parameter(type: type.self, label: "") // preparation for a new method
let compare = type.createMethod(name: "==", parameters: other, result: Bool.self, implement: Equatable.==)
for member in type.members {
guard let member = member as? Implementor<Equatable> else {
#error("Cannot make type \(type) Equatable because of instance variable \(member)"
}
compare.appendCode {
if member.value(in: compare.self) != member.value(in: other) {
compare.return(false)
}
}
}
compare.return(true)
}
}
}
It's probably way out of scope, but imo it's a really cool concept.
5 Likes
Is it possible to add attributes through the macro (@dynamicMemberLookup)?
Is it possible to add attributes through the macro (@dynamicMemberLookup)?
If macros are implemented as procedural code generators (like procedural macros in rust) then this would definitely be possible. I canât really think of an approach where it wouldnât be possible unless it was overlooked
2 Likes