There is a typo in one of the examples. An extra quote is in the middle of this string.
NSPredicate(format: "SUBQUERY(recipients, $recipient, recipient.firstName == sender.firstName").@count > 0")
There is a typo in one of the examples. An extra quote is in the middle of this string.
NSPredicate(format: "SUBQUERY(recipients, $recipient, recipient.firstName == sender.firstName").@count > 0")
Hi all, thank you everyone for your feedback so far! We've made some small updates to the pitch, and I've updated the pitch document linked in this post (the document here for reference). The main changes include:
#predicate
to #Predicate
to align with capitalized names typically used for invoking a type's initializerbuild_KeyPath
function invoked by the macro for previous uses of dynamic member lookupWe appreciate all of your input, and we'd love any feedback you may have with these new revisions. Feel free to let us know if you have any comments or questions!
I don't think we've established macro naming as part of the guidelines, but this seems to be an odd choice to me. If it starts with #
it's a macro, whether or not that macro initializes a type under the hood. I'd expect macros to be lower cased. If you want to initialize a type, can't you make a normal Predicate
type that takes the macro closure in one of the initializers? Frankly I'd prefer that rather than seeing #
hanging around so much.
This is very interesting; having an example of multiple cutting-edge language features (variadic generics, macros) working in unison to tighten type-safety is compelling.
I'm curious how the set of operations supported by the Predicate
macro system will be understood by developers. I see a couple avenues where this may arise:
For example, suppose a developer is interested in achieving the semantics of this example from the proposal:
let predicate = #Predicate<Message> { message in
message.recipients.filter {
$0.firstName == message.sender.firstName
}.count > 0
}
but, seeing what looks like freely-written Swift in the provided closure, instead attempts to write:
let predicate = #Predicate<Message> { message in
message.recipients.map(\.firstName).filter {
$0 == message.sender.firstName
}.count > 0
}
where map
is an example of an unsupported operator (and presumably this fails to compile).
Can diagnostics & autocomplete inform the developer that e.g. the only supported operations on sequences are filter(_:)
, contains(_:)
, contains(where:)
, allSatisfy(_:)
, etc.?
If supported in autocomplete, how is that relationship between macros and the IDE communicated? I'm guessing the macro definition itself isn't enough, but if that relationship is described in one of the macro proposals and I've missed it, my apologies.
(I recognize some developer experience-type questions may be outside the scope of this pitch, but thought I'd ask!)
You're right, we haven't quite established naming as part of the guidelines (tagging @Douglas_Gregor since we've briefly discussed this). I don't think dropping the #
altogether is a direction we want to go towards. While the semantics of the pre- and post-expansion code are the same, there's quite a bit of heavy lifting going on in the macro here that we'd like to be clearly evident to the developer by writing #Predicate
at the construction site rather than hiding the macro invocation in the declaration of the initializer. Given the choice between something like #predicate
and #Predicate
, we felt that #Predicate
looked more natural to indicate that the macro initializes a type, rather than something like #assert
which acts more like a function call. Doug might have some more thoughts here.
Potentially a bit out of scope for the pitch here, but still an important question nonetheless! Currently diagnostics are our main tool here, and in fact this is one of the compelling reasons why we'd like to use macros instead of a solution like operator overloading. The macro will produce diagnostics for this case that tell the developer that the function is not able to be used in the context of the predicate they are creating. For common functions, we also have the opportunity to provide fix-its or suggestions as applicable within these diagnostics, but in general the diagnostic will just alert the developer that this function can't be used. Currently, our main source of truth for developers to see a list of supported operations would be the documentation (for example, predicates support all expressions that conform to StandardPredicateExpression
). Macros don't currently have a way to influence the autocomplete results, but that is an avenue that could be interesting to bring up on the macro proposal.
This is a much larger discussion that doesn't really need to be here, so I've threaded it.
This is an interesting pitch which provides a lot of useful functionality!
So I have one fundamental question - what would be the approach for allowing for a UI that edits a representation of a predicate and transforms it back to the serialised entity that can be sent over to another process?
Specifically, we'd like to use predicates as a generic filter mechanism, where e.g. a frontend can edit a predicate (including values that are used as part of the expressions) and get a serialised representation that can be sent over the network to another process that then applies it (either directly or using the described support for transform to e.g. SQL).
It seems the needed mechanisms are halfway there ("Tree walking" section), but it is not (yet) quite clear to me if this would be flexible enough of a hook to go from serialised predicate -> UI. The reverse (going from some flexible/dynamic internal representation that allows a human being to interact with it, rather than coding stuff, over to the Predicate format) seems out of scope currently (but would be a critical piece to make this deeply more useful for many use cases).
Perhaps I'm missing something, but would be interested to hear your ideas here.
What do you see as the obstacles to doing what you propose? I think it's an entirely reasonable goal we want to support.
Not sure there are obstacles, I'm just trying to understand how do to a few things based of the pitch description (it's harder when you can't play with it ) - maybe two questions to begin with:
myPredicate
- is it by simply calling evaluate
? In that case, will a full evaluation always be done of all nodes in the tree, or can logic be short circuited before all nodes are traversed in the tree?PredicateExpression
types.Predicate
and all the PredicateExpression
types you want to handle to that protocol via extensions. If the protocol is (e.g.) ParseToResult
like so:protocol ParseToResult {
func parse() -> Result
}
after writing extensions to conform Predicate
et. al. to ParseToResult
, you'd then do this:
let myResult = aPredicate.parse()
Note that the parse function could have an argument that would act as global state that could be passed down to component PredicateExpression
values. While the example works if you have your own Predicate
type, to do it for standard Predicate
you need to cast:
extension Predicate: ParseToResult
func parse() -> Result {
return (expression as! ParseToResult).parse()
}
}
If you wanted to allow for the case where you don't handle every PredicateExpression
type, you could make parse()
return an optional and use as?
instead.
evaluate()
is not involved in tree walking. You only use that if you wish to supply input to the predicate and evaluate it.
Thanks for the clarification @dgoldsmith (we missed parse()
somehow, it was the missing piece for us - EDIT: in fact, looking at the pitch, I can't find it, perhaps it can be added to the Tree Walking section as an example? It would be clearer than spotlightQuery
at least for us) - we've discussed the pitch with our team internally and overall we think it would be a great addition and definitely would make heavy use of this, looks really promising.
Our only remaining major concern (which obviously is hard to know from a pitch, but please view this as an open question) would as mentioned be performance (of evaluate()
- specifically the use of keypaths, as even though SE-061 have the following note on performance:
The performance of interacting with a property/subscript via KeyPaths should be close to the cost of calling the property directly.
There are a few references so far to (quite significant) performance issues with the current keypath implementation, e.g.:
and
I understand the optimization of keypath handling is handled by a different set of priorities, I just wanted to point it out if there is anything that can be done to minimize that possible impact.
I guess it's only tangential to the pitch, but wanted to at least call it out as performance of evaluate()
is critical to the usability of Predicates (at least for us) - and the pitch overall really looks great and we'd be super happy to use it (as long as performance is ok).
So, anyway - big +1 for the pitch overall.
Thanks for the explanation! I played around with expression macros and (although I got quite a few errors and couldn’t run anything) I understand why they’re used in the pitch. My only concern of macros in general, which does extend to this pitch, is the risk duplicating common functionality in different projects. For example, even after the advent of macros, property wrappers are still a great way of adding behavior to properties in a way most Swift programmers understand. I think this is the case with the proposal’s macros aa they are mainly used for custom operators. However, this behavior is not unique to predicates, the power assert discussed in the expression-macro threads is another great example and I imagine scoped operators being used for numerical computing too. In other words, you made a great case for why predicates would benefit from scoped operators instead of overloading, but we should generalize this feature to extend beyond Foundation. Otherwise the Swift ecosystem will become fragmented with each library author choosing their own version of scoped operators. The following is a simple, generic design we could use for this feature:
macro Predicate<R>(body: () -> R) = #scopedOperators(
OperatorDescriptor(
infix: “+”,
implementation: PredicateExpressions.Equal.init
),
body
)
Quite an interesting idea. A similar idea, but for result builders, was recently pitched here:
Maybe a more general feature could solve both problems. I guess namespace and automatic usages of namespaces could be a nice thing but of course a lot of work to design and implement. I imagine this would also make autocomplete work more seamless.
I actually hadn’t considered a unified result-builder and operator namespacing feature; it’s a great idea!
The design would definitely be time consuming. For one, there’s the question of whether operators outside the namespace are just prioritized over global ones, or completely prohibited (e.g. there’s no bitshift operator in SwiftPredicates). However, the implementation, at least for the prototype in my previous post, was actually quite simple. You just parse the operators given to the macro, visit all operator nodes and substitute with the correct implementation.
Hi all, as mentioned in this pitch, we've posted a followup pitch regarding the serialization behavior of predicates at [Pitch] Swift Predicates: Archiving.
I'm glad to see this proposal went into live with Xcode 15 Beta, but I noticed that the variadic generic APIs are not implemented. In fact the variadic generic semantics are not fully described in the proposal, but it did mention an example of the macro and listed an entry in detailed designs.
So my question is: Is the variadic generic version of Predicate
and #Predicate
still planned? How soon can we use it on Darwin & how soon can we see it in swift-foundation
?
Yes! We still plan to adopt variadic generics / parameter packs for the new Predicate
type. I don't have exact details that I can provide on when it'll appear in Darwin but I've posted a new and updated PR for swift-foundation
at Variadic Generics Support for Predicate by jmschonfeld · Pull Request #178 · apple/swift-foundation · GitHub. I'm just working out a few final details before hopefully landing this.
Glad to see the PR being merged! @jmschonfeld
I've got another question: Why there's no Equatable
and Hashable
conformance to StandardPredicateExpression
s? This is really useful when forming a fact list, etc.
If there's no particular reason for not adding the conformance, I wish to see it before the new OS fall releases, or else we may encounter the ABI hell.
Great question! I've thought about this a little bit when I was working on the initial implementation of Predicate
. I chose not to add this for a few reasons. First, each conformance requirement on the expressions contained within a Predicate
also translate to a conformance requirement on every value captured by a Predicate
. We could require that every captured value be Hashable
(in addition to the current Codable & Sendable
requirement), but that's a choice we'd have to make if Predicate
were also Hashable
/Equatable
.
However, likely more importantly, I didn't feel that the Equatable
conformance for Predicate
would actually be very useful in practice. Delegating down to an expression's Equatable
conformance would check whether the structure of the expression is identical to that of another structure, but not truly equal. For example, consider the following two predicates:
// Predicate A
#Predicate<Bool, Bool, Bool> { ($0 && $1) && $2 }
// Predicate B
#Predicate<Bool, Bool, Bool> { $0 && ($1 && $2) }
These two predicates are semantically equivalent given they will always produce the same results and are equivalent boolean-algebra-wise. However, an Equatable
conformance on the expressions would return that these are not equal because the expressions are actually entirely different types. Since Equatable
doesn't afford for a way to "normalize" the expression tree before checking equality, in practice I suspect that an Equatable
or Hashable
conformance wouldn't produce desired results and likely wasn't worth the additional "cost" of requiring all captured values to be Hashable
/Equatable
as well.