Yes! We still plan to adopt variadic generics / parameter packs for the new Predicate type. I don't have exact details that I can provide on when it'll appear in Darwin but I've posted a new and updated PR for swift-foundation at Variadic Generics Support for Predicate by jmschonfeld · Pull Request #178 · apple/swift-foundation · GitHub. I'm just working out a few final details before hopefully landing this.
3 Likes
Glad to see the PR being merged! @jmschonfeld
I've got another question: Why there's no Equatable and Hashable conformance to StandardPredicateExpressions? This is really useful when forming a fact list, etc.
If there's no particular reason for not adding the conformance, I wish to see it before the new OS fall releases, or else we may encounter the ABI hell.
2 Likes
Great question! I've thought about this a little bit when I was working on the initial implementation of Predicate. I chose not to add this for a few reasons. First, each conformance requirement on the expressions contained within a Predicate also translate to a conformance requirement on every value captured by a Predicate. We could require that every captured value be Hashable (in addition to the current Codable & Sendable requirement), but that's a choice we'd have to make if Predicate were also Hashable/Equatable.
However, likely more importantly, I didn't feel that the Equatable conformance for Predicate would actually be very useful in practice. Delegating down to an expression's Equatable conformance would check whether the structure of the expression is identical to that of another structure, but not truly equal. For example, consider the following two predicates:
// Predicate A
#Predicate<Bool, Bool, Bool> { ($0 && $1) && $2 }
// Predicate B
#Predicate<Bool, Bool, Bool> { $0 && ($1 && $2) }
These two predicates are semantically equivalent given they will always produce the same results and are equivalent boolean-algebra-wise. However, an Equatable conformance on the expressions would return that these are not equal because the expressions are actually entirely different types. Since Equatable doesn't afford for a way to "normalize" the expression tree before checking equality, in practice I suspect that an Equatable or Hashable conformance wouldn't produce desired results and likely wasn't worth the additional "cost" of requiring all captured values to be Hashable/Equatable as well.
5 Likes
Oh, that’s indeed a downside of it! I suspect this could break a lot of usages.
This is expected from my side. Equatable doesn’t mean to be semantically identical (i.e. behaves exactly the same) — I would like the functionality of matching, let’s say, structurally identical predicates as possible. Semantic equality doesn’t generally make sense because such associative law doesn’t apply to all operators.
Here’s a possible solution: Can we just add default Equatable and Hashable conformance, without modifying StandardPredicateExpression? Default conformance of the protocols can only be added in the same file, so it’s important to make them into Foundation.
Ah interesting. Perhaps an example might help then, are you able to describe a bit more about the use case for when you think this might be pertinent? I'm worried about providing this API in a manner that might suggest a semantic equivalence that behaves as a structural/identity equivalence. But, if there is a great use case for this that I haven't thought of yet, I'd definitely like to hear about it!
I think I have a similar thought here - at a technical level we can, however it might be good to discuss what the use case is for this. If individual expressions are Equatable and Hashable, but StandardPredicateExpression and Predicate itself are not, is that useful? I suppose you could use it if you're storing pieces of expressions separately, but I think that use case might not be very common so I'd love to hear more about it before deciding on adding this conformance. I think my main concern currently is that the potential utility of the conformance doesn't quite seem to outweigh the possibility of confusion around its use and the restrictions it imposes (in my eyes currently).
So we are looking at migrating to #Predicates from our own internal implementation - one question we have is what seems to be the lack of support for dynamically building predicates?
Very specifically, let's say if we (or the SwiftUI team!) would like to build NSPredicateEditor | Apple Developer Documentation for the new #Predicate, how would that be done?
1 Like
Presumably for anything dynamic you'd have to drop down to constructing the underlying predicate types directly instead of relying on the macro. Those appear to be nested in the PredicateExpressions namespace; I think this list would be a good start: PredicateExpression | Apple Developer Documentation
2 Likes
Yep, @allevato's point above is correct. The macro is designed around constructing a predicate based on static swift code. If you'd like to generate a predicate's contents dynamically (based on input from an editor, for example) you'll want to implement that logic to initialize the predicate expression types directly without using the macro. For this, you can directly call the Predicate initializer that takes a closure, and within that closure (which will provide you with the appropriate PredicateExpression.Variables to use for your inputs), you can dynamically return different expressions based on other input from your editor.
3 Likes
Hi!
Thank you for the reply.
We played a bit with #Predicate macro and a builder closure, but it still not clear how we could build predicate with dynamic expressions. For example we have to create a predicate for
struct Monster {
let level: Int
let name: String
}
to find all monsters with particular level and name, or level, or name, so we could use the #Predicate macro like:
func makePredicate(level: Int?, name: String?) -> Predicate<Monster> {
#Predicate<Monster> {
((level == nil) || ($0.level == level!))
&& ((name == nil) || ($0.name == name!))
}
}
That macro expands to
Foundation.Predicate<Monster>({
PredicateExpressions.build_Conjunction(
lhs: PredicateExpressions.build_Disjunction(
lhs: PredicateExpressions.build_Equal(
lhs: PredicateExpressions.build_Arg(level),
rhs: PredicateExpressions.build_NilLiteral()
),
rhs: PredicateExpressions.build_Equal(
lhs: PredicateExpressions.build_KeyPath(
root: PredicateExpressions.build_Arg($0),
keyPath: \.level
),
rhs: PredicateExpressions.build_ForcedUnwrap(
PredicateExpressions.build_Arg(level)
)
)
),
rhs: PredicateExpressions.build_Disjunction(
lhs: PredicateExpressions.build_Equal(
lhs: PredicateExpressions.build_Arg(name),
rhs: PredicateExpressions.build_NilLiteral()
),
rhs: PredicateExpressions.build_Equal(
lhs: PredicateExpressions.build_KeyPath(
root: PredicateExpressions.build_Arg($0),
keyPath: \.name
),
rhs: PredicateExpressions.build_ForcedUnwrap(
PredicateExpressions.build_Arg(name)
)
)
)
)
})
So we check level and name for nil each time we evaluate predicate, which is probably not good if we have millions of monsters. In that particular case for the very synthetic example we could write it like:
func makePredicate(level: Int?, name: String?) -> Predicate<Monster> {
if let level, let name {
return #Predicate<Monster> {
($0.level == level) && ($0.name == name)
}
} else if let level {
return #Predicate<Monster> {
($0.level == level)
}
} else if let name {
return #Predicate<Monster> {
($0.name == name)
}
}
}
But what if we have more than 2 conditions and we want to have different logical operation?
The idea was to add the code checking level and name for nil inside the predicate building closure and create different expressions. I tried many different things but not one compiled 
Could you please help us and show how we can dynamically build a predicate expressions inside the predicate building closure for the example above?
1 Like
Hey @ser-0xff, thanks for the additional explanation! The last approach you have there would work as you mentioned, but as you add more properties it would become much more complex. If you need to dynamically create a predicate while analyzing what should go into the predicate, you can do so by manually constructing the expression tree. Unfortunately, since this is a more advanced use case you wouldn't be able to use the macro to help here, but you could write something along the lines of the following:
func makePredicate(level: Int?, name: String?) -> Predicate<Monster> {
func buildConjunction(lhs: some StandardPredicateExpression<Bool>, rhs: some StandardPredicateExpression<Bool>) -> any StandardPredicateExpression<Bool> {
PredicateExpressions.Conjunction(lhs: lhs, rhs: rhs)
}
return Predicate<Monster>({ input in
var conditions: [any StandardPredicateExpression<Bool>] = []
if let level {
conditions.append(
PredicateExpressions.Equal(
lhs: PredicateExpressions.KeyPath(root: input, keyPath: \Monster.level),
rhs: PredicateExpressions.Value(level)
)
)
}
if let name {
conditions.append(
PredicateExpressions.Equal(
lhs: PredicateExpressions.KeyPath(root: input, keyPath: \Monster.name),
rhs: PredicateExpressions.Value(name)
)
)
}
guard let first = conditions.first else {
return PredicateExpressions.Value(true)
}
let closure: (any StandardPredicateExpression<Bool>, any StandardPredicateExpression<Bool>) -> any StandardPredicateExpression<Bool> = {
buildConjunction(lhs: $0, rhs: $1)
}
return conditions.dropFirst().reduce(first, closure)
})
}
In short, we create a list of the conditions that need to be met and build up the list based on which parameters are specified to the makePredicate function. We can then reduce this array into a single tree of conjunctions to ensure that all of the conditions are met. There are a few small hoops to jump through here in order to satisfy the type-checker with the use of generics such as the closure and separate buildConjunction function, but this allows you to just append to conditions for each new property rather than needing to work with a combinatorial explosion of conditions using the macro.
4 Likes
Thank you very much for the reply!
Using an intermediate generic buildConjunction() function looks like some magic, never would try that.
Not clear to me why it does not compile if I construct PredicateExpressions.Conjunction in the closure directly, while it is a generic as well...
Equality implies substitutability—any two instances that compare equally can be used interchangeably in any code that depends on their values.
Also, the literal dictionary definition of 'equal' is "uniform in application or effect".
But:
To maintain substitutability, the
==operator should take into account all visible aspects of anEquatabletype.
Of course that hinges on what you consider "visible". It isn't as simple as technically visible - Array and Set have a capacity, for example, not just their actual contents, and that capacity does have a visible effect on their behaviour yet is not factored into their hashing or equality. It's considered not essential to their function, because it doesn't technically change the outcome of any operations, just its speed.
Now, some people might care about the speed (for good and valid reasons), but it's uncommon, so they're the ones that get lumped with extra work to account for it (if a == b && a.capacity == b.capacity etc).
I think what you're asserting is that for your purposes the structure of the predicate is what you care about, not the predicate's effect. i.e. you consider the structure meaningfully visible. Which is fine, but I think not what most people want. As @jmschonfeld suggested, I expect that most people care only whether predicate a and predicate b will produce the same result for every given input.
I know that in some domains performance is really important, e.g. relational databases where there are many ways of writing the same effective query but they can differ in performance massively. Still, I don't think domain-specific concerns should dominate the broad definition of equality, because:
- There's no realistic way to determine abstractly whether two predicates are of universally equivalent performance (and determining that even just empirically can be non-trivial).
- Practically-speaking if you really care about performance that much, you're most likely to address it procedurally, e.g. by running a predicate optimiser, and/or educationally, by teaching your users which forms are superior.
- You'd likely want a way to compare predicates for behavioural equality anyway, e.g. as a validation check to ensure any transformations don't mistakenly alter the predicate's behaviour.
Now, the problem is that determining whether two predicates are behaviourally equivalent might also be hard. So that might make it impractical to provide that equality comparison at all. Even so, I think having no Equatable support is better than falling back to exact tree comparison, because it's going to be noisy with false negatives.
It does indeed seem a little unintuitive, but the behavior comes from the combination of unwrapping existentials and calling generic functions. PredicateExpressions.Conjunction is a generic type where the generic type parameters are inferred from the parameters to the initializer. This means that in order to form the return type of the initializer (i.e. PredicateExpressions.Conjunction<SomeLHSType, SomeRHSType>) the parameter types need to be statically known. However, in the closure, the parameter types are existentials, so their concrete types are not statically known. Instead we create a helper function which uses the some keyword so that the concrete types of these parameter inputs are statically known. This means that we can construct the concrete PredicateExpressions.Conjunction type with its known type parameters, and then box it in an existential and return the value. We are able to call buildConjunction from closure because the compiler understands how to implicitly unwrap existentials and call generic functions with their values thanks to the features added in SE-0352.
2 Likes
Just one quick feedback before we try to dig further down into it, but initial samples shows that the new Predicates are around 10x slower than our current implementation for fairly simple use cases, it seems key path usage is a major factor here (see two instances of swift_getAtKeyPath in the sample below highlighted with ***** consuming the majority of the time):
2.99 Gc 100.0% - PredicateBenchmark (91361)
2.99 Gc 100.0% - completeTaskAndRelease(swift::AsyncContext*, swift::SwiftError*)
2.99 Gc 100.0% - specialized thunk for @escaping @convention(thin) @async () -> ()
2.99 Gc 100.0% - static BenchmarkRunnerHooks.main()
2.99 Gc 100.0% - BenchmarkRunner.run()
2.99 Gc 100.0% - BenchmarkExecutor.run(_:)
2.98 Gc 99.7% - Benchmark.run()
2.98 Gc 99.7% - closure #1 in Benchmark.init(_:configuration:closure:setup:teardown:)
2.98 Gc 99.7% - partial apply for closure #4 in closure #1 in variable initialization expression of benchmarks
2.97 Gc 99.3% - closure #4 in closure #1 in variable initialization expression of benchmarks
2.86 Gc 95.6% - Predicate.evaluate(_:)
2.33 Gc 77.8% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Disjunction<A, B>
2.32 Gc 77.6% - PredicateExpressions.Disjunction.evaluate(_:)
1.18 Gc 39.6% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Disjunction<A, B>
1.18 Gc 39.4% - PredicateExpressions.Disjunction.evaluate(_:)
1.16 Gc 38.8% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Equal<A, B>
1.15 Gc 38.3% - PredicateExpressions.Equal.evaluate(_:)
1.08 Gc 36.2% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.KeyPath<A, B>
1.08 Gc 35.9% - PredicateExpressions.KeyPath.evaluate(_:)
***** 803.31 Mc 26.8% - swift_getAtKeyPath
214.28 Mc 7.1% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Variable<A>
41.86 Mc 1.3% - destroy for Transaction
6.00 Mc 0.2% 6.00 Mc PredicateExpressions.KeyPath.evaluate(_:)
3.75 Mc 0.1% - swift_bridgeObjectRelease
3.00 Mc 0.1% - PredicateExpressions.Variable.evaluate(_:)
2.00 Mc 0.0% - KeyPath._projectReadOnly(from:)
1.00 Mc 0.0% - swift_release
6.00 Mc 0.2% - ___chkstk_darwin
1.00 Mc 0.0% - swift_getAtKeyPath
1.00 Mc 0.0% - destroy for Transaction
33.67 Mc 1.1% - protocol witness for static Equatable.== infix(_:_:) in conformance <A> A?
9.00 Mc 0.3% - swift::metadataimpl::ValueWitnesses<swift::metadataimpl::NativeBox<unsigned long long, 8ul, 8ul, 8ul>>::initializeWithCopy(swift::OpaqueValue*, swift::OpaqueValue*, swift::TargetMetadata<swift::InProcess> const*)
7.00 Mc 0.2% - _platform_memmove
5.98 Mc 0.2% 5.98 Mc PredicateExpressions.Equal.evaluate(_:)
3.00 Mc 0.1% - DYLD-STUB$$memcpy
2.00 Mc 0.0% - swift_getAssociatedTypeWitness
1.00 Mc 0.0% - PredicateExpressions.Value.evaluate(_:)
1.00 Mc 0.0% - PredicateExpressions.KeyPath.evaluate(_:)
6.00 Mc 0.2% - ___chkstk_darwin
3.00 Mc 0.1% - swift::metadataimpl::ValueWitnesses<swift::metadataimpl::NativeBox<unsigned long long, 8ul, 8ul, 8ul>>::destroy(swift::OpaqueValue*, swift::TargetMetadata<swift::InProcess> const*)
2.00 Mc 0.0% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Value<A>
2.00 Mc 0.0% - protocol witness for static Equatable.== infix(_:_:) in conformance <A> A?
1.00 Mc 0.0% - DYLD-STUB$$swift_getAssociatedTypeWitness
1.00 Mc 0.0% - protocol witness for static Equatable.== infix(_:_:) in conformance AutoreleasingUnsafeMutablePointer<A>
11.31 Mc 0.3% - initializeWithCopy for PredicateExpressions.Disjunction
3.28 Mc 0.1% - destroy for PredicateExpressions.Disjunction
3.02 Mc 0.1% - destroy for PredicateExpressions.Equal
1.00 Mc 0.0% - PredicateExpressions.Equal.evaluate(_:)
1.00 Mc 0.0% 1.00 Mc PredicateExpressions.Equal.evaluate(_:)
1.00 Mc 0.0% 1.00 Mc PredicateExpressions.Disjunction.evaluate(_:)
3.00 Mc 0.1% - destroy for PredicateExpressions.Equal
1.00 Mc 0.0% - ___chkstk_darwin
1.05 Gc 35.2% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Conjunction<A, B>
1.05 Gc 34.9% - PredicateExpressions.Conjunction.evaluate(_:)
835.15 Mc 27.9% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Equal<A, B>
829.55 Mc 27.7% - PredicateExpressions.Equal.evaluate(_:)
760.72 Mc 25.4% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.KeyPath<A, B>
754.72 Mc 25.2% - PredicateExpressions.KeyPath.evaluate(_:)
***** 548.11 Mc 18.3% - swift_getAtKeyPath
180.71 Mc 6.0% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Variable<A>
18.23 Mc 0.6% - destroy for Transaction
3.67 Mc 0.1% - swift_release
2.00 Mc 0.0% - swift_getAssociatedTypeWitness
1.00 Mc 0.0% - KeyPath._projectReadOnly(from:)
1.00 Mc 0.0% - PredicateExpressions.Variable.evaluate(_:)
1.00 Mc 0.0% - destroy for Transaction
1.00 Mc 0.0% - ___chkstk_darwin
1.00 Mc 0.0% - DYLD-STUB$$swift_getAtKeyPath
1.00 Mc 0.0% - swift_getAtKeyPath
1.00 Mc 0.0% 1.00 Mc protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.KeyPath<A, B>
1.00 Mc 0.0% - destroy for Transaction
54.83 Mc 1.8% - protocol witness for static Equatable.== infix(_:_:) in conformance <A> A?
3.00 Mc 0.1% - DYLD-STUB$$memcpy
3.00 Mc 0.1% - _platform_memmove
2.00 Mc 0.0% - swift_getAssociatedTypeWitness
2.00 Mc 0.0% - swift::metadataimpl::ValueWitnesses<swift::metadataimpl::NativeBox<unsigned long long, 8ul, 8ul, 8ul>>::destroy(swift::OpaqueValue*, swift::TargetMetadata<swift::InProcess> const*)
1.00 Mc 0.0% - protocol witness for static Equatable.== infix(_:_:) in conformance AutoreleasingUnsafeMutablePointer<A>
1.00 Mc 0.0% - pod_copy(swift::OpaqueValue*, swift::OpaqueValue*, swift::TargetMetadata<swift::InProcess> const*)
1.00 Mc 0.0% - swift::metadataimpl::FixedSizeBufferValueWitnesses<swift::metadataimpl::ValueWitnesses<swift::metadataimpl::NativeBox<unsigned long long, 8ul, 8ul, 8ul>>, true, 8ul, 8ul, false>::getEnumTagSinglePayload(swift::OpaqueValue const*, unsigned int, swift::TargetMetadata<swift::InProcess> const*)
1.00 Mc 0.0% 1.00 Mc PredicateExpressions.Equal.evaluate(_:)
2.00 Mc 0.0% - ___chkstk_darwin
2.00 Mc 0.0% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Value<A>
1.00 Mc 0.0% - pod_destroy(swift::OpaqueValue*, swift::TargetMetadata<swift::InProcess> const*)
596.61 Kc 0.0% - DYLD-STUB$$swift_getAssociatedTypeWitness
152.54 Mc 5.1% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Disjunction<A, B>
26.74 Mc 0.8% - initializeWithCopy for PredicateExpressions.Conjunction
11.00 Mc 0.3% - destroy for PredicateExpressions.Equal
7.00 Mc 0.2% - destroy for PredicateExpressions.Conjunction
5.00 Mc 0.1% - destroy for PredicateExpressions.Disjunction
4.00 Mc 0.1% 4.00 Mc PredicateExpressions.Conjunction.evaluate(_:)
2.00 Mc 0.0% - pod_destroy(swift::OpaqueValue*, swift::TargetMetadata<swift::InProcess> const*)
1.00 Mc 0.0% - destroy for PredicateExpressions.KeyPath
1.00 Mc 0.0% - DYLD-STUB$$swift_release
3.00 Mc 0.1% - destroy for PredicateExpressions.Conjunction
3.00 Mc 0.1% - destroy for PredicateExpressions.Disjunction
1.00 Mc 0.0% - protocol witness for PredicateExpression.evaluate(_:) in conformance PredicateExpressions.Disjunction<A, B>
1.00 Mc 0.0% - ___chkstk_darwin
43.87 Mc 1.4% - initializeWithCopy for PredicateExpressions.Disjunction
23.00 Mc 0.7% - destroy for PredicateExpressions.Disjunction
8.00 Mc 0.2% 8.00 Mc PredicateExpressions.Disjunction.evaluate(_:)
5.00 Mc 0.1% - destroy for PredicateExpressions.Equal
3.00 Mc 0.1% - destroy for PredicateExpressions.Conjunction
1.00 Mc 0.0% - swift_release
1.00 Mc 0.0% - destroy for PredicateExpressions.KeyPath
3.00 Mc 0.1% - destroy for PredicateExpressions.Disjunction
2.00 Mc 0.0% - ___chkstk_darwin
1.00 Mc 0.0% - initializeWithCopy for PredicateExpressions.Disjunction
250.80 Mc 8.3% - PredicateBindings.init<each A>(_:)
102.35 Mc 3.4% - bool swift::RefCounts<swift::RefCountBitsT<(swift::RefCountInlinedness)1>>::doDecrementSlow<(swift::PerformDeinit)1>(swift::RefCountBitsT<(swift::RefCountInlinedness)1>, unsigned int)
63.10 Mc 2.1% - __swift_instantiateGenericMetadata
...
This ties together with my question here:
I think that performance of predicates is a crucial feature as mentioned earlier up thread, without it many use cases are precluded - it seems key paths throws a spanner in the wheel here - are you aware of this performance problem and aware of any efforts to alleviate it by improving keypaths?
We will try to create proper issues on this if needed, but the simple benchmark has code like this:
func makeFoundationPredicate(_ frostflake: Frostflake, userId: UserIdentifier, accountId: AccountIdentifier) -> Predicate<Transaction> {
let wrongId = frostflake.generate()
// let now = InternalUTCClock.now
// let secBefore = now.advanced(by: .seconds(-1))
// let secAfter = now.advanced(by: .seconds(1))
return #Predicate<Transaction> { transaction in
((transaction.user == userId) && ((transaction.account == accountId) || (transaction.account == wrongId)))
|| ((transaction.account == accountId)
|| ((transaction.id == wrongId) /*|| (transaction.created >= secBefore) && (transaction.created <= secAfter)*/))
}
}
...
Benchmark("FoundationPredicate #1 - evaluate", configuration: .init(scalingFactor: .kilo)) { benchmark in
let users = (1...5).map { _ in UserIdentifier(frostflake.generate()) }
let accounts = (1...5).map { _ in AccountIdentifier(frostflake.generate()) }
let transactions = makeTransactions(count: 100, users, accounts)
let predicate = makeFoundationPredicate(frostflake, userId: users[0], accountId: accounts[0])
var matched = 0
benchmark.startMeasurement()
for idx in benchmark.scaledIterations {
if try predicate.evaluate(transactions[idx % transactions.count]) {
matched += 1
}
}
benchmark.stopMeasurement()
blackHole(matched)
}
EDIT:
Another sincere question, are key paths truly required to implement predicates evaluation? (we primarily care about the hot evaluation path, not so much about serialisation / inspection / setup)
7 Likes
Thanks for the report! We've been getting quite a few reports, questions, and concerns regarding keypath performance. Joe mentioned in a few other threads a while back that the runtime implementation of keypath projection could use an eye for performance to really fix some of the things it's doing currently.
If you could make a small reproducer that would be really useful. The snippet you provided is something we can use, but types like Frostflake, UserIdentifier, and AccountIdentifier are unknown to us and keypath performance may depend on how those types are built in their respectful module.
3 Likes
Thanks Alejandro, that’s super, we will package together a nice package with a few reproducible benchmarks asap, and will post a follow up then.
(Those types are basically uint64:s)
2 Likes
We prepared a reproducer with few benchmarks, please have a look here.
Details are in the PR description.
3 Likes
The PR has been reworked a bit now and now clearly reproducing the issue (and shows a fair number of mallocs for a predicate evaluation which would be nice to avoid), looking forward to any feedback there (or here :-)!
1 Like
Hello,
I'm attempting to parse predicates into trees as discussed in this post: [Pitch] Swift Predicates - #34 by dgoldsmith. The end goal is being able to modify the tree and create arbitrary expressions, but for now it's just parsing and recreating. The tree walking approach seems to mostly work, with one critical exception - variables.
As a first attempt I tried to extract all properties of each predicate node I visited when walking the tree so I could later pass those back into their respective initializer. E.g., for a conjunction I would save lhs and rhs in the tree node, so I could recreate the expression with Conjuncion(lhs: lhs, rhs: rhs).
But how can Variable be created? It has a key property of type VariableID, which seems to be a numeric identifier, but Variable has no initializer.
I then attempted to just copy each node and using them as is without attempting to recreate them. This works for things like Conjunction but not for Variable. Trying to evaluate a predicate with a copied Variable results in an exception Encountered an undefined variable. I assume there's some internal state here that binds the variables to each identifier, but how can I capture that?
I've written a sample project showing the issue, available at https://github.com/ordo-one/external-reproducers/tree/main/swift/predicate-parser.
1 Like
A VariableID is indeed a numeric identifier (internally, it's just a UInt currently but that implementation detail isn't exposed directly to clients). However it's also important that not only is it an identifier, but it's a unique identifier (within a given process). In the future we'd like to leave the door open for the ability to nest predicates within each other, so having unique VariableIDs is critical to ensure that you can't end up with a Predicate that contains two variables that use the same ID. For this reason, VariableID is not directly able to be initialized, but rather whenever you initialize a Predicate, we create Variable instances to represent each input and provide them to the closure. So just like you store an expression and re-initialize it with it's LHS and RHS when creating the new Predicate, you'll also need to map the old PredicateExpressions.Variable instances to the newly provided instances which will represent the variables in your new Predicate. It might not feel very straightforward at first, but requiring this mapping of variables ensures that we don't end up with multiple variables with the same ID and helps to reduce the chance that you end up with a Predicate that contains mismatched variables.
2 Likes