I'm not sure if this limitation could be lifted in cases where the compiler can locally prove the instance is a class, but at least right now you need to indicate your protocol is class based.
Declare protocol P : AnyObject {} and you can assign with reference semantics.
I'm aware that declaring a protocol like P : AnyObject or P : class will yield the expected behaviour, but I would also expect the reference semantics even without doing that when the instance is known to be of a reference type.
From the extension's point of view, it doesn't know whether it's getting applied to a class or not. If you define a mutating operation in the extension, it could wholesale replace the reference:
extension P {
mutating func swap(with x: inout Self) {
let tmp = x
self = tmp
x = tmp
}
}
and since we can't rule out that possibility from the call site, the reference you invoke the method on must be mutable.