@Alejandro has just posted a pitch for Atomic<T> on the evolution forum -- the text has some examples that illuminate the concept a little.
You can do atomic operations on borrowed Atomic values, despite the fact that they may change the stored value. (As Karl mentioned, this is exactly what the accepted revision of SE-0282 was mostly about -- it introduced the idea of "atomic" access to avoid having to do galaxy-brain mental gymnastics like categorizing certain mutating operations as "read-only".)
In fact, borrowing operations are the only way to perform concurrent atomic access in Swift. If we defined any consuming or mutating operations on Atomic<T>, then it would not be possible to call them concurrently -- by definition, any concurrent, overlapping calls would count as exclusivity violations. The consuming/mutating modifiers are explicitly incompatible with overlapping concurrent use, by design.
(This doesn't necessarily mean that such operations would be entirely useless, though. For example, we've been playing with the idea of adding a consuming func dispose() operation that returned the "final" value of the atomic cell before destroying it. However, it doesn't seem this would carry its weight, and the restrictions on its use would very likely be too confusing to consider.)