After diving back into UnsafeRawPointer docs, I believe using load(fromByteOffset:as:) would be the correct usage.
assert(a.count * MemoryLayout<UInt8>.stride >= MemoryLayout<Int>.size)
let b = UnsafeRawPointer(a).load(as: Int.self).bigEndian
You'd still have memory alignment consideration to be aware of, but that holds true for previous method as well.