Yes,
That's probably too much... too easy to bypass the protection:
// file.c
#import "Module-Swift.h"
void c_call_claimed_to_be_safe() attribute((SAVE_ATTRIBUTE)) {
somethingVeryUnsafeInSwift()
}
// file.swift
/*safe*/ func swiftSafeCall() {
c_call_claimed_to_be_safe()
}
How is that any different than doing this in Swift?
unsafe func somethingVeryUnsafeInSwift() {
// ...
}
/* safe */ func swiftSafeCall() {
unsafe {
somethingVeryUnsafeInSwift()
}
}
Well, if C/C++/Objective-C follows the same pattern (and I'm afraid they don't), where a safe call can only call other safe calls, the there would be no difference indeed. If not - it would be quite easy to miss unsafely:
void some_other_unmarked_c_call() { // unmarked, treated unsafe, right?
something_very_unsafe()
}
void c_call_claimed_to_be_safe() attribute((SAVE_ATTRIBUTE)) {
some_other_unmarked_c_call() // is this allowed?!
}
In other words without those explicit gate markers in C/C++ land we'd need to audit all callouts done in the calls marked "SAVE_ATTRIBUTE" - this can be huge audit effort.
AFAICS, a function with the safe attribute would work exactly the same as an unsafe block in swift. So you can call other unsafe functions from there (which in C/C++/ObjC is every unmarked function).
The rule for auditing would then be that every unsafe block and every function having the SAFE_ATTRIBUTE would have to be audited. Of course this can be a big effort, but it ought to be kinda expected that thoroughly checking C/C++/ObjC code for safeness is more work than doing the same for swift code.
Personally I agree with Joe that some balance is needed. It's an interesting thing to think about because we want these languages to be usable from Swift, but we also want strict safety checking. We are kind of asking to have our cake and eat it, too.
It's up to the developers of the C/C++/Obj-C library to consider carefully when they add those annotations whether their functions are truly memory-safe for all inputs (fuzzing can help determine this. People need to fuzz more often). We have a similar situation already for nullability annotations - if an Obj-C API promises that a returned pointer will never be null but gets it wrong, I think that will also cause UB from Swift.
I don't think it's UB, it just causes a crash in the bridging machinery. We've seen such crashes in Apple's APIs before. Of course, that may just be Obj-C objects. Raw memory pointers that are improperly null may in fact be UB.
Yep, the behavior of a nonnull ObjC method returning nil to Swift code is defined to be a crash. Or, since the runtime prevents the nil value from ever reaching Swift in the first place, you can consider the behavior outside the realm of any behavior that Swift contemplates, defined or undefined.
To be fair, it wasn't that way in earlier versions of Swift, when we would put unchecked trust in Objective-C annotations. That might be where the question arises.
As someone who is almost certainly responsible for one or more of those broken promises, I appreciate the current behavior. :)
Karl's description above is spot on and needs to inform the discussion here. unsafe is fundamentally about whether there are preconditions for memory safety that aren't being checked by the API, and it should always be a "call to action" where users can do something meaningful to protect themselves.
Given that, I think it does not make sense to default to treating imported APIs as unsafe just because they're not written in Swift. Certainly an arbitrary C / C++ / Objective-C function could scribble over the stack and corrupt the process, but what can a caller do about that? We risk both making the language unusable and undermining the purported value of unsafe by overwhelming people with it.
Instead, unsafe should be largely opt-in for imported functions, the same as we'd expect it to be in Swift. At most, we should infer it for functions that have unbounded pointer arguments. C doesn't normally associate bounds with specific pointer arguments, but maybe we can take advantage of -fbounds-safety annotations to do some caller-side safety checks, which ought to let us import more C APIs as safe under this understanding.
An arbitrary "safe" swift function can also do that 
func foo(count: Int) {
if count != 0 {
foo(count: count - 1)
}
}
I don't know if this is precedented or not: we may have one another more strict colour (or a language mode / compiler setting / lint rule) under which you won't be able calling C/C++/Obj-C or any other unsafe {...} block.
bikeshedding:
utmostSafe func foo() {
unsafe { unsafeCall() } // error
}
/*safe*/ func foo() {
unsafe { unsafeCall() } // ok
}
unsafe func foo() {
unsafeCall() // non brainer
}
With this mode people would be totally protected from unsafety by paying the price of not being able calling C/C++/Obj-C, etc.
As I understand it we are talking here about memory safety only, and yes there are other types of safety ("crash safety", "trap safety", "hang safety", "realtime safety", "bug safety" to name a few).
All useful abstractions are going to have unsafe in them at some level. All of the standard collections APIs are built on UnsafePointers under the hood. All IO is based on unsafe operations. Not being able to call unsafe functions is limiting to the point where it's almost useless. Any safety gained by not being able to call unsafe functions is wasted by the fact that you can only use it on trivial leaf functions that would be completely safe regardless.
Perhaps not the whole app, but some parts of it could be "utmost safe" while still being useful.
The list of "utmost safe" things (things that do not require calling unsafe blocks / C/C++, etc):
† - operation overrides that change from safe to unsafe is the whole new pandora box in its own... we'd need to somehow protect against it.
†† - stack overflow issue aside which (if considered) makes function calls not memory safe to begin with.
I am pretty sure it is possible to implement, say, a tic tac toe algorithm using only utmost safe operations. Or the "life app" algorithm. The chess algorithm. Even the multilayer machine learning training with back propagation, etc - the above list is more than enough for these mentioned tasks and many others.
As for the I/O.... some time ago I implemented a toy app to see what's possible to do without using standard library. That app doesn't use any unsafe operation at all, yet being able doing some sort of "I/O" (video / audio / keyboard/mouse input & output). Granted that environment is quite limited and the app makes some brave and ungrounded assumptions about memory address stability – something swift doesn't guarantee – so take that particular example with a grain of salt.
I mostly agree with what others have said here. I'd like to add that it doesn't have to be a blanket for all imported APIs. For example, iterators are particularly unsafe in Swift, so maybe methods that return iterators should be imported as unsafe. (They also have a clear path to removing their unsafety.)
More generally, I think we could do a pretty good job finding various patterns and determining their safety (and of course API authors could override this).
I sort of agree this should be the goal. But all C functions can access global variables which will include pointers and thus can be unsafe. Maybe it's a good idea to assume they don't do that in general, but it'd be nice if there was an annotation to differentiate what is assumed safe from what we're sure is safe.
We could have three levels: "safe & audited", "assumed safe", and "known unsafe", where "assumed safe" would be the default. It's a bit similar to implicitly unwrapped optionals but unfortunately without a way to check for safety like we can check for nil.
Most Swift code would then fall in the default "assumed safe" category. You'd use the "safe & audited" flag when you actually want to audit things and have the compiler flag any caller not audited yet.
I suppose it could look like this:
// safe & audited
safe func foo() {
safeCall() // ok
trust { assumedSafeCall() } // ok
trust(unsafe) { unsafeCall() } // ok
}
// assumed safe
func foo() {
safeCall() // ok
assumedSafeCall() // ok
trust(unsafe) { unsafeCall() } // ok
}
// known unsafe
unsafe func foo() {
safeCall() // ok
assumedSafeCall() // ok
unsafeCall() // ok
}
This sounds a bit like @tera's "unmost safe" concept, but it's different in that the "safe & audited" can still call unsafe and "assumed safe" functions (within a trust block).
The biggest risk with the general concept of unsafe is source breakage when the unsafely level is increased. This can happen either when something was too eagerly marked "safe & audited" or because something previously assumed safe suddenly becomes "known unsafe". If we want to avoid source breakage, we need to limit what becomes "unsafe" and be very sure of ourselves when marking something "safe & audited".
I don't think it's so grim, consider this example chess algorithm that uses "utmost safe" functions throughout up until I/O is needed (drawing to the screen).
// Simple chess game using upmost safe functions (mostly)
// TODO:
// - pawn promotions
// - castling
// - en passan
// - draw rules
// - checks / mate checks
// - game termination constion
// - good cost estimation
// - various optimizations (alpha beta pruning, etc)
import Foundation
#if canImport(Cocoa)
import Cocoa
#endif
struct Tuple8<T> {
var elements: (T, T, T, T, T, T, T, T)
subscript(_ i: Int) -> T {
get {
precondition(i >= 0 && i < 8)
switch i {
case 0: return elements.0
case 1: return elements.1
case 2: return elements.2
case 3: return elements.3
case 4: return elements.4
case 5: return elements.5
case 6: return elements.6
case 7: return elements.7
default: fatalError()
}
}
set {
precondition(i >= 0 && i < 8)
switch i {
case 0: elements.0 = newValue
case 1: elements.1 = newValue
case 2: elements.2 = newValue
case 3: elements.3 = newValue
case 4: elements.4 = newValue
case 5: elements.5 = newValue
case 6: elements.6 = newValue
case 7: elements.7 = newValue
default: fatalError()
}
}
}
}
typealias Row = Tuple8<Piece>
typealias Rows = Tuple8<Row>
struct Point {
var x, y: Int
init(_ x: Int, _ y: Int) {
self.x = x
self.y = y
}
var valid: Bool {
x >= 0 && x < 8 && y >= 0 && y < 8
}
static func + (lhs: Self, rhs: Self) -> Self {
Point(lhs.x + rhs.x, lhs.y + rhs.y)
}
}
struct Move {
let from: Point
let to: Point
}
enum PieceNominal: Int {
case none, pawn, knight, bishop, rook, queen, king
var cost: Int {
switch self {
case .none: return 0
case .pawn: return 100
case .knight: return 300
case .bishop: return 300
case .rook: return 500
case .queen: return 900
case .king: return 10000000
}
}
}
enum PieceColor: Int {
case black = -1, `none` = 0, white = 1
var opposite: PieceColor {
switch self {
case .black: return .white
case .white: return .black
case .none: return .none
}
}
}
struct Piece {
var nominal: PieceNominal
var color: PieceColor
init(_ nominal: PieceNominal, _ color: PieceColor) {
self.nominal = nominal
self.color = color
}
static let empty = Piece(.none, .none)
static let no = Piece(.none, .none)
static let wp = Piece(.pawn, .white)
static let wn = Piece(.knight, .white)
static let wb = Piece(.bishop, .white)
static let wr = Piece(.rook, .white)
static let wq = Piece(.queen, .white)
static let wk = Piece(.king, .white)
static let bp = Piece(.pawn, .black)
static let bn = Piece(.knight, .black)
static let bb = Piece(.bishop, .black)
static let br = Piece(.rook, .black)
static let bq = Piece(.queen, .black)
static let bk = Piece(.king, .black)
}
var silly = 0
var moveCount = 0
struct Board {
static let initial = Board(rows: Rows(elements: (
Row(elements: (.wr, .wn, .wb, .wq, .wk, .wb, .wn, .wr)),
Row(elements: (.wp, .wp, .wp, .wp, .wp, .wp, .wp, .wp)),
Row(elements: (.no, .no, .no, .no, .no, .no, .no, .no)),
Row(elements: (.no, .no, .no, .no, .no, .no, .no, .no)),
Row(elements: (.no, .no, .no, .no, .no, .no, .no, .no)),
Row(elements: (.no, .no, .no, .no, .no, .no, .no, .no)),
Row(elements: (.bp, .bp, .bp, .bp, .bp, .bp, .bp, .bp)),
Row(elements: (.br, .bn, .bb, .bq, .bk, .bb, .bn, .br))
)))
private var rows: Rows
private subscript(_ p: Point) -> Piece {
get { rows[p.y][p.x] }
set { rows[p.y][p.x] = newValue }
}
private func iterateLongMoves(_ color: PieceColor, _ p0: Point, _ dp: Point, oneStep: Bool = false, _ execute: (Point) -> Void) {
var p = p0
repeat {
p.x += dp.x
p.y += dp.y
if !p.valid { break }
let c = self[p].color
if c == color { break }
execute(p)
if c != .none { break }
} while !oneStep
}
private func iterateKingOrQueenMoves(color: PieceColor, p0: Point, isKing: Bool, _ execute: (Point) -> Void) {
for dy in -1 ... +1 {
for dx in -1 ... +1 {
if !(dx == 0 && dy == 0) {
iterateLongMoves(color, p0, Point(dx, dy), oneStep: isKing, execute)
}
}
}
}
private func iterateQueenMoves(_ color: PieceColor, _ p0: Point, _ execute: (Point) -> Void) {
iterateKingOrQueenMoves(color: color, p0: p0, isKing: false, execute)
}
private func iterateKingMoves(_ color: PieceColor, _ p0: Point, _ execute: (Point) -> Void) {
// TODO: castle
iterateKingOrQueenMoves(color: color, p0: p0, isKing: true, execute)
}
private func iterateBishopMoves(_ color: PieceColor, _ p0: Point, _ execute: (Point) -> Void) {
iterateLongMoves(color, p0, Point(-1, -1), execute)
iterateLongMoves(color, p0, Point(-1, +1), execute)
iterateLongMoves(color, p0, Point(+1, +1), execute)
iterateLongMoves(color, p0, Point(+1, -1), execute)
}
private func iterateKnightMoves(_ color: PieceColor, _ p0: Point, _ execute: (Point) -> Void) {
iterateLongMoves(color, p0, Point(-2, -1), oneStep: true, execute)
iterateLongMoves(color, p0, Point(-2, +1), oneStep: true, execute)
iterateLongMoves(color, p0, Point(-1, +2), oneStep: true, execute)
iterateLongMoves(color, p0, Point(+1, +2), oneStep: true, execute)
iterateLongMoves(color, p0, Point(+2, +1), oneStep: true, execute)
iterateLongMoves(color, p0, Point(+2, -1), oneStep: true, execute)
iterateLongMoves(color, p0, Point(+1, -2), oneStep: true, execute)
iterateLongMoves(color, p0, Point(-1, -2), oneStep: true, execute)
}
private func iterateRookMoves(_ color: PieceColor, _ p0: Point, _ execute: (Point) -> Void) {
iterateLongMoves(color, p0, Point(-1, -0), execute)
iterateLongMoves(color, p0, Point(+0, +1), execute)
iterateLongMoves(color, p0, Point(+1, +0), execute)
iterateLongMoves(color, p0, Point(-0, -1), execute)
}
private func iteratePawnMoves(_ color: PieceColor, _ p0: Point, _ execute: (Point) -> Void) {
// TODO: en passant, promotions
let isWhite = color == .white
let dy = isWhite ? +1 : -1
let dp = Point(0, dy)
let p1 = p0 + dp
if p1.valid && self[p1].color == .none {
execute(p1)
if p0.y == (isWhite ? 1 : 6) {
let p2 = p1 + dp
if p2.valid && self[p2].color == .none {
execute(p2)
}
}
}
let p3 = p0 + Point(-1, dy)
if p3.valid && self[p3].color == color.opposite {
execute(p3)
}
let p4 = p0 + Point(+1, dy)
if p4.valid && self[p4].color == color.opposite {
execute(p4)
}
}
private func iterateMoves(_ piece: Piece, _ p0: Point, _ execute: (Point) -> Void) {
let color = piece.color
switch piece.nominal {
case .none: break
case .pawn: iteratePawnMoves(color, p0, execute)
case .knight: iterateKnightMoves(color, p0, execute)
case .bishop: iterateBishopMoves(color, p0, execute)
case .rook: iterateRookMoves(color, p0, execute)
case .queen: iterateQueenMoves(color, p0, execute)
case .king: iterateKingMoves(color, p0, execute)
}
}
mutating func traverse(color: PieceColor, depth: Int = 1, maxDepth: Int) -> (Move, Int) {
var bestMove = Move(from: Point.init(0, 0), to: Point(0, 0))
var bestCost = -100000
var (y, dy, sentinel) = (color == .white) ? (7, -1, -1) : (0, +1, 8)
while y != sentinel {
for x in 0 ..< 8 {
let p0 = Point(x, y)
let piece0 = self[p0]
if piece0.color == color {
iterateMoves(piece0, p0) { p1 in
precondition(self[p1].color == .none || self[p1].color == color.opposite)
let piece1 = self[p1]
self[p0] = .empty
self[p1] = piece0
var cost = piece1.nominal.cost
if depth < maxDepth {
cost -= traverse(color: color.opposite, depth: depth + 1, maxDepth: maxDepth).1
} else {
// TODO: positional estimate
if moveCount < 10 && piece0.nominal == .pawn {
cost += 3
}
if x > 1 && x < 6 {
cost += 3
}
if moveCount < 20 && piece0.nominal == .rook {
cost -= 5
}
if moveCount < 10 && piece0.nominal == .queen {
cost -= 4
}
if piece0.nominal == .king {
cost -= 7
}
cost += (silly + x + y + depth + moveCount) & 0x03 // some silly randomization
silly += 1
}
if bestCost < cost {
bestMove = Move(from: p0, to: p1)
bestCost = cost
}
self[p1] = piece1
self[p0] = piece0
}
}
}
y += dy
}
return (bestMove, bestCost)
}
mutating func makeMove(_ move: Move) {
// TODO: castle, en passant
self[move.to] = self[move.from]
self[move.from] = .empty
}
}
// ---------------------------------------
// Unsafe area
extension Piece {
/*unsafe*/ var face: String {
switch (nominal, color) {
case (.pawn, .white): return "♙"
case (.pawn, .black): return "♟"
case (.knight, .white): return "♘"
case (.knight, .black): return "♞"
case (.bishop, .white): return "♗"
case (.bishop, .black): return "♝"
case (.rook, .white): return "♖"
case (.rook, .black): return "♜"
case (.queen, .white): return "♕"
case (.queen, .black): return "♛"
case (.king, .white): return "♔"
case (.king, .black): return "♚"
default: return "·"
}
}
}
extension Board {
/*unsafe*/ func draw() {
for y in 0 ..< 8 {
for x in 0 ..< 8 {
let piece = self[Point(x, (8 - y - 1))]
print(piece.face, terminator: " ")
}
print("") // line terminator
}
print()
}
}
/*unsafe*/ func beep() {
#if os(macOS)
NSSound.beep()
#endif
}
/*unsafe*/ func play() {
silly = Int(clock() & 0x7FFFFFFF) // some silly randomization
var board = Board.initial
board.draw()
var color = PieceColor.white
// TODO: game termination
while true {
print("Next move if \(color == .white ? "white" : "black")\n")
#if DEBUG
let maxDepth = 4
#else
let maxDepth = 5
#endif
let (move, cost) = board.traverse(color: color, maxDepth: maxDepth)
beep()
board.makeMove(move)
board.draw()
color = color.opposite
if color == .white {
moveCount += 1
}
}
}
play()
It's doing an unoptimised brute force and quite a few bits and pieces are missing game wise (promotions, mate checks, etc). Possibly there are a few bugs. Release mode is recommended and max depths up to 5 or 6.
I've been thinking about this for some time and may now found a solution.
Here I'll be talking now about "unsafe" v "safe" (and potentially v "utmostSafe") colouring only, but it could be applied to other function colouring as well.
Consider an unsafe call example:
var inited = false
unsafe func foo_unsafe(...) {
if !inited {
inited = true
malloc(...)
some_unsafe_call()
}
while condition1 {
...
if condition2 {
some_unsafe_call()
}
...
// some other safe code
}
}
or anything similar and as complex as possible. As you've said we may have some knowledge about this code that compiler doesn't have, for example we know that at the time we want to use this call – unsafe paths will never be taken, so we put this call in the "unsafe" / "safeUnchecked" / "trusted" block (the name is under discussion):
/*safe*/ func bar() {
safeUnchecked {
// Note to the auditor:
// I swear this call is safe under the current conditions:
foo_unsafe(...)
}
}
Now imagine that compiler is autogenerating an extra functions (as a result of some explicit opt-in, so it doesn't do this unnecessary):
var inited = false
/*safe*/ func foo_safe(...) {
if !inited {
inited = true
fatalError() // unsafe code substituted
fatalError() // unsafe code substituted
}
while condition1 {
...
if condition2 {
fatalError() // unsafe code substituted
}
...
// some other safe code
}
}
with some method of name disambiguation which here is shown as bikesheded "_safe" suffix attached.
Now we can change our original bar to be safe without the need of "trust me" block:
/*safe*/ func bar() {
foo_safe(...)
}
and if we do so we won't have the ability of lying or be mistaken!
Equally, if we do have a further distinction between "safe" and "utmostSafe" functions, then given a "safe" function:
/*safe*/ func baz_safe() {
some_utmostSafe_code()
if condition {
safeUnchecked {
// but we know it will never trigger unsafe parts
some_unsafe_code()
}
}
some_other_utmostSafe_code()
}
compiler could autogenerate the "utmostSafe" version:
utmostSafe func baz_utmostSafe() {
some_utmostSafe_code() // utmost safe - so ok
if condition {
fatalError() // "just" safe so substituted
}
some_other_utmostSafe_code() // utmost safe - so ok
}
and now we'd be able calling the "utmost safe" version of baz() from "utmost safe" contexts.
This feels a bullet proof solution to me.
Edit: a more complete example that turns unsafe code into safe.
The transformation algorithm goes as follows:
/*safe*/ func main_safe() {
some_code_safe()
TRUST_ME_THIS_WONT_TRIGGER_UNSAFE_PATHS_I_SWEAR {
op1_unsafe()
}
}
unsafe func op1_unsafe() {
some_code_safe()
if condition {
op2_unsafe()
}
}
unsafe func op2_unsafe() {
some_code_safe()
if condition {
malloc(...) // "leaf" unsafe code
}
}
/*safe*/ func main_safe() {
some_code_safe()
op1_safe()
}
/*safe*/ func op1_safe() {
some_code_safe()
if condition {
op2_safe()
}
}
/*safe*/ func op2_safe() {
some_code_safe()
if condition {
fatalError()
// "leaf" / C / C++ / compiled / etc function was here,
// so we'd just replace it with "fatalError" without drilling any deeper.
// if this path is triggered the assumption that "unsafe paths won't be
// triggered" was false, but with "fatalError" here we know for sure
// that unsafe code will not be triggered.
}
}
We started with unsafe code (or the code that's claimed to be safe but safety of which was unchecked), and after this simple transformation we've got a safe version of this code, safety of which is now checked by the compiler. It's obvious how to do this transformation manually, and hopefully its equivalent could be performed by the compiler.
The fact that Array and String are banned by your “utmost safe” feature is a serious flaw, and in my opinion a disqualifying one. Sure, it’s possible to work around them being banned, but what is the advantage of doing so? How does it improve the code? If there is some sort of improvement, does it outweigh the costs of having to invent nonstandard collection types that aren’t used by anything else?
My sense is that nothing of actual value is gained and much is lost by avoiding or limiting use of these common-currency types simply because they are implemented with unsafe operations.
That's not just about collections: the whole host of potential bugs could be eliminated if you are under a restrictive colour that prohibits using C/C++/etc. As for the standard collections: some algorithms don't need collections (standard or otherwise) at all, and some can't use standard collections even if they wanted to (e.g. realtime code). For the purposes of the latter we already have @nolocks and @noallocations; perhaps there's some cross between those and "utmost safe".
If to move these discussed colour schemes to their logical conclusions (including other flavours of safety I mentioned above), we may end up having a robust eco system where if you don't have a certain level of colour "clearance" you won't be able deploying the code to certain environments (realtime code for aircraft or a nuclear power station). Normal user apps won't need those (except maybe those using swift in realtime audio code).
Here’s the thing: “no locks” and “no allocations” exist because there are scenarios in which use of locks or allocations is bad and avoiding them is good. Nobody has articulated any situation in which using operations implemented with unsafe primitives is bad and avoiding them is good. It’s like having a function color for @usesEvenNumberOfCPUInstructions—sure, that’s a property you could theoretically enforce, but what’s the point?