How to handle LAPACK errors from matrix inversion

I'm using LAPACK via Accelerate to invert a matrix. In the code below, the matrix is represented as a Matrix struct and the inversion is performed by the inverseMatrix function. A precondition is used to ensure the input matrix is square. The error variable stores the INFO output from the dgetrf and dgetri LAPACK functions.

The error variable indicates if the matrix inversion is successful or not. The error may also indicate if the matrix is ill-conditioned therefore the inverse should not be used. Right now I'm just printing out the integers that represent these errors. Is there a better way to handle these errors and convey to the user what the error actually means?

Also, the matrix inversion may fail so how should I handle that failure? Should I make the return value optional and return nil if it fails? Or is there some other way I should handle it?

Oh, one more thing, I build/run the code with the ACCELERATE_NEW_LAPACK=1 and ACCELERATE_LAPACK_ILP64=1.

import Accelerate

struct Matrix<T> {
    let rows: Int
    let columns: Int
    var data: [T]

    func copy() -> Matrix {
        Matrix(rows: self.rows, columns: self.columns, data: self.data)
    }
}

func inverseMatrix(_ a: Matrix<Double>) -> Matrix<Double> {
    precondition(a.rows == a.columns, "Matrix must be square")

    var mat = a.copy()
    var n = a.rows
    var pivots = [Int](repeating: 0, count: a.rows)
    var workspace = [Double](repeating: 0.0, count: a.rows)
    var error = 0

    // Perform LU decomposition
    dgetrf_(&n, &n, &mat.data, &n, &pivots, &error)
    if error != 0 {
        print("LU decomposition error: \(error)")
        error = 0
    }

    // Calculate inverse based on LU decomposition
    dgetri_(&n, &mat.data, &n, &pivots, &workspace, &n, &error)
    if error != 0 {
        print("Inversion error: \(error)")
    }

    return mat
}

// --- Examples

let A = Matrix<Double>(rows: 2, columns: 2, data: [1, 2, 3, 4])
let inverseA = inverseMatrix(A)
print(inverseA, "\n")

let B = Matrix<Double>(rows: 3, columns: 3, data: [1, 2, 3, 4, 5, 6, 7, 8, 9])
let inverseB = inverseMatrix(B)
print(inverseB, "\n")
1 Like

First, I would note that there are two different types of errors diagnosed by LAPACK. There are structural program errors (matrix dimensions do not match up, leading dimensions are too small, illegal character parameters, etc). These are indicated by negative info results, and are best diagnosed immediately via preconditionFailure, because they indicate that the API is being used incorrectly--this is a serious program error in your wrappers that should be fixed.

The other class of errors are numerical errors, indicated by positive info results; for instance, in dgetrf, a positive info indicates that the matrix was singular, and therefore LU factorization could not be completed. There are several ways that you might expose these idiomatically in Swift, and which one is "correct" is mostly a matter of taste and the expected usage patterns for your library.

  1. You could make inverseMatrix return an optional Matrix, which is nil if the factorization or inverse fails--this is pretty reasonable because that only happens when there is no inverse matrix, is easy to document, and is immediately familiar to most Swift programmers.

    The downside to this approach is that it loses information. dgetrf's error code n tells you that the leading nxn sub-matrix was non-singular, and has been successfully factored. In some applications, it is numerically appropriate to add a small regularization term to the (n,n) matrix element and continue the factorization from there. In other applications you might use only the leading nxn factored corner and pair that with some iterative technique on the trailing submatrix. Returning an optional is easy for programmers who are not numerical programming experts to understand, but eliminates information that experts are able to use productively, so it would be undesirable in a library intended to let expert numerical analysts build sophisticated numerical methods.

  2. You could return a Result<Matrix, FactorizationError> or have your wrappers throw. These introduce some extra work for the caller (they have to handle the error cases), but allow you to preserve some of the information returned by LAPACK for sophisticated users. Between the two options, throwing is probably more idiomatic these days, but both are viable options.

  3. For a very low-level wrapper, you could very literally follow the LAPACK conventions and return a (result, status) pair. This provides the most information (because you can produce both the partially-factored result and the error code, rather than one or the other¹), so it's potentially the most useful for expert users, but it's also the most confusing for people who are not already familiar with LAPACK's conventions. Realistically, the set of people who can profitably use this sort of binding is pretty small.


¹ you can also do this with result/throws by packing not only the error code but also the partial result into the Error, but I'm not sure how many users will actually take advantage of this, so it's a bunch of fuss for limited utility. On the other hand, you could put the partial result into a PartialFactorization type or similar that distinguishes it at the type level from a successful result, which has some real value.

5 Likes

What do you think about providing two inversion functions: a user friendly function and an expert function?

The user friendly function would return an optional, nil is returned if singular or inputs are not defined properly. It would also print out the info and error text for why nil is returned. The user friendly function would also provided cleaner code since you could just use an if-let or guard statement.

The expert function would return the result along with info/error status; it could return a partial result for certain situations too. A user could start with the user friendly function, if they get nil then they could use the expert function to get more detailed information. A consequence of the expert function might be messy code since you would have to work with a result type or handle a throw statement.

Based on your suggestions, I made a function that throws the error and also allows the user to get the matrix from that error. For the error text, I used the same wording that is in the LAPACK documentation. But it seems odd to define InverseError<Double>.notSquare where the numeric type isn't used for that error case. Anyway, does this seem like a reasonable approach?

enum InverseError<T>: Error {
    case notSquare
    case illegalValue(String, Matrix<T>)
    case singular(String, Matrix<T>)
}

func inverseMatrix(_ a: Matrix<Double>) throws -> Matrix<Double> {
    if a.rows != a.columns {
        throw InverseError<Double>.notSquare
    }
    var mat = a.copy()
    var n = a.rows
    var pivots = [Int](repeating: 0, count: a.rows)
    var workspace = [Double](repeating: 0.0, count: a.rows)
    var info = 0

    // Perform LU decomposition
    dgetrf_(&n, &n, &mat.data, &n, &pivots, &info)
    if info < 0 {
        let err = "The \(info)-th argument had an illegal value."
        throw InverseError.illegalValue(err, mat)
    } else if info > 0 {
        let err = """
        U(\(info),\(info)) is exactly zero. The factorization has been completed, but \
        the factor U is exactly singular, and division by zero will occur if it is used \
        to solve a system of equations.
        """
        throw InverseError.singular(err, mat)
    }
    info = 0

    // Calculate inverse based on LU decomposition
    dgetri_(&n, &mat.data, &n, &pivots, &workspace, &n, &info)
    if info < 0 {
        let err = "The \(info)-th argument had an illegal value."
        throw InverseError.illegalValue(err, mat)
    } else if info > 0 {
        let err = """
        U(\(info),\(info) is exactly zero; the matrix is singular and its \
        inverse could not be computed.
        """
        throw InverseError.singular(err, mat)
    }

    return mat
}

Here are some examples of using the function. The printed output is shown after the code.

// This matrix has an inverse therefore no errors are thrown

let A = Matrix<Double>(rows: 2, columns: 2, data: [1, 2, 3, 4])
let invA = try inverseMatrix(A)
print(invA, "\n")
Matrix<Double>(rows: 2, columns: 2, data: [-2.0, 1.0, 1.5, -0.5]) 
// This matrix is singular

let B = Matrix<Double>(rows: 3, columns: 3, data: [1, 2, 3, 4, 5, 6, 7, 8, 9])

do {
    let invB = try inverseMatrix(B)
    print(invB)
} catch InverseError<Double>.illegalValue(let err, let mat) {
    print("Error message is:\n\(err)")
    print("Partial matrix is:\n\(mat)")
} catch InverseError<Double>.singular(let err, let mat) {
    print("Error message is:\n\(err)")
    print("Partial matrix is:\n\(mat)")
} catch InverseError<Double>.notSquare {
    print("Input matrix is not square.")
}
Error message is:
U(3,3) is exactly zero. The factorization has been completed, but the factor U is exactly singular, and division by zero will occur if it is used to solve a system of equations.
Partial matrix is:
Matrix<Double>(rows: 3, columns: 3, data: [3.0, 0.3333333333333333, 0.6666666666666666, 6.0, 2.0, 0.5000000000000001, 9.0, 4.0, 0.0])
// This matrix is not square

let C = Matrix<Double>(rows: 2, columns: 3, data: [1, 2, 3, 4, 5, 6])

do {
    let invC = try inverseMatrix(C)
    print(invC)
} catch InverseError<Double>.notSquare {
    print("Input matrix is not square.")
}
Input matrix is not square.
1 Like

Personally, I would make a non-square matrix (and other structural errors) be a precondition failure rather than a throw, but that's largely a matter of taste. Otherwise, that seems pretty reasonable.

Why a preconditionFailure instead of a precondition?

Two ways to describe the same thing. The precondition is the requirement. The precondition failure is the error resulting from failing it.

So you would do something like:

    if a.rows != a.columns {
        preconditionFailure("Input matrix is not square.")
    }

instead of this?

    if a.rows != a.columns {
        throw InverseError<Double>.notSquare
    }

I like using throw here because it allows the program to finish. Whereas the precondition failure causes it to halt where the precondition is defined.

Right. That's where the matter of taste comes in. I would say that these structural errors are always evidence of a serious program bug (e.g. interchanging two matrix dimensions) that is likely to give incorrect or corrupted results even when it's not caught, and therefore it never makes sense to attempt to continue execution through that point. Other people reasonably disagree.

1 Like

Regarding the LAPACK functions, the value given by INFO is considered to be the i-th element in the matrix (see below). Is this INFO value for zero-based or one-based indexing? I think Fortran uses one-based indexing by default; unless Accelerate changed that for the Swift interface. When I use the Swift inverseMatrix function I shared above, the INFO value seems to be one-based. So if the error message gives U(3, 3) then I should subtract one so that it is U(2, 2) for the Swift matrix. Am I interpreting this correctly?

INFO is INTEGER
          = 0:  successful exit
          < 0:  if INFO = -i, the i-th argument had an illegal value
          > 0:  if INFO = i, U(i,i) is exactly zero; the matrix is
                singular and its inverse could not be computed.

One last thing. Swift already has a function named inverse which I was hoping to use for the name of my function instead of inverseMatrix. I know that NumPy and Julia use inv as the function name but spelling out inverse seems more Swift-like to me. I tried using inverse(of a: Matrix<Double>) but that still clashes with the Swift inverse function. I guess I could name it inverseOf or invert. Do you have any naming suggestions?

LAPACK indexing is 1-based. Even if it wasn’t, the error codes would have to be (or use some other encoding, because 0 is success).

It’s totally ok to shadow the inverse name.

1 Like

As you previously mentioned, there are essentially three ways to handle this in Swift:

  1. Return an optional Matrix
  2. Use throws to handle errors and partial matrix results
  3. Return a (result, status) pair similar to LAPACK conventions

Which one would be the most performant?

My guess is method (3) would be the most performant since you are not dealing with optional values or catching errors. For method (1) you have to unwrap the optional matrix which seems like a performance hit, but maybe you can do a force unwrap to negate that? For method (2) you have to define a do-catch block for the errors, but maybe just using try without the do-catch or a try! would avoid degraded performance. I'm not familiar enough with optionals and throws in Swift to know how much they would affect code performance from a numerical perspective. So any comments on this would be very helpful.

The performance of these really does not matter even a little bit, because compared to the cost of a matrix factor or solve of any size larger than say 8x8 or so, they are all free (and for matrices smaller than that, the overhead of LAPACK itself dominates the numerical work, so an implementation pursuing top performance would be doing something else). It is purely a question of usage patterns and taste.

One thing I would note is that looking at matrix inversion can be a little bit misleading if you're interested in more general LAPACK bindings, in that it really has a single input and output and pretty simple error conditions, so the patterns that you adopt to bind it may not scale nicely to routines with more complex interface conventions like, say, GESVX.

You mentioned that throwing errors is more idiomatic these days for Swift. So why don't functions in Accelerate throw errors? For example, this vDSP_wiener function just provides an error flag. Is it just an old API or is there some other reason why throws was not used for this function?

The vDSP API are as old as much of LAPACK (and have a similar Fortran legacy). They are very, very low-level operations that do not generally take advantage of any higher-level language features (and most vDSP operations do not have data-dependent failure modes that would merit throwing--e.g. summing two vectors only admits structural errors like a length mismatch, which are best enforced via precondition).

The Accelerate team has done some work to expose Swiftier bindings for parts of vDSP, e.g. sumAndSumOfSquares(_:) | Apple Developer Documentation, but those are precisely the parts that do not have interesting failure modes that might merit throwing.

1 Like