Alamofire on Linux: Possible but not release ready!

We are testing FoundationNetworking with XCTest right now without specific workarounds, so it may need more investigation on your part for your specific threading setup. Certainly we want to fix this if it causes a behavior difference on Linux vs Darwin.

Until recently it wasn't easy to use Alamofire with command line projects anyway, so it's not something we've ever tested (not being able to easily run tests against them is another issue). What would you suggest instead?

It looks like the request tests are running and many passing, there are just failures where tests expect certain events. Some behavioral and equality differences in certain tests too.

You can get the process name from ProcessInfo, and have an API (which you probably have somewhere) for setting version information explicitly. Both Darwin and Linux command-line processes can have an Info.plist added to them if needed (the former via linker settings to embed it into a section, the latter via freestanding bundles), but it’s not the default.

Sounds like an additional level of fallback would work well there.

In general, there are changes in this PR that can be played separately, like adding a mutex lock, rearranging some methods to be easier to work around, and things like the IANA parsing and this bundle workaround.

Writing down the differences you’re seeing as issues would help tremendously in figuring out how to approach this!

You can see the results of the latest test run here. From the top it looks like:

  • Tests using URLCredential fail.
  • Some tests for URLCache behavior fail due to timestamp and other issues.
  • Tests around download resumeData fail.
  • Tests around file operations fail.
  • Tests of number handling in our URLEncodedFormEncoder fail, likely due to the NSNumber issues pointed out earlier.
  • Various lifetime event failures due to missing metrics callbacks which haven't be avoided, possibly other changes to behavior.
  • Tests of URLSession's behavior when using invalidateAndCancel fail.
  • Somehow there are failures on the number of times adapters and retriers are called, which is weird.
  • Eventually the tests crash.

Investigating those failures will need a full Linux instance, so it may be a while.

The link and the list already help a lot, thank you!

@millenomi If I could get a little more advice around the User-Agent I'd appreciate it. Here's the current code:

/// Example: `iOS Example/1.0 (org.alamofire.iOS-Example; build:1; iOS 13.0.0) Alamofire/5.0.0`
public static let defaultUserAgent: HTTPHeader = {
    let userAgent: String = {
        if let info = Bundle.main.infoDictionary {
            let executable = info[kCFBundleExecutableKey as String] as? String ?? "Unknown"
            let bundle = info[kCFBundleIdentifierKey as String] as? String ?? "Unknown"
            let appVersion = info["CFBundleShortVersionString"] as? String ?? "Unknown"
            let appBuild = info[kCFBundleVersionKey as String] as? String ?? "Unknown"

            let osNameVersion: String = {
                let version = ProcessInfo.processInfo.operatingSystemVersion
                let versionString = "\(version.majorVersion).\(version.minorVersion).\(version.patchVersion)"
                // swiftformat:disable indent
                let osName: String = {
                #if os(iOS)
                    return "iOS"
                #elseif os(watchOS)
                    return "watchOS"
                #elseif os(tvOS)
                    return "tvOS"
                #elseif os(macOS)
                    return "macOS"
                #elseif os(Linux)
                    return "Linux"
                #elseif os(Windows)
                    return "Windows"
                #else
                    return "Unknown"
                #endif
                }()
                // swiftformat:enable indent

                return "\(osName) \(versionString)"
            }()

            let alamofireVersion = "Alamofire/\(version)"

            return "\(executable)/\(appVersion) (\(bundle); build:\(appBuild); \(osNameVersion)) \(alamofireVersion)"
        }

        return "Alamofire"
    }()

    return .userAgent(userAgent)
}()

I'm assuming that if there's no Info.plist at all that Bundle.main.infoDictionary will be nil. I also know that I can use ProcessInfo to get the executable name through arguments.first, but there doesn't seem to be any additional information available, like integrating app version (Alamofire has its own version info) or build numbers.

One other bit of missing functionality: MIME type parsing. Alamofire tries to infer the type of a file used in multipart form data uploads to properly represent it in the form boundaries using API from CoreServices. Obviously that won't be available but is there any sort of equivalent API for:

if
    let id = UTTypeCreatePreferredIdentifierForTag(kUTTagClassFilenameExtension, pathExtension as CFString, nil)?.takeRetainedValue(),
    let contentType = UTTypeCopyPreferredTagWithClass(id, kUTTagClassMIMEType)?.takeRetainedValue() {
    return contentType as String
}
1 Like

I've replaced the implementation of our isBool check with String(cString: objCType) == "c" which seems to work. That's present on Linux too, despite no Obj-C runtime?

I think it's okay if we go through NSNumber for everything, though there seems to be encoding differences in the Linux tests, so we may need to look at this again later.

1 Like

I've put up a PR which boils down many of the foundational changes into a single set of changes. It should make it easier to ultimately support Linux while not affecting Alamofire now.

1 Like

Indeed. Linux NSValue simulates encodings for known types, though it cannot handle encoding C structs the way the Darwin version can. It works for numbers the way you’d expect, though!

I will take a look at the PR at some point this week. Thank you!

1 Like

The test suite has completed for the first time! With the probable Progress bug out of the way just 57 failures to investigate!

1 Like

It looks like the current issue is some inconsistent behavior of the Progress type between macOS and Linux. I’ve filed this as SR-12391 so we can track it separately.

Another additional wrinkle: with the upcoming DataStreamRequest we now use Stream.getBoundStreams, which is unimplemented in swift-corelibs-foundation. Luckily it should be easy to workaround.

I've conditionally compiled out the call to NSStream.getBoundStreams(withBufferSize:inputStream:outputStream:) for now, with an additional todo marked in the PR to see if that API can be made to work on Linux.

Another issue as I go through the test failures: it looks like URLSession on Linux isn’t calling methods for URLAuthenticationChallenge. I’ve created SR-12676 to track it.

1 Like

Apple has released a new security framework, called SwiftCrypto, with an open source implementation for Linux a while ago. It is also available on all Apple platforms as system framework but only with the newest OS version (iOS 13, macOS 10.15, etc.).
This could be a better alternative than OpenSSL on Linux.

Unfortunately SwiftCrypto isn't really suitable for the type of work Alamofire needs to do, like certificate pinning. Additionally, there isn't an alternate path for security challenges in URLSession, so there's not really a way to expose them on Linux unless Apple's willing to create Linux-only APIs, or a new API that can be used on all platforms. I'd prefer the second if it means being able to stop using the Security framework.

2 Likes