1

Due to the heroic efforts of @SlaunchaMan, there's a WIP PR up that gets Alamofire mostly working on Linux, including tests! However, it's not really in a releasable state, even if were were ready for it, due to a few issues around Foundation on Linux, namely:

  • URLSessionTaskMetrics is still completely missing, leading to a bifurcation of a lot of APIs which expect it.
  • There is no InputStream-based uploading using URLSessionUploadTask, leading to further bifurcation issues.
  • No Security framework means all of our error handling and other logic around trust management must be avoided.
  • A behavioral difference in URLSessionTask when calling resume() multiple times. On Apple platforms it's fine but on Linux it produces a runtime crash complaining about duplicate resume() calls.
  • Lack of IANA String.Encoding parsing, meaning we have to write our own mapping.
  • A few API sync issues between Foundation on 2019 Apple platforms and Foundation in 5.1 on Linux.
  • Replacement for Bundle access to containing executable name and such, for our User-Agent string.

Those issues aside, I believe it is usable and once workarounds aren't so severe, we hope to support it officially.

cc @millenomi @Tony_Parker

14 Likes
2

Ive been looking at URLSession bugs on Linux recently and I think I have a fix for the resume() issue [SR-12336] Difference in Behavior between Foundation and FoundationNetworking · Issue #3265 · apple/swift-corelibs-foundation · GitHub

1 Like
3

So:

  • URLSessionTaskMetrics should absolutely exist at least as a currency type, but I need to research how I can get that information from curl.

  • InputStream-based upload (via the asynchronous getBody fixes) should be there. Did I miss something?

  • resume() should absolutely work as in Darwin. I’ll take a look at @spevans‘s patch.

  • What API is missing specifically for IANA encoding?

  • API sync likewise needs to be fixed — which ones?

  • Bundle.main, counterintuitively, should work as is, as it would for eg. Darwin command-line processes. What are you seeing there?

The bad news:

  • Security as a module is really unlikely to come to Linux. What are your needs around it? It may be the one thing that needs the longer-term workarounds.
1 Like
4

… I completely missed the link to the WIP PR, which I’m sure will answer some of my questions, but high-level overviews to aid in planning are always appreciated. :)

5

Hmm, you're right about the InputStream APIs. I must have been getting other errors with them and removed them from the Linux implementation thinking they weren't included.

For the IANA encoding, Alamofire on Apple platforms uses CFStringConvertIANACharSetNameToEncoding and CFStringConvertEncodingToNSStringEncoding.

And as far as the bundle, we’re using keys from the info dictionary like kCFBundleExecutableKey, which don’t really map to Linux anyway.

As far as security goes, I think the right thing to do would be to abstract out the Security framework on Apple platforms, then have another implementation for Linux. I’m not familiar enough with OpenSSL or other libraries to be able to write that, but we could at least give others a way to provide their own implementations.

6

A few notes on your PR:

  • NSNumber is not guaranteed to preserve the type of something when bridged, but it does know what type a number is. Use .objcType to know. (It also nicely gets you out of Core Foundation API usage there — CF is available on Linux but it’s largely something we want projects to avoid using going forward, and it’s eg not available in @compnerd’s Windows port.)

  • You’re probably never hitting your as? Bool branches after checking as? NSNumber. Types that are automatically bridged tend to make counterintuitive issues occur that way.

A way to check that actually works is:

protocol IsActualNSNumber {}
extension NSNumber: IsActualNSNumber {}

…
if number is IsActualNSNumber { … }

Since the cast to IsActualNSNumber does not trigger any implicit bridging, you can be confident that’s not a value of a Swift numeric type being bridged.

7

Specifically, Alamofire needs URLSessionTaskMetrics to:

  • Exist
  • Have the taskInterval property.
  • Properly fire the delegate methods when collected, as Alamofire uses them as a lifetime event.

@SlaunchaMan can answer this one, but it may be a Foundation@5.1 vs. 5.2 thing.

Great. This was a rather surprising difference.

Namely, CFStringConvertIANACharSetNameToEncoding and CFStringConvertEncodingToNSStringEncoding.

In this case, a newer URLCache overlay: URLCache(memoryCapacity:diskCapacity:directory:).

We access various properties to build our User-Agent:

let executable = info[kCFBundleExecutableKey as String] as? String ?? "Unknown"
let bundle = info[kCFBundleIdentifierKey as String] as? String ?? "Unknown"
let appVersion = info["CFBundleShortVersionString"] as? String ?? "Unknown"
let appBuild = info[kCFBundleVersionKey as String] as? String ?? "Unknown"

Is there a replacement on Linux?

Yeah, we've known this for a while, I just didn't do a great job of grouping our errors that use the API together, so they're a bit spread out. If there's ever a replacement for it on Linux that provides the same delegate call back we could use that, but in the meantime I'm happy just disabling the functionality on Linux.

8

Note that you’re doing a disservice to command-line (unbundled) Darwin processes as well. What’s the behavior there? Do you require an __infoplist… segment for them?

9

A more serious issue for this port is that when run in XCTest, we never see the callbacks for requests. I can compile a simple project using Alamofire that runs fine on macOS and Linux and fetches a simple JSON payload from the network, but the tests never seem to get any data. I don’t know if this is XCTest or something else about the Linux implementation. Maybe a runloop or threading issue? A lot of the default callbacks use the main queue, so it could be something to do with how XCTest waits for expectations, too.

10

We are testing FoundationNetworking with XCTest right now without specific workarounds, so it may need more investigation on your part for your specific threading setup. Certainly we want to fix this if it causes a behavior difference on Linux vs Darwin.

11

Until recently it wasn't easy to use Alamofire with command line projects anyway, so it's not something we've ever tested (not being able to easily run tests against them is another issue). What would you suggest instead?

12

It looks like the request tests are running and many passing, there are just failures where tests expect certain events. Some behavioral and equality differences in certain tests too.

13

You can get the process name from ProcessInfo, and have an API (which you probably have somewhere) for setting version information explicitly. Both Darwin and Linux command-line processes can have an Info.plist added to them if needed (the former via linker settings to embed it into a section, the latter via freestanding bundles), but it’s not the default.

14

Sounds like an additional level of fallback would work well there.

In general, there are changes in this PR that can be played separately, like adding a mutex lock, rearranging some methods to be easier to work around, and things like the IANA parsing and this bundle workaround.

15

Writing down the differences you’re seeing as issues would help tremendously in figuring out how to approach this!

16

You can see the results of the latest test run here. From the top it looks like:

  • Tests using URLCredential fail.
  • Some tests for URLCache behavior fail due to timestamp and other issues.
  • Tests around download resumeData fail.
  • Tests around file operations fail.
  • Tests of number handling in our URLEncodedFormEncoder fail, likely due to the NSNumber issues pointed out earlier.
  • Various lifetime event failures due to missing metrics callbacks which haven't be avoided, possibly other changes to behavior.
  • Tests of URLSession's behavior when using invalidateAndCancel fail.
  • Somehow there are failures on the number of times adapters and retriers are called, which is weird.
  • Eventually the tests crash.

Investigating those failures will need a full Linux instance, so it may be a while.

17

The link and the list already help a lot, thank you!

18

@millenomi If I could get a little more advice around the User-Agent I'd appreciate it. Here's the current code:

/// Example: `iOS Example/1.0 (org.alamofire.iOS-Example; build:1; iOS 13.0.0) Alamofire/5.0.0`
public static let defaultUserAgent: HTTPHeader = {
    let userAgent: String = {
        if let info = Bundle.main.infoDictionary {
            let executable = info[kCFBundleExecutableKey as String] as? String ?? "Unknown"
            let bundle = info[kCFBundleIdentifierKey as String] as? String ?? "Unknown"
            let appVersion = info["CFBundleShortVersionString"] as? String ?? "Unknown"
            let appBuild = info[kCFBundleVersionKey as String] as? String ?? "Unknown"

            let osNameVersion: String = {
                let version = ProcessInfo.processInfo.operatingSystemVersion
                let versionString = "\(version.majorVersion).\(version.minorVersion).\(version.patchVersion)"
                // swiftformat:disable indent
                let osName: String = {
                #if os(iOS)
                    return "iOS"
                #elseif os(watchOS)
                    return "watchOS"
                #elseif os(tvOS)
                    return "tvOS"
                #elseif os(macOS)
                    return "macOS"
                #elseif os(Linux)
                    return "Linux"
                #elseif os(Windows)
                    return "Windows"
                #else
                    return "Unknown"
                #endif
                }()
                // swiftformat:enable indent

                return "\(osName) \(versionString)"
            }()

            let alamofireVersion = "Alamofire/\(version)"

            return "\(executable)/\(appVersion) (\(bundle); build:\(appBuild); \(osNameVersion)) \(alamofireVersion)"
        }

        return "Alamofire"
    }()

    return .userAgent(userAgent)
}()

I'm assuming that if there's no Info.plist at all that Bundle.main.infoDictionary will be nil. I also know that I can use ProcessInfo to get the executable name through arguments.first, but there doesn't seem to be any additional information available, like integrating app version (Alamofire has its own version info) or build numbers.

19

One other bit of missing functionality: MIME type parsing. Alamofire tries to infer the type of a file used in multipart form data uploads to properly represent it in the form boundaries using API from CoreServices. Obviously that won't be available but is there any sort of equivalent API for:

if
    let id = UTTypeCreatePreferredIdentifierForTag(kUTTagClassFilenameExtension, pathExtension as CFString, nil)?.takeRetainedValue(),
    let contentType = UTTypeCopyPreferredTagWithClass(id, kUTTagClassMIMEType)?.takeRetainedValue() {
    return contentType as String
}
1 Like
20

I've replaced the implementation of our isBool check with String(cString: objCType) == "c" which seems to work. That's present on Linux too, despite no Obj-C runtime?

I think it's okay if we go through NSNumber for everything, though there seems to be encoding differences in the Linux tests, so we may need to look at this again later.

1 Like