I'm moving a tool I made some time ago to swift6+swift static sdk. I am willing to move to async/await for my URLSession-based operations, and if I use the plain swift6 toolchain it works fine, but when I try with the swift-linux-musl sdk, I have found two issues already
print with no terminator doesn't show,
URLSession.shared.data(for:) throws an error Error Domain=NSURLErrorDomain Code=-1001 "(null)"
so I was wondering if the swift static linux sdk is supposed to support async operations or if I have to go back to completions for it.
Then again, it works fine without the static SDK
here's a simplified program that shows the same behavior:
import Foundation
@preconcurrency import FoundationNetworking
#if canImport(Glibc)
import Glibc
#elseif canImport(Musl)
import Musl
#endif
@main
struct Main {
static func main() async {
let url = URL(string:"https://forums.swift.org/c/general-announce/24")!
var request = URLRequest(url: url)
request.httpMethod = "POST"
print(request)
do {
let (data, response) = try await URLSession.shared.data(for: request)
print(data)
print(response)
} catch {
print(error)
}
}
}
// swift-tools-version: 6.0
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "test123",
platforms: [
.macOS(.v10_15)
],
targets: [
// Targets are the basic building blocks of a package, defining a module or a test suite.
// Targets can depend on other targets in this package and products from dependencies.
.executableTarget(
name: "test123"),
]
)
I'm building on ubuntu-22.04 and macOS 14.5 with the same results
but switching to your model of command line (sdk full name as downloaded, tripe specified with --triple instead) it works as expected (got a static executable and my original case involving a REST operation worked this time.)
OK, the only reason I had to specify the full SDK name is because I had multiple SDKs installed, which most won't. After deleting them all except for the matching SDK version, I'm unable to reproduce with the commands you linked either:
I don't know if this is related to the issue with x86_64-swift-linux-musl on ubuntu, The same program builds but never connects to the test URL on Ubuntu 22.04 aarch64 running on a raspberrypi4: both ways of specifying the swift sdk produce a static executable that will fail to connect to the test url.
There I installed only swift6. Also gcc, CMake and then the prereq list for ubuntu 20.04
I tried executing the (static) program again (raspberry pi), and again, like I was waiting for it to magically work, and at least in the raspberry pi that seems to be the case: sometimes it works, sometimes it doesn't.
Then again, if I don't use the static linux sdk, the executable always connects successfully.
The networking functions are largely provided by the kernel rather than Musl, and I think we'd know if Musl had flaky networking.
I'm honestly not sure why you're seeing it not work, or why you're seeing flaky behaviour. The one gotcha I do know about with it is that it does need to be able to locate the root certificates for TLS to work (this has to be done specially because libcurl doesn't know how to do it at runtime, but the statically linked program could find itself on any kind of Linux and there are a number of different locations where we might find them), but that should work for Ubuntu already.
It'd be great if it was a weird networking issue on my side!... but what happens is I can't reproduce the issue if I don't involve the swift static sdk.
If I stick now to the raspberry pi with Ubuntu 22.04-aarch64,
when I build without the static sdk, the executable always produces the expected results (connects and displays some info on the request results)
when I build with swift build --swift-sdk $(arch)-swift-linux-musl, it builds OK always, but the results of running the executable are it fails/works as expected in what it looks like a random sequence
when I build with swift build --swift-sdk $(swift sdk list) --triple $(arch)-swift-linux-musl the behavior is the same as with the previous case
(after retrying the same (always removing .build first, before building) the x86_64 ubuntu 22.04 environment has the same behavior.)
Again, possibly unrelated, but it only seems to happen in the raspberry pi 4, swift build with the fully specified sdk missed the aarch64 _Concurrency module. It happens sometimes, too. Not always.
rm -rf .build; swift build --swift-sdk $(swift sdk list) --triple $(arch)-swift-linux-musl
warning: multiple Swift SDKs match ID `swift-6.0-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1` and host triple aarch64-unknown-linux-gnu, selected one at /home/marcelo/.swiftpm/swift-sdks/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1.artifactbundle/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1/swift-linux-musl
Building for debugging...
error: emit-module command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: could not find module '_Concurrency' for target 'aarch64-swift-linux-musl'; found: x86_64-swift-linux-musl, at: /home/marcelo/.swiftpm/swift-sdks/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1.artifactbundle/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/x86_64/usr/lib/swift_static/linux-static/_Concurrency.swiftmodule
<unknown>:0: error: could not find module '_Concurrency' for target 'aarch64-swift-linux-musl'; found: x86_64-swift-linux-musl, at: /home/marcelo/.swiftpm/swift-sdks/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1.artifactbundle/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/x86_64/usr/lib/swift_static/linux-static/_Concurrency.swiftmodule
I don't think --triple interacts well with --swift-sdk (@Max_Desiatov?), so I think I'd avoid that for now.
I wonder whether there is indeed some Musl-related issue here; I wonder if it's DNS related. I'd check your /etc/resolv.conf (Musl does implement the resolver; that isn't a kernel function, and there are behavioural differences from Glibc there — see https://wiki.musl-libc.org/functional-differences-from-glibc.html).
Here's resolv.conf from the raspberry pi 4, but again it works if I don't use the static sdk
# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.
nameserver 127.0.0.53
options edns0 trust-ad
search .
note that resovectl status show it's using my internal network DNS server
and in the x86_64 (WSL2) environment resolv.conf contains
I might be inclined to try writing a small test program that just attempts to resolve the hostname you're trying to talk to, and see if that is somehow unreliable when using the static SDK. That'll at least narrow the problem down a bit. If that works reliably, then that isn't the problem.
You could also try adding
nameserver 8.8.8.8
to your /etc/resolv.conf and see if that makes things more reliable.
I'll try to work on that later today. Also, for if we've forgotten, the code uses async calls. I believe that may be in the root of the problem somehow. But you're right, URLSession.shared.data(for:) resolves the DNS entry first, so the timeout can be as early as that, depending on how it's implemented.
Ok, I'm not sure how to do this. I recall there's the C function gethostbyname(). Would it be ok just make the function that will call gethostbyname async?
The project with the original program is here (main)
