[Feedback] NIO-based HTTP Client

The package namespace clash issue seems like it will only get worse and worse.

I notice that it was discussed three years ago (Revisiting package conflicts)

How about if package manifests had a macOS-style 'bundleID' in reverse DNS style? Then you could use the short package name, until there was a clash, then the compiler could prompt you to use the unambiguous bundleID instead? (I have put literally seconds of thought into this idea and have no idea how anything is implemented under the covers!)

@aciid

1 Like

That would just remain a time interval and as soon as an operation starts that makes a connection, it would set that as a deadline with .now() + bootstrap.connectionTimeout. Does that make sense?

This is not a trivial problem but we do need some solution in the long term. @johannesweiss @jrose can you post a summary of the problems somewhere?

1 Like

I totally agree with you. To truly solve it, we need will need improvements very likely both in the language as well in the package manager. We started talking about this topic in the SSWG recently and I'm sure there will soon be public discussions about properly solving the problem in the language. As @tkremenek said on the SSWG call, right now "this is an inherently under-designed space" and I think there's widespread agreement that the status quo in Swift just isn't good enough.

However, we can't just wait until the problem is resolved properly and therefore, as a stop-gap, we're trying to avoid conflicts as much as possible by trying to avoid them with Github searches :slight_smile:. I'm hoping that buys us enough time until we can solve this properly.

Yes, I'm working on the summary part that affects the source level, sorry that this is taking a while.

We should however separate the clashing issue in the language from the HTTP client that is proposed here. I don't think anyone expects the language level problems to be solved before we tag the first version of the HTTP client.

5 Likes

@artemredkin Swift has an issue where if a module contains a type with the same name, it becomes impossible to disambiguate type names in certain cases. For example:

// module HTTPClient
public final class HTTPClient { ... }
// module Foo
public final class HTTPClient { ... }
// module MyApp
import Foo
import HTTPClient

// error: ambiguous type name
let client = HTTPClient()
// ok, let's disambiguate? can't:
// error: type HTTPClient doesn't have any member named HTTPClient
let client = HTTPClient.HTTPClient()
1 Like

Hm, I don't think I follow. Right now I have code like this:

bootstrap.connectionTimeout(timeout)

I can schedule a task on an eventLoop to "cancel" a task, but what happen if I haven't received a channel yet? How can I emulate bootstrap.connectionTimeout(timeout) with a deadline?

Apart from possible clashes with NIO, why do you think having clashes with other libraries is a big issue? Event if we decide on a different name we are not "safe" from someone releasing a library with the same name as well. Are you concerned about people importing two libraries themselves or getting them by transitive dependencies?

1 Like

That's exactly the concern, and one that we had a real situation with SwiftLog, Vapor, and IBM.

In some aspects, we can wash our hands clean and say "That's the developer's error. These two libraries are incompatible", but we can't do that in the SwiftLog, SwiftMetrics, etc. case where it's expected to be mostly compatible, or when there's several layers to an end-user's dependency tree and two "middleware" packages aren't aware of needing to warn their users that their implementation details are incompatible with others out in the ecosystem.

It's also the case that these errors manifest in awful compile errors that are unresolvable and undetectable by the portion of your dependency tree that introduces it. When I add a new dependency or create a new module, nothing breaks in my project, but things may break in yours. Whose responsibility is it to take action in this case?

Pending a proper solution as mentioned above, the best we can do is to encourage the community to achieve diversity in naming. I think this means that "prefixes" and "suffixes" are an anti pattern: as @johannesweiss already alluded to, they reduce the space of possible names and so perversely increase the risk of name clashes. After all, if you start working on a SwiftNIO-based library to talk to, say, Twilio that you'd like to progress through the SSWG, the obvious name choice under @IanPartridge's suggestion is "swift-twilio-nio". If two people start at once, they are guaranteed to collide.

If the guidance is instead "avoid purely descriptive names", things get easier. Call your framework or library something interesting and you're much less likely to encounter clashes. In the case of purely structural tools some leniency can be applied here (logging and metrics abstract APIs are a good example), but even for HTTP clients I think we're in the world of wanting to have interesting names.

Researching prior art in other language ecosystems can be helpful in this regard. For example, notable HTTP clients:

  • Requests for Python
  • Hyper for Rust
  • Faraday for Ruby
  • Alamofire for Swift

These are (with the exception of Hyper, which is very common in the HTTP ecosystem and no longer google-useful) great names! They get good Google results, are easy to remember if you're familiar, and help avoid namespace clashing.

I know this is tough advice to give: after all, naming is hard. But some creativity is really valuable here, both in terms of avoiding naming collisions and in terms of helping people remember you tool exists.

What about libraries like swift-metrics and swift-log and their implementations? Should they also be renamed then?

I think metrics and logs are just borderline enough to stay as is: the libraries themselves don’t do anything for a given backend and so have limited opportunity for renaming, and are extremely low level currency APIs unlikely to be independently reinvented without explicitly planning to supersede them (which makes naming conflicts unlikely).

The back ends are another story though.

1 Like

Another question, If we pick a new name, say like swift-requests, do you think it would be better to rename HTTPClient class as well?

Yes, that would be helpful I think. The fewer opportunities for clashing, the better.

The way the SSWG treats this issue is: Competition is good. By any means, people should be able to mix multiple (for example) HTTP clients in one application. That's also extremely likely because web frameworks like Vapor/Kitura will depend on an HTTP client. But an author of a web application written using Vapor/Kitura might prefer a different HTTP client. And there's no harm caused by using both of them so we should do anything we can to enable this.

There are however notable exceptions: The low-level API packages are cases where they must remain singletons (as in the ecosystem should ideally only have one logging API package). Having competing logging APIs is bad because that will lead to different packages logging using different APIs which will lead to messages being lost. That's why we decided to make SwiftLog an API-only package and make users put all the implementations in separate 'logging backend' packages. We tried to keep SwiftLog as free of opinion as possible and try to accommodate almost all use-cases in order to make in less likely that someone cannot use SwiftLog. Needless to say that the 'logging backend' packages should make sure they don't clash with others. The same applies to SwiftMetrics which is also an API-only package.

So the question is then how should we name this, my preference would be:
0. keep it like it is now (:sadpanda:)

  1. swift-requests
  2. async-http-client

cc @tanner0101 @johannesweiss @lukasa @Mordil @IanPartridge @tomerd wdyt?

My vote would be 2. I.e. async-http-client, module name AsyncHTTPClient and "main" type anything but AsyncHTTPClient because types that are named exactly the same as modules cause trouble as @tanner0101 points out. We could do HTTPClient for the main type.

1 Like

My vote is also 2: following the "requests" name suggests an attempt to match the API, which we have not done.

2 Likes

Hello guys, I just could rename my package, if you decide to stick on swift-nio-http-client name. Just let me know what is your final decision, for me is not a problem at all.

Okay, I just renamed my package. So swift-nio-http-client is free, cheers!

3 Likes

swift-nio-http-client is available now, and would get my vote FWIW.

@artemredkin given that the client was accepted into sandbox :partying_face: should we lock this thread? or further discussion needed?

3 Likes