SR-6405: URLRequest does not capitalise HTTP methods

Hi, it looks like Foundation on Darwin capitalises certain HTTP
methods but not others:

let methods = ["get", "head", "post", "put", "delete", "connect",
"options", "trace", "patch"]

var x = URLRequest(url: URL(string: "/hello")!)

for m in methods {
    x.httpMethod = m
    print(x.httpMethod!)
}

Output on Darwin:
GET
HEAD
POST
PUT
DELETE
CONNECT
options
trace
patch

Currently on Linux we don't do any capitalization so I'd like to fix this.

Is my list of 6 methods above the definitive list of which HTTP
methods should be capitalized?

Thanks,

···

--
Ian Partridge

Brandon Williams and I have come across a lot of inconsistencies between Foundations in our Swift web work. We’ve been trying to file bugs when we remember to, but I’m curious if there’s a better way to catch these. Is the Foundation test suite run against both implementations to attempt to catch these kinds of things? If not it would seem like a big win to do so. If so, I suppose we could help by beefing up the test coverage?

Stephen

···

On Nov 16, 2017, at 12:18 PM, Ian Partridge via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:

Hi, it looks like Foundation on Darwin capitalises certain HTTP
methods but not others:

let methods = ["get", "head", "post", "put", "delete", "connect",
"options", "trace", "patch"]

var x = URLRequest(url: URL(string: "/hello")!)

for m in methods {
   x.httpMethod = m
   print(x.httpMethod!)
}

Output on Darwin:
GET
HEAD
POST
PUT
DELETE
CONNECT
options
trace
patch

Currently on Linux we don't do any capitalization so I'd like to fix this.

Is my list of 6 methods above the definitive list of which HTTP
methods should be capitalized?

Thanks,

--
Ian Partridge
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

There is a TestFoundation target in the swift-corelibs-foundation project, which can allow the tests to be run against the open source codebase.

Note that there's no requirement for the methods to be capitalised in URLRequest. Chances are that the implementation is such that there are some pre-defined values which can be used/replaced for keys, but other ones will take the case of whatever you put in them.

I also don't think it makes sense to capitalise everything because in most cases it will have no effect, but is wasted computation. So in other words, don't pass lowercase values into the x.httpMethod if you don't want it.

Alex

···

On 16 Nov 2017, at 17:41, Stephen Celis via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:

Brandon Williams and I have come across a lot of inconsistencies between Foundations in our Swift web work. We’ve been trying to file bugs when we remember to, but I’m curious if there’s a better way to catch these. Is the Foundation test suite run against both implementations to attempt to catch these kinds of things? If not it would seem like a big win to do so. If so, I suppose we could help by beefing up the test coverage?

Stephen

On Nov 16, 2017, at 12:18 PM, Ian Partridge via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:

Hi, it looks like Foundation on Darwin capitalises certain HTTP
methods but not others:

let methods = ["get", "head", "post", "put", "delete", "connect",
"options", "trace", "patch"]

var x = URLRequest(url: URL(string: "/hello")!)

for m in methods {
  x.httpMethod = m
  print(x.httpMethod!)
}

Output on Darwin:
GET
HEAD
POST
PUT
DELETE
CONNECT
options
trace
patch

Currently on Linux we don't do any capitalization so I'd like to fix this.

Is my list of 6 methods above the definitive list of which HTTP
methods should be capitalized?

Thanks,

--
Ian Partridge
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Brandon Williams and I have come across a lot of inconsistencies between Foundations in our Swift web work. We’ve been trying to file bugs when we remember to, but I’m curious if there’s a better way to catch these.

Please continue to file bugs! Even better, open a PR with a fix and a
testcase :slight_smile:

Is the Foundation test suite run against both implementations to attempt to catch these kinds of things?
If not it would seem like a big win to do so.

Not yet, but there is work underway to do this. See
https://github.com/apple/swift-corelibs-foundation/pull/1286 which
adds an Xcode project to enable just that. The goal is to get this
into CI so that we automatically run the SCLF testsuite against both
implementations.

If so, I suppose we could help by beefing up the test coverage?

That would be fantastic, we always need more tests. We cannot build
confidence in the SCLF implementation without a robust testsuite that
passes against both implementations (thinks Java TCK...)

···

On 16 November 2017 at 17:41, Stephen Celis <stephen.celis@gmail.com> wrote:

--
Ian Partridge

There is a TestFoundation target in the swift-corelibs-foundation project, which can allow the tests to be run against the open source codebase.

Sorry, maybe I wasn’t clear, I was wondering if there’s a test suite that regularly runs against _both_ the open-source Foundation implementation _and_ the closed-source Foundation implementation in order to catch inconsistencies across code bases.

Note that there's no requirement for the methods to be capitalised in URLRequest. Chances are that the implementation is such that there are some pre-defined values which can be used/replaced for keys, but other ones will take the case of whatever you put in them.

I also don't think it makes sense to capitalise everything because in most cases it will have no effect, but is wasted computation. So in other words, don't pass lowercase values into the x.httpMethod if you don't want it.

I don’t have a strong enough opinion to argue for or against auto-capitalizing the HTTP method, but I _do_ care for consistency across platforms. We have a significant test suite in our code bases with a lot of unit tests and snapshot tests that pass on our dev machines (Mac), but fail on Linux.

Stephen

···

On Nov 16, 2017, at 2:02 PM, Alex Blewitt <alblue@apple.com> wrote:

Fantastic! That’s the information I was looking for, thanks!

Stephen

···

On Nov 16, 2017, at 3:30 PM, Ian Partridge <ian@poncho.org.uk> wrote:

Is the Foundation test suite run against both implementations to attempt to catch these kinds of things?
If not it would seem like a big win to do so.

Not yet, but there is work underway to do this. See
https://github.com/apple/swift-corelibs-foundation/pull/1286 which
adds an Xcode project to enable just that. The goal is to get this
into CI so that we automatically run the SCLF testsuite against both
implementations.

There is a TestFoundation target in the swift-corelibs-foundation project, which can allow the tests to be run against the open source codebase.

Sorry, maybe I wasn’t clear, I was wondering if there’s a test suite that regularly runs against _both_ the open-source Foundation implementation _and_ the closed-source Foundation implementation in order to catch inconsistencies across code bases.

I understood your question, but I can only point to you as to what is available and run on the swift-corelibs-foundation open source project.

I don’t have a strong enough opinion to argue for or against auto-capitalizing the HTTP method, but I _do_ care for consistency across platforms. We have a significant test suite in our code bases with a lot of unit tests and snapshot tests that pass on our dev machines (Mac), but fail on Linux.

Yes, being consistent between platforms is one of the purposes of the swift-corelibs-foundation project. However, there are both run-time and implementation differences between the two platforms; the existence (or not) of the Objective-C runtime, whether or not comparisons are performed using the rules based on the Darwin version of Foundation or the ICU rules on the open-source version of Foundation, and so on.

Some of the differences are just bugs, in which case we can try and resolve the issues when they're filed at bugs.swift.org <http://bugs.swift.org/> or via pull requests on the GitHub repository. Some of the issues are ones where the implementation in the open source version is missing; either because the functionally cannot be implemented (e.g. dynamic unloading of bundles) or because it's not been prioritised yet (e.g. https://github.com/apple/swift-corelibs-foundation/search?utf8=✓&q=nsunimplemented <https://github.com/apple/swift-corelibs-foundation/search?utf8=✓&q=nsunimplemented>). In particular the big items that are missing at the moment are encoder/decoder implementations and those relating to nspredicate/expression, which both can be implemented in runtimes that have dynamic introspection of data structures but which therefore can't be implemented in Linux.

Having said that, if you do find consistency issues which are important to you then please file bugs, and optionally provide pull requests for them. We can help on the mailing list to try and resolve any issues or questions as they arise.

Alex

···

On 16 Nov 2017, at 19:14, Stephen Celis <stephen.celis@gmail.com> wrote:

On Nov 16, 2017, at 2:02 PM, Alex Blewitt <alblue@apple.com> wrote:

Note that there's no requirement for the methods to be capitalised in URLRequest.

I'm not sure what you mean by requirement. It may not be documented,
but Darwin does behave this way.

Chances are that the implementation is such that there are some pre-defined values which can be used/replaced for keys, but other ones will take the case of whatever you put in them.

Perhaps. That's what I'd like to enquire about. Is making SCLF
compatible with the output in my example sufficient, or are there
other factors to consider?

I also don't think it makes sense to capitalise everything because in most cases it will have no effect, but is wasted computation.

Darwin doesn't capitalise everything, as I've shown, so SCLF shouldn't either.

So in other words, don't pass lowercase values into the x.httpMethod if you don't want it.

I'm sorry, I don't understand this.

x.httpMethod accepts an arbitrary String - you can write `x.httpMethod
= frobnicate` if you want.

Darwin seems to treat some values e.g. "get" specially though. This
isn't documented, but SCLF should behave the same because there's code
out there that does `x.httpMethod = "get"`, `y.httpMethod = "GET",
compares the URLRequests and expects them to be equal.

···

On 16 November 2017 at 19:02, Alex Blewitt <alblue@apple.com> wrote:

--
Ian Partridge

I understood your question, but I can only point to you as to what is available and run on the swift-corelibs-foundation open source project.

Sorry to continue on this train (especially since it’s a big of a derailment). I’m really not trying to be obtuse, but I don’t understand your answer. I was asking what I thought were simple yes/no questions that didn’t require any private/confidential information about the closed source code. Do you or does anyone else know if the open-source project has a test target that runs its test suite against the closed-source implementation? If not, shouldn’t it?

I read your first response as: swift-corelibs-foundation has a test suite that runs against its own implementation. Am I missing something?

Yes, being consistent between platforms is one of the purposes of the swift-corelibs-foundation project. However, there are both run-time and implementation differences between the two platforms; the existence (or not) of the Objective-C runtime, whether or not comparisons are performed using the rules based on the Darwin version of Foundation or the ICU rules on the open-source version of Foundation, and so on.

I understand this, but I’m not talking about run-time/implementation differences, just testable input/output expectations on the functions and methods that swift-corelibs-foundation can implement. Assuming that the functions and methods swift-corelibs-foundation can implement should behave the same, its test suite should be able to run against Darwin Foundation.

Some of the differences are just bugs, in which case we can try and resolve the issues when they're filed at bugs.swift.org <http://bugs.swift.org/> or via pull requests on the GitHub repository. Some of the issues are ones where the implementation in the open source version is missing; either because the functionally cannot be implemented (e.g. dynamic unloading of bundles) or because it's not been prioritised yet (e.g. https://github.com/apple/swift-corelibs-foundation/search?utf8=✓&q=nsunimplemented <https://github.com/apple/swift-corelibs-foundation/search?utf8=✓&q=nsunimplemented>). In particular the big items that are missing at the moment are encoder/decoder implementations and those relating to nspredicate/expression, which both can be implemented in runtimes that have dynamic introspection of data structures but which therefore can't be implemented in Linux.

Having said that, if you do find consistency issues which are important to you then please file bugs, and optionally provide pull requests for them. We can help on the mailing list to try and resolve any issues or questions as they arise.

I totally have been and will continue to! I was hopeful that we could discuss a system that automates some of this and can catch regressions and inconsistencies more quickly. Is there any reason not to discuss such a solution?

Stephen

···

On Nov 16, 2017, at 2:57 PM, Alex Blewitt <alblue@apple.com> wrote: