swift-corelibs-dev Digest, Vol 1, Issue 18

Development Core Libraries
1

While I totally agree that that is a great long term goal, getting Swift running JSON parsing on the server as fast as humanly possible is a priority for many people. Would it be acceptable to vendor a minimal JSON tokenizer like jsmn to handle the brunt of the implementation, until such time as we can write one ourselves?

Tom, I’d love to collaborate with you on NSJSONSerialization and take some ownership of enhancing this crucial slice of Foundation. It’ll touch on many aspects of Foundation’s data types and as such will make a great platform for unearthing bugs elsewhere and serving as an integration test.

Thomas

···

On 4 Dec 2015, at 21:09, swift-corelibs-dev-request@swift.org wrote:

Send swift-corelibs-dev mailing list submissions to
  swift-corelibs-dev@swift.org

To subscribe or unsubscribe via the World Wide Web, visit
  https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
or, via email, send a message with subject or body 'help' to
  swift-corelibs-dev-request@swift.org

You can reach the person managing the list at
  swift-corelibs-dev-owner@swift.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of swift-corelibs-dev digest..."

Today's Topics:

  1. Re: Rules on adding dependencies (Tom Leavy)
  2. Re: Rules on adding dependencies (Tony Parker)

----------------------------------------------------------------------

Message: 1
Date: Fri, 4 Dec 2015 21:01:43 +0000
From: Tom Leavy <tom@wickr.com>
To: "anthony.parker@apple.com" <anthony.parker@apple.com>
Cc: "swift-corelibs-dev@swift.org" <swift-corelibs-dev@swift.org>
Subject: Re: [swift-corelibs-dev] Rules on adding dependencies
Message-ID: <60882613-69F6-4F75-9FDE-AACE85EA73AD@wickr.com>
Content-Type: text/plain; charset="utf-8"

Thanks!

Is there a list of current dependencies someplace? I understand you use a specific version of ICU as part of CoreFoundation.

Thomas Leavy | Wickr Inc.
VP Mobile Applications, Architecture | Newark, NJ

On Dec 4, 2015, at 3:52 PM, anthony.parker@apple.com<mailto:anthony.parker@apple.com> wrote:

Good idea, I filed a bug for that here.

[SR-59] Add information about dependencies for core libraries to README.md · Issue #4207 · apple/swift-corelibs-foundation · GitHub

- Tony

On Dec 4, 2015, at 12:49 PM, Tom Leavy <tom@wickr.com<mailto:tom@wickr.com>> wrote:

Hey Tony,

That sounds like a good objective overall, thanks for clarifying. I think you guys should write that someplace in the documentation on github (if it hasn’t been already)

Thomas Leavy | Wickr Inc.
VP Mobile Applications, Architecture | Newark, NJ

On Dec 4, 2015, at 2:39 PM, anthony.parker@apple.com<mailto:anthony.parker@apple.com> wrote:

Hi Tom,

I believe it’s important for us to keep our dependencies to an absolute minimum for the core libraries, since they will be distributed very widely. I would prefer to have our own implementation of the JSON parser in Foundation rather than use one from another project.

The one from Darwin is about 2k lines of Objective-C (actually, mostly C).

- Tony

On Dec 4, 2015, at 11:21 AM, Tom Leavy <tom@wickr.com<mailto:tom@wickr.com>> wrote:

I was wondering what the official position on introducing dependencies currently is. I would love to contribute an implementation of NSJSONSerialization. I already have written a wrapper that uses yajl to take a CFDictionaryRef or CFArrayRef and encodes them into CFDataRef or CFStringRef JSON values.. Is it acceptable to use yajl and introduce a dependency, or is it a goal of the foundation project to supply its own implementation of a JSON parser?
This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org<mailto:swift-corelibs-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-corelibs-dev/attachments/20151204/3b3aa317/attachment-0001.html&gt;

------------------------------

Message: 2
Date: Fri, 04 Dec 2015 13:09:09 -0800
From: Tony Parker <anthony.parker@apple.com>
To: Tom Leavy <tom@wickr.com>
Cc: "swift-corelibs-dev@swift.org" <swift-corelibs-dev@swift.org>
Subject: Re: [swift-corelibs-dev] Rules on adding dependencies
Message-ID: <1D8FA53C-37CD-4653-8BDD-6C9F6DEE5015@apple.com>
Content-Type: text/plain; charset="utf-8"

On Dec 4, 2015, at 1:01 PM, Tom Leavy <tom@wickr.com> wrote:

Thanks!

Is there a list of current dependencies someplace? I understand you use a specific version of ICU as part of CoreFoundation.

The best source of this info is going to be the build.py file and Xcode project.

You are correct that on Apple platforms we use a specific version of ICU (which is itself open source, on opensource.apple.com). For now, on Linux, we try to use the installed one. There are certainly some areas where we require the functionality provided by the Apple ICU but we have just omitted those from the build on Linux for the time being (look for __HAS_APPLE_ICU__ in CF). Eventually we’ll have to figure out a unified story for this.

- Tony

Thomas Leavy | Wickr Inc.
VP Mobile Applications, Architecture | Newark, NJ

On Dec 4, 2015, at 3:52 PM, anthony.parker@apple.com <mailto:anthony.parker@apple.com> wrote:

Good idea, I filed a bug for that here.

[SR-59] Add information about dependencies for core libraries to README.md · Issue #4207 · apple/swift-corelibs-foundation · GitHub

- Tony

On Dec 4, 2015, at 12:49 PM, Tom Leavy <tom@wickr.com <mailto:tom@wickr.com>> wrote:

Hey Tony,

That sounds like a good objective overall, thanks for clarifying. I think you guys should write that someplace in the documentation on github (if it hasn’t been already)

Thomas Leavy | Wickr Inc.
VP Mobile Applications, Architecture | Newark, NJ

On Dec 4, 2015, at 2:39 PM, anthony.parker@apple.com <mailto:anthony.parker@apple.com> wrote:

Hi Tom,

I believe it’s important for us to keep our dependencies to an absolute minimum for the core libraries, since they will be distributed very widely. I would prefer to have our own implementation of the JSON parser in Foundation rather than use one from another project.

The one from Darwin is about 2k lines of Objective-C (actually, mostly C).

- Tony

On Dec 4, 2015, at 11:21 AM, Tom Leavy <tom@wickr.com <mailto:tom@wickr.com>> wrote:

I was wondering what the official position on introducing dependencies currently is. I would love to contribute an implementation of NSJSONSerialization. I already have written a wrapper that uses yajl to take a CFDictionaryRef or CFArrayRef and encodes them into CFDataRef or CFStringRef JSON values.. Is it acceptable to use yajl and introduce a dependency, or is it a goal of the foundation project to supply its own implementation of a JSON parser?
This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-corelibs-dev/attachments/20151204/023274f1/attachment.html&gt;

------------------------------

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

End of swift-corelibs-dev Digest, Vol 1, Issue 18
*************************************************

2

I would rather go right to the end game here and just work on coding up an implementation of JSON encoding / decoding and making it conform to the NSJSONSerialization spec. I can set up a repo for us to work on it.

The main decision here, is do we code a small lib in C and then call it from the Swift side, or do we just write the entire implementation in Swift? From a performance standpoint C is usually my go to, but if we use swift with only structs and we really focus on minimizing use of functions that would be costly / do a lot of profiling and optimization we can get very close to equivalent performance? At least to the level of the current objective c / c version? I'm also not sure of potential performance penalties of calling C functions from swift. Someone with more knowledge will have to weigh in on that one.

Thomas Leavy | Wickr Inc.
VP Mobile Applications & Architecture | Newark, NJ<x-apple-data-detectors://0/1>

···

On Dec 4, 2015, at 6:24 PM, swizzlr <me@swizzlr.co<mailto:me@swizzlr.co>> wrote:

Re: Rules on adding dependencies

This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

3

I’m not sure about performance hit of calling C from Swift either (it should be statically linked, so I suspect there is none), but the performance is only marginally slower than NSJSONSerialization as is, in the pure Swift version I am making. My test character set is about 2000 characters long, running serialize on it 5000 times, my current Swift-only version takes 1 second for each test run, and the existing (C-backed) NSJSONSerialization takes about 0.75 seconds.

The difference isn’t huge and I suspect with more optimisation of the Swift code and of the standard library (in particular comparing UTF16 strings) this gap would disappear. I for one would be in favour of a pure Swift solution for readability, and also as a challenge to improve the standard library’s performance.

Another thing to note is that Swift is supposed to be future-proofed in terms of UTF String handling (I’m not sure about JSON though, to be honest – is it supposed to support CharSets beyond UTF8?), maybe having the strings decode in UTF16 by default is not such a bad idea, even if it has a minor performance hit for now.

···

On Sat, Dec 5, 2015 at 5:17 PM, Tom Leavy <tom@wickr.com> wrote:

I would rather go right to the end game here and just work on coding up an implementation of JSON encoding / decoding and making it conform to the NSJSONSerialization spec. I can set up a repo for us to work on it.
The main decision here, is do we code a small lib in C and then call it from the Swift side, or do we just write the entire implementation in Swift? From a performance standpoint C is usually my go to, but if we use swift with only structs and we really focus on minimizing use of functions that would be costly / do a lot of profiling and optimization we can get very close to equivalent performance? At least to the level of the current objective c / c version? I'm also not sure of potential performance penalties of calling C functions from swift. Someone with more knowledge will have to weigh in on that one.
Thomas Leavy | Wickr Inc.
VP Mobile Applications & Architecture | Newark, NJ<x-apple-data-detectors://0/1>
On Dec 4, 2015, at 6:24 PM, swizzlr <me@swizzlr.co<mailto:me@swizzlr.co>> wrote:
Re: Rules on adding dependencies
This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

4

What I’m not so sure about is the most efficient way of working with NSData in this context, considering we are very much assuming the data to have come from a String of some kind (correct me if I’m wrong). So it seems to me like taking NSData insead of just a String is a regression into Objective-C land.

Can anyone explain why we might want to do this rather than working in an idiomatic Swift way (obviously backwards compatibility is a big one – is it the only one?). The only other thing I can think of is that NSData can be streamed. That is going into territory that I haven’t dealt with in Swift yet though. Would love to hear some input from others on this.

···

On Sat, Dec 5, 2015 at 5:29 PM, Geordie Jay <geojay@gmail.com> wrote:

I’m not sure about performance hit of calling C from Swift either (it should be statically linked, so I suspect there is none), but the performance is only marginally slower than NSJSONSerialization as is, in the pure Swift version I am making. My test character set is about 2000 characters long, running serialize on it 5000 times, my current Swift-only version takes 1 second for each test run, and the existing (C-backed) NSJSONSerialization takes about 0.75 seconds.
The difference isn’t huge and I suspect with more optimisation of the Swift code and of the standard library (in particular comparing UTF16 strings) this gap would disappear. I for one would be in favour of a pure Swift solution for readability, and also as a challenge to improve the standard library’s performance.
Another thing to note is that Swift is supposed to be future-proofed in terms of UTF String handling (I’m not sure about JSON though, to be honest – is it supposed to support CharSets beyond UTF8?), maybe having the strings decode in UTF16 by default is not such a bad idea, even if it has a minor performance hit for now.
On Sat, Dec 5, 2015 at 5:17 PM, Tom Leavy <tom@wickr.com> wrote:

I would rather go right to the end game here and just work on coding up an implementation of JSON encoding / decoding and making it conform to the NSJSONSerialization spec. I can set up a repo for us to work on it.
The main decision here, is do we code a small lib in C and then call it from the Swift side, or do we just write the entire implementation in Swift? From a performance standpoint C is usually my go to, but if we use swift with only structs and we really focus on minimizing use of functions that would be costly / do a lot of profiling and optimization we can get very close to equivalent performance? At least to the level of the current objective c / c version? I'm also not sure of potential performance penalties of calling C functions from swift. Someone with more knowledge will have to weigh in on that one.
Thomas Leavy | Wickr Inc.
VP Mobile Applications & Architecture | Newark, NJ<x-apple-data-detectors://0/1>
On Dec 4, 2015, at 6:24 PM, swizzlr <me@swizzlr.co<mailto:me@swizzlr.co>> wrote:
Re: Rules on adding dependencies
This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

5

Sorry to spam, but I’ve just run some tests re: memory usage and found that running NSJSONSerialization.JSONObjectWithData() repeatedly results in huge memory usage (unbounded into the gigabytes). The pure Swift version I’ve been working on remains at a constant and minimal ~2 MB memory usage.

I’m not sure what the reason for this is huge discrepancy is. I assume the current Foundation version is using as much memory as it can for efficiency, but this may be another reason not to trust NSJSONSerialization is the best possible outcome, especially for embedded systems, servers, and the like. Considering the minimal CPU time difference, I would again vote for a pure Swift version, and focus on improving the standard library decoding speeds if possible.

···

On Sat, Dec 5, 2015 at 5:58 PM, Geordie Jay <geojay@gmail.com> wrote:

What I’m not so sure about is the most efficient way of working with NSData in this context, considering we are very much assuming the data to have come from a String of some kind (correct me if I’m wrong). So it seems to me like taking NSData insead of just a String is a regression into Objective-C land.
Can anyone explain why we might want to do this rather than working in an idiomatic Swift way (obviously backwards compatibility is a big one – is it the only one?). The only other thing I can think of is that NSData can be streamed. That is going into territory that I haven’t dealt with in Swift yet though. Would love to hear some input from others on this.
On Sat, Dec 5, 2015 at 5:29 PM, Geordie Jay <geojay@gmail.com> wrote:

I’m not sure about performance hit of calling C from Swift either (it should be statically linked, so I suspect there is none), but the performance is only marginally slower than NSJSONSerialization as is, in the pure Swift version I am making. My test character set is about 2000 characters long, running serialize on it 5000 times, my current Swift-only version takes 1 second for each test run, and the existing (C-backed) NSJSONSerialization takes about 0.75 seconds.
The difference isn’t huge and I suspect with more optimisation of the Swift code and of the standard library (in particular comparing UTF16 strings) this gap would disappear. I for one would be in favour of a pure Swift solution for readability, and also as a challenge to improve the standard library’s performance.
Another thing to note is that Swift is supposed to be future-proofed in terms of UTF String handling (I’m not sure about JSON though, to be honest – is it supposed to support CharSets beyond UTF8?), maybe having the strings decode in UTF16 by default is not such a bad idea, even if it has a minor performance hit for now.
On Sat, Dec 5, 2015 at 5:17 PM, Tom Leavy <tom@wickr.com> wrote:

I would rather go right to the end game here and just work on coding up an implementation of JSON encoding / decoding and making it conform to the NSJSONSerialization spec. I can set up a repo for us to work on it.
The main decision here, is do we code a small lib in C and then call it from the Swift side, or do we just write the entire implementation in Swift? From a performance standpoint C is usually my go to, but if we use swift with only structs and we really focus on minimizing use of functions that would be costly / do a lot of profiling and optimization we can get very close to equivalent performance? At least to the level of the current objective c / c version? I'm also not sure of potential performance penalties of calling C functions from swift. Someone with more knowledge will have to weigh in on that one.
Thomas Leavy | Wickr Inc.
VP Mobile Applications & Architecture | Newark, NJ<x-apple-data-detectors://0/1>
On Dec 4, 2015, at 6:24 PM, swizzlr <me@swizzlr.co<mailto:me@swizzlr.co>> wrote:
Re: Rules on adding dependencies
This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

6

Hi Geordie,

The choice of NSData instead of String was very deliberate for NSJSONSerialization. The reason is that JSON is received from disk or the network as a data blob, not a String. Converting it into a String requires figuring out the encoding and doing a conversion. NSJSONSerialization looks at the leading bytes and then parses the data directly instead of converting the entire thing to a String first. This is a major perf boost, since there is usually a huge portion of the data which never needs to be a string (numbers, for example). Only actual result Strings need the conversion.

On a higher-level note, I agree with the approach of implementing the existing API first before considering alternate APIs. There is no doubt that dealing with untyped JSON output in Swift results in a less than ideal experience of casts or other things. I want to think about how to improve it, but first I want to make sure we have our cross-platform story in place. This means that clients of Foundation can simply use the same NSJSONSerialization API that they are used to and get the results that they expect.

- Tony

···

On Dec 5, 2015, at 8:58 AM, Geordie Jay <geojay@gmail.com> wrote:

What I’m not so sure about is the most efficient way of working with NSData in this context, considering we are very much assuming the data to have come from a String of some kind (correct me if I’m wrong). So it seems to me like taking NSData insead of just a String is a regression into Objective-C land.

Can anyone explain why we might want to do this rather than working in an idiomatic Swift way (obviously backwards compatibility is a big one – is it the only one?). The only other thing I can think of is that NSData can be streamed. That is going into territory that I haven’t dealt with in Swift yet though. Would love to hear some input from others on this.

On Sat, Dec 5, 2015 at 5:29 PM, Geordie Jay <geojay@gmail.com <mailto:geojay@gmail.com>> wrote:

I’m not sure about performance hit of calling C from Swift either (it should be statically linked, so I suspect there is none), but the performance is only marginally slower than NSJSONSerialization as is, in the pure Swift version I am making. My test character set is about 2000 characters long, running serialize on it 5000 times, my current Swift-only version takes 1 second for each test run, and the existing (C-backed) NSJSONSerialization takes about 0.75 seconds.

The difference isn’t huge and I suspect with more optimisation of the Swift code and of the standard library (in particular comparing UTF16 strings) this gap would disappear. I for one would be in favour of a pure Swift solution for readability, and also as a challenge to improve the standard library’s performance.

Another thing to note is that Swift is supposed to be future-proofed in terms of UTF String handling (I’m not sure about JSON though, to be honest – is it supposed to support CharSets beyond UTF8?), maybe having the strings decode in UTF16 by default is not such a bad idea, even if it has a minor performance hit for now.

On Sat, Dec 5, 2015 at 5:17 PM, Tom Leavy <tom@wickr.com <mailto:tom@wickr.com>> wrote:

I would rather go right to the end game here and just work on coding up an implementation of JSON encoding / decoding and making it conform to the NSJSONSerialization spec. I can set up a repo for us to work on it.

The main decision here, is do we code a small lib in C and then call it from the Swift side, or do we just write the entire implementation in Swift? From a performance standpoint C is usually my go to, but if we use swift with only structs and we really focus on minimizing use of functions that would be costly / do a lot of profiling and optimization we can get very close to equivalent performance? At least to the level of the current objective c / c version? I'm also not sure of potential performance penalties of calling C functions from swift. Someone with more knowledge will have to weigh in on that one.

Thomas Leavy | Wickr Inc.
VP Mobile Applications & Architecture | Newark, NJ <x-apple-data-detectors://0/1>

On Dec 4, 2015, at 6:24 PM, swizzlr <me@swizzlr.co <mailto:me@swizzlr.co>> wrote:

Re: Rules on adding dependencies

This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

7

I started on the pretty printing side of it last night. There are going to be issues when Swift types are used within object hierarchy as the AnyObject bridging makes the original type unknown, especially true for the Swift.Bool type.

I'm not sure why we'd want to use C though. That seems to defeat the point.

···

Sent from my iPhone

On Dec 5, 2015, at 9:17 AM, Geordie Jay <geojay@gmail.com> wrote:

Sorry to spam, but I’ve just run some tests re: memory usage and found that running NSJSONSerialization.JSONObjectWithData() repeatedly results in huge memory usage (unbounded into the gigabytes). The pure Swift version I’ve been working on remains at a constant and minimal ~2 MB memory usage.

I’m not sure what the reason for this is huge discrepancy is. I assume the current Foundation version is using as much memory as it can for efficiency, but this may be another reason not to trust NSJSONSerialization is the best possible outcome, especially for embedded systems, servers, and the like. Considering the minimal CPU time difference, I would again vote for a pure Swift version, and focus on improving the standard library decoding speeds if possible.

On Sat, Dec 5, 2015 at 5:58 PM, Geordie Jay <geojay@gmail.com> wrote:
What I’m not so sure about is the most efficient way of working with NSData in this context, considering we are very much assuming the data to have come from a String of some kind (correct me if I’m wrong). So it seems to me like taking NSData insead of just a String is a regression into Objective-C land.

Can anyone explain why we might want to do this rather than working in an idiomatic Swift way (obviously backwards compatibility is a big one – is it the only one?). The only other thing I can think of is that NSData can be streamed. That is going into territory that I haven’t dealt with in Swift yet though. Would love to hear some input from others on this.

On Sat, Dec 5, 2015 at 5:29 PM, Geordie Jay <geojay@gmail.com> wrote:
I’m not sure about performance hit of calling C from Swift either (it should be statically linked, so I suspect there is none), but the performance is only marginally slower than NSJSONSerialization as is, in the pure Swift version I am making. My test character set is about 2000 characters long, running serialize on it 5000 times, my current Swift-only version takes 1 second for each test run, and the existing (C-backed) NSJSONSerialization takes about 0.75 seconds.

The difference isn’t huge and I suspect with more optimisation of the Swift code and of the standard library (in particular comparing UTF16 strings) this gap would disappear. I for one would be in favour of a pure Swift solution for readability, and also as a challenge to improve the standard library’s performance.

Another thing to note is that Swift is supposed to be future-proofed in terms of UTF String handling (I’m not sure about JSON though, to be honest – is it supposed to support CharSets beyond UTF8?), maybe having the strings decode in UTF16 by default is not such a bad idea, even if it has a minor performance hit for now.

On Sat, Dec 5, 2015 at 5:17 PM, Tom Leavy <tom@wickr.com> wrote:
I would rather go right to the end game here and just work on coding up an implementation of JSON encoding / decoding and making it conform to the NSJSONSerialization spec. I can set up a repo for us to work on it.

The main decision here, is do we code a small lib in C and then call it from the Swift side, or do we just write the entire implementation in Swift? From a performance standpoint C is usually my go to, but if we use swift with only structs and we really focus on minimizing use of functions that would be costly / do a lot of profiling and optimization we can get very close to equivalent performance? At least to the level of the current objective c / c version? I'm also not sure of potential performance penalties of calling C functions from swift. Someone with more knowledge will have to weigh in on that one.

Thomas Leavy | Wickr Inc.
VP Mobile Applications & Architecture | Newark, NJ

On Dec 4, 2015, at 6:24 PM, swizzlr <me@swizzlr.co> wrote:

Re: Rules on adding dependencies

This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

8

Sorry to spam, but I’ve just run some tests re: memory usage and found that running NSJSONSerialization.JSONObjectWithData() repeatedly results in huge memory usage (unbounded into the gigabytes). The pure Swift version I’ve been working on remains at a constant and minimal ~2 MB memory usage.

I’d be interested in seeing your test case. I certainly didn’t intend for NSJSONSerialization to take gigabytes of memory. I wonder if it’s possible that an autorelease pool is simply never being popped when calling from Swift.

- Tony

···

On Dec 5, 2015, at 9:17 AM, Geordie Jay <geojay@gmail.com> wrote:

I’m not sure what the reason for this is huge discrepancy is. I assume the current Foundation version is using as much memory as it can for efficiency, but this may be another reason not to trust NSJSONSerialization is the best possible outcome, especially for embedded systems, servers, and the like. Considering the minimal CPU time difference, I would again vote for a pure Swift version, and focus on improving the standard library decoding speeds if possible.

On Sat, Dec 5, 2015 at 5:58 PM, Geordie Jay <geojay@gmail.com <mailto:geojay@gmail.com>> wrote:

What I’m not so sure about is the most efficient way of working with NSData in this context, considering we are very much assuming the data to have come from a String of some kind (correct me if I’m wrong). So it seems to me like taking NSData insead of just a String is a regression into Objective-C land.

Can anyone explain why we might want to do this rather than working in an idiomatic Swift way (obviously backwards compatibility is a big one – is it the only one?). The only other thing I can think of is that NSData can be streamed. That is going into territory that I haven’t dealt with in Swift yet though. Would love to hear some input from others on this.

On Sat, Dec 5, 2015 at 5:29 PM, Geordie Jay <geojay@gmail.com <mailto:geojay@gmail.com>> wrote:

I’m not sure about performance hit of calling C from Swift either (it should be statically linked, so I suspect there is none), but the performance is only marginally slower than NSJSONSerialization as is, in the pure Swift version I am making. My test character set is about 2000 characters long, running serialize on it 5000 times, my current Swift-only version takes 1 second for each test run, and the existing (C-backed) NSJSONSerialization takes about 0.75 seconds.

The difference isn’t huge and I suspect with more optimisation of the Swift code and of the standard library (in particular comparing UTF16 strings) this gap would disappear. I for one would be in favour of a pure Swift solution for readability, and also as a challenge to improve the standard library’s performance.

Another thing to note is that Swift is supposed to be future-proofed in terms of UTF String handling (I’m not sure about JSON though, to be honest – is it supposed to support CharSets beyond UTF8?), maybe having the strings decode in UTF16 by default is not such a bad idea, even if it has a minor performance hit for now.

On Sat, Dec 5, 2015 at 5:17 PM, Tom Leavy <tom@wickr.com <mailto:tom@wickr.com>> wrote:

I would rather go right to the end game here and just work on coding up an implementation of JSON encoding / decoding and making it conform to the NSJSONSerialization spec. I can set up a repo for us to work on it.

The main decision here, is do we code a small lib in C and then call it from the Swift side, or do we just write the entire implementation in Swift? From a performance standpoint C is usually my go to, but if we use swift with only structs and we really focus on minimizing use of functions that would be costly / do a lot of profiling and optimization we can get very close to equivalent performance? At least to the level of the current objective c / c version? I'm also not sure of potential performance penalties of calling C functions from swift. Someone with more knowledge will have to weigh in on that one.

Thomas Leavy | Wickr Inc.
VP Mobile Applications & Architecture | Newark, NJ <x-apple-data-detectors://0/1>

On Dec 4, 2015, at 6:24 PM, swizzlr <me@swizzlr.co <mailto:me@swizzlr.co>> wrote:

Re: Rules on adding dependencies

This e-mail message is intended only for the named recipient(s) above and is covered by the Electronic Communications Privacy Act 18 U.S.C. Section 2510-2521. This e-mail is confidential and may contain information that is privileged or exempt from disclosure under applicable law. If you have received this message in error please immediately notify the sender by return e-mail and delete this e-mail message from your computer, mobile devices and any cloud storage backup systems as well as destroy any printed copy you might have made.

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev