100% bikeshed topic: DictionaryLiteral

Evolution Discussion
41

Right. When I say “statically link” here, I really mean “put the .dylib for the library into the app bundle”. That’s what happens with Swift today. It can be important to share that code between apps and appex’s for example.

-Chris

···

On Jan 10, 2018, at 5:02 PM, Wallacy <wallacyf@gmail.com> wrote:

If I understand this correctly. This is exactly the same argument in favor to maintain the ABI unstable and always bundle the standard library and the runtime with the app.

Another alternative is, not only, separate the standard library on small pieces, but also dynamic link this specific version of this piece and keep all versions on the SO. (I think the Microsoft did something similar to this)

On Linux the package manager probably will handle this very well like any other library.

Even 100 versions in the same SO will not consume enough space to bother anyone.

Em qua, 10 de jan de 2018 às 17:04, Hooman Mehr via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> escreveu:
Excellent Idea!

I am all for this. It shouldn’t be too complicated to add a second implicit import and only code that is actually using this stuff will have increased code size.

> On Jan 9, 2018, at 10:29 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
>
> Disclaimer: I’m reordering your comments below to suit my nefarious purposes: :-)
>
> On Jan 9, 2018, at 3:48 PM, Ben Cohen <ben_cohen@apple.com <mailto:ben_cohen@apple.com>> wrote:
>>> More to the point though, this seems like an implementation detail of Mirrors. What is the plan for Mirrors + ABI stability?
>>>
>>
>> Absent a proposal to change them to something better (and I’m not aware of one pending), the plan is they are what they are, at least the public API of Swift.Mirror. I would guess this whole area is a candidate to be overhauled significantly at some point in the post-Swift 5 future with some more fully-fledged reflection system, but there is little chance of this happening before ABI stability, and interim tinkering doesn’t seem worthwhile to me.
>
> Ok, I understand that (among all the other things going on that are clearly more important) revamping this is probably not the highest priority thing to do. That said, it would be really unfortunate to carry around these “suboptimal” APIs forever, particularly given how marginal they are (as in, not widely used). I’m sure that there are other examples beyond these that are similarly unfortunate.
>
> Given that, I have a meta question for you: have you considered an approach where you take the Swift standard library and split it into two conceptual pieces:
>
> 1) The "ABI stable” subset of the library that gets burned into the OS.
> 2) The “ABI unstable” subset, which gets statically linked into apps, just like the Swift 4 library used to?
>
> Given that “import Swift” is implicit anyway, you could just have the compiler implicitly import *both* of these modules.
>
>
> The point of doing this is that it gives you a very general and low friction way to handle compatibility gunk. In addition to putting obscure stuff like Mirror and “DictionaryLiteral” into this (without breaking source code!) you now get the ability to put the various deprecated forwarding functions in this module as well, avoiding them becoming part of the ABI.
>
> The nice thing about this is that only people who use these things would have to pay the cost, and you can directly message this by deprecating all the stuff in it. Think about it as an overlay for the Swift standard library :-)
>
>>> +1 for renaming it to something that isn’t super confusing, but +100 for removing it outright.
>>>
>>> Rationale: if we didn’t have this functionality in the stdlib today, we would not consider adding it. It doesn’t meet the bar we’ve set for what is in the standard library. It only exists there by historical accident. The Mirror API was not carefully considered.
>>>
>>
>> Personally, I feel like this argument for removal as past its use-by date. It was a good reason for Swift 3, tenuous for 4 and should be ruled out for 5, since the source stability bar has been raised since. Like I said, IMO the criteria should now be “active harm”. I also don’t think searches of GitHub etc are sufficient justification, except when combined with the active-harm argument. I also don’t think the origin story of the type – whether accidental or intentional – is relevant to the decision of what to do with it now. It exists and people can legitimately use it for their own purposes independent of mirrors.
>
> I’m generally in agreement with you, but in this specific case, I seriously doubt people are using this. All rules are malleable in the right circumstances.
>
> More importantly though, if we had a meta design that allowed to gracefully phase this sort of thing out (without it becoming part of ABI under any name) there becomes very little cost to leaving it around in the “abi unstable” library for…. ever if need be. Given that, I would have much less objection to keeping it around.
>
> -Chris
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org <mailto:swift-evolution@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

42

For people who don’t read all the way down, here is the gist of what Chris is saying:

1. The vast majority of the standard library goes into the OS. This includes all the string, array, and other stuff people use.
2. The deprecated wrappers and obscure APIs go into a new module, and that module doesn’t go into the OS. Only people that use it pay to cost.

As I said, I totally agree.

43

Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur. I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.

It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.

In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS. Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.

With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.

- One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app. We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle

In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib. That solution would have the following characteristics:

- Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.

- Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs. This would be a benefit in the code size for the OS, but not for the app itself.

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application. They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

- Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib. This factorization can only happen once.

- There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs. It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around. We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs. At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.

With these points in mind, both scenarios are (by construction) very similar. Scenario (B) potentially provides a bit more flexibility in outright sunsetting deprecated APIs down the road — but only the APIs we deem today should be on that path. In the future when we decide to deprecate APIs in the Standard Library we will need to follow the same kind of deprecation-and-obsoletion path as other binary frameworks in macOS/iOS, which may mean keeping API implementations around for a very long time just for the purpose of binary compatibility.

If you agree with me up to this point, to me this boils down to a tradeoff of some slightly increased opportunity to sunset some APIs in the future for a slight reduction in size of the Standard Library today (as it ships in the OS). Most apps that don’t use these APIs won’t care either way, because even if we go with scenario B if they don’t use one of the deprecated APIs they will essentially be in scenario A (and thus get the benefit of using a Standard Library from the OS). In scenario B, those apps that use the deprecated APIs will be slightly punished with slightly increased app bundle size and startup time. Further, these APIs really can’t be deprecated right now (in terms of marking them deprecated in the module) until we have actual alternatives, so for the users that really want to use Mirrors today (which I agree may be small) that is the main tool they have to obtain that functionality. Such users will pay a cost — albeit probably pretty minor — in scenario B.

I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).

Do these observations align with what you are proposing, or am I missing a critical detail?

Ted

···

On Jan 11, 2018, at 11:56 AM, Chris Lattner <clattner@nondot.org> wrote:

On Jan 11, 2018, at 12:01 AM, Ted Kremenek <kremenek@apple.com <mailto:kremenek@apple.com>> wrote:

The nice thing about this is that only people who use these things would have to pay the cost, and you can directly message this by deprecating all the stuff in it. Think about it as an overlay for the Swift standard library :-)

Hi Chris,

Even if we split the APIs into two sets, with an “ABI unstable” subset, we are still going to “carry round these ’suboptimal APIs’” because clients are using them today and those APIs — without an active plan to remove them — *are* a part of the Standard Library. We’ve past the point in Swift where we can just rip them out (you said so yourself in a follow up email).

Who is “we” and how do you define carry?

I think you’re saying that they will remain part of the swift project (in the “SwiftDeprecated” module) for effectively ever. If so, I agree, and that’s the point. The point is that the code can continue to exist and service the few people who need it (and will become even fewer over the years) without causing *harm* to the Swift ecosystem over the near or long term. That’s the feature.

It feels to me that the main advantage of the overlay is a solution to remove out pieces of the Standard Library that will be used in practice less, and thus clients don’t “pay the cost” when they don’t use them.

Yes, exactly.

But I think we should be honest about what that “cost” actually is, as these APIs aren’t going away right now. These suboptimal APIs will stay around at least until better ones are created, and even then some existing clients will want to rely on the existing suboptimal APIs anyway even if others want to take advantage of newer, better APIs and idioms.

I’m not sure what you mean. I’m talking about two things specifically:

1) APIs that are already deprecated and have replacements. The deprecated version goes in this library, avoid carrying them around forever.

2) APIs that are used by extremely narrow audiences, which were added without much consideration and that have not been redesigned since Swift 1. Mirror’s are an example of this. I’m arguing that these should be moved to this SwiftDeprecated module - but not actually deprecated until a replacement exists.

I don’t see how this is bad for anyone, and I think this provides a significant process improvement compared to your current approach, which forces you to lock down APIs now without careful consideration just because they are not “important enough”.

The main cost I see here that we save by having an “overlay for the Standard library" is code size.

I don’t understand this point.

On macOS/iOS it is true that applications embed a copy of the Standard Library within them and thus pay a real cost in the Standard Library’s actually payload size in their own app. However, the only reason that is necessary today is because we don’t have ABI stability. With ABI stability, other options for shipping the Standard Library — such as shipping it in the OS — become possible, and thus the burden of shouldering that “cost” of code size shifts around. Then the tradeoffs of having an overlay for the Standard Library are different. In a world with ABI stability, is it better to have a Standard Library "overlay" that only some clients use, and embed within their application bundle, than just eat that cost in the Standard Library to be shared by potentially multiple clients installed on a system? I think it really comes down to the numbers. In that case, are we really just talking about Mirrors? I haven’t run the numbers here, but my intuition tells me we are talking about a relatively small code size impact overall.

This comment doesn’t make a lot of sense to me, perhaps we’re talking across each other. The point of the design that I’m advocating for is that all apps (that deploy to iOS-next or later) get a immediate improvement from abi stability: the vast majority of the standard library is not put into their app bundle. Further, the vastly most common apps - those that are not using mirrors or deprecated APIs, also do not include the SwiftDeprecated module.

Only the tiny minority of apps that do use these APIs would have a copy of SwiftDeprecated in their app dylib, but they have a path to fix that, and they would still see a huge improvement from the bulk of the standard library being out of their app bundle.

There’s also other potential performance implications from doing this split.

This doesn’t make sense to me. We’re not talking about taking comonly used apis here. Mirrors are not performance sensitive at all, and neither are deprecated wrappers of renamed APIs.

Being able to have the Standard Library be in the OS has potential major implications on the startup time of Swift applications.

It is great for app developers to have an incentive to stop using deprecated APIs! :-)

Can you elaborate a bit more on the specific “cost” factors you are using when proposing to have a Standard Library overlay?

I think it is possible that you’re reading too much into the “overlay" word here, so ignore that. The design I’m proposing is really simple:

1. The vast majority of the standard library goes into the OS. This includes all the string, array, and other stuff people use.
2. The deprecated wrappers and obscure APIs go into a new module, and that module doesn’t go into the OS. Only people that use it pay to cost.

This isn’t particularly complicated, if give you new ways to manage the process of stdlib ABI stability, and I don’t see the downside.

-Chris

44

How about renaming DictionaryLiteral to Row, TabularRow or TableRow?

I think most developers are familiar with the idea that a table row contains multiple columns (in specific order), and each column has a name and a value (key/value).

Some other name suggestions:
- Record (kind of an old name for table rows)
- SortedDictionary (sorted dictionaries are missing on the standard library, and could give a chance to make this type more widely used)

Thanks,
Eneko

···

On Jan 9, 2018, at 9:19 AM, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 9, 2018, at 11:00 AM, Gwendal Roué via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Le 9 janv. 2018 à 17:16, Zach Waldowski via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

I'm not sure a valid use case by a third party makes it hold its weight for inclusion in the stdlib.

You're definitely right, and that's why I wrote with the most humble tone I could.

Yet, the design of the stdlib *requires* some speculation about use cases, and speculation is *helped* by the exposition of actual uses. I'm not sure readers of the mailing list had any idea of the use cases of the current DictionaryLiteral, and maybe I helped a little.

Reproducing its feature set is extremely trivial, and would probably allow you to hint the implementation details better for your use case.

Please define "trivial”.

In case anybody would wonder, in the line below the `row` variable is of type Row which happens to adopt ExpressibleByDictionaryLiteral. It is not of type DictionaryLiteral. The use case here is the ability to express a row with a dictionary literal that accepts duplicated keys and preserves ordering:

  XCTAssertEqual(row, ["a": 1, "a": 2])

That’s great! In this case you aren’t using the DictionaryLiteral type, but a “dictionary literal”, which no one is suggesting we remove. If I’m understanding what you wrote, this is another case where the terrible name is making it super hard to discuss what we’re talking about. “Dictionary literals” and the ExpressibleByDictionaryLiteral protocol are safe!

I don't see how anything could better fit this use case than the current DictionaryLiteral. This is not *my* use case, but the use case of anyone that wants to model database rows beyond the traditional (and ill-advised) dictionary.

Some other users may come with other use cases that may also help the stdlib designers choose the best solution.

Gwendal

Zach
zach@waldowski.me <mailto:zach@waldowski.me>

On Tue, Jan 9, 2018, at 2:12 AM, Gwendal Roué via swift-evolution wrote:

Le 9 janv. 2018 à 08:06, Gwendal Roué via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

Le 9 janv. 2018 à 06:40, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

The ulterior question of whether preserving “DictionaryLiteral” is worthwhile, is apparently out of scope. Personally, I have a hard time imagining a compelling use-case outside of the standard library, and I doubt it’s being used “in the wild” (I checked several projects in the source-compatibility suite and found zero occurrences).

DictionaryLiteral is worthwhile. The SQLite library GRDB uses DictionaryLiteral in order to build database rows (which may have duplicated column names, and whose column ordering is important). This is mostly useful for tests:

    let row = try Row.fetchOne(db, "SELECT 1 AS a, 2 AS a")!
    XCTAssertEqual(row, ["a": 1, "a": 2])

Gwendal

Chris Lattner's wrote:

why is maintaining duplicate keys a feature?

Since it is immutable, why not sort the keys in the initializer, allowing an efficient binary search to look up values?

I really wish both duplicated keys and key ordering would be preserved, since both are needed for the above sample code.

Should those features be lost, the sky wouldn't fall, that's sure. But we'd have to write something much less easy to wrote and read:

XCTAssertEqual(row.map { $0 }, [("a", 1), ("a", 2)])

Gwendal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

45

Now that I think of it, this type would be great for storing results from a SQL query run on a database, for instance.

This is a valid SQL statement:

SELECT `firstname`, `lastname`, `firstname` FROM `employees`;

Note there is two copies of “firstname”. Don’t ask why. All that matters is that is valid SQL and that storing the results on a Dictionary wouldn’t be possible without manipulation.

Thus, using DictionaryLiteral (aka TableRow) would make much sense. Users could then convert to a Dictionary with the new unifyingKeys initializer, or “deserialize” into custom models that can be initialized with the TableRow (maybe via Decodable).

Given a proper name, this type could be really useful in many cases.

Thanks,
Eneko

···

On Jan 9, 2018, at 9:28 AM, Eneko Alonso via swift-evolution <swift-evolution@swift.org> wrote:

How about renaming DictionaryLiteral to Row, TabularRow or TableRow?

I think most developers are familiar with the idea that a table row contains multiple columns (in specific order), and each column has a name and a value (key/value).

Some other name suggestions:
- Record (kind of an old name for table rows)
- SortedDictionary (sorted dictionaries are missing on the standard library, and could give a chance to make this type more widely used)

Thanks,
Eneko

On Jan 9, 2018, at 9:19 AM, Nate Cook via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jan 9, 2018, at 11:00 AM, Gwendal Roué via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Le 9 janv. 2018 à 17:16, Zach Waldowski via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

I'm not sure a valid use case by a third party makes it hold its weight for inclusion in the stdlib.

You're definitely right, and that's why I wrote with the most humble tone I could.

Yet, the design of the stdlib *requires* some speculation about use cases, and speculation is *helped* by the exposition of actual uses. I'm not sure readers of the mailing list had any idea of the use cases of the current DictionaryLiteral, and maybe I helped a little.

Reproducing its feature set is extremely trivial, and would probably allow you to hint the implementation details better for your use case.

Please define "trivial”.

In case anybody would wonder, in the line below the `row` variable is of type Row which happens to adopt ExpressibleByDictionaryLiteral. It is not of type DictionaryLiteral. The use case here is the ability to express a row with a dictionary literal that accepts duplicated keys and preserves ordering:

  XCTAssertEqual(row, ["a": 1, "a": 2])

That’s great! In this case you aren’t using the DictionaryLiteral type, but a “dictionary literal”, which no one is suggesting we remove. If I’m understanding what you wrote, this is another case where the terrible name is making it super hard to discuss what we’re talking about. “Dictionary literals” and the ExpressibleByDictionaryLiteral protocol are safe!

I don't see how anything could better fit this use case than the current DictionaryLiteral. This is not *my* use case, but the use case of anyone that wants to model database rows beyond the traditional (and ill-advised) dictionary.

Some other users may come with other use cases that may also help the stdlib designers choose the best solution.

Gwendal

Zach
zach@waldowski.me <mailto:zach@waldowski.me>

On Tue, Jan 9, 2018, at 2:12 AM, Gwendal Roué via swift-evolution wrote:

Le 9 janv. 2018 à 08:06, Gwendal Roué via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

Le 9 janv. 2018 à 06:40, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

The ulterior question of whether preserving “DictionaryLiteral” is worthwhile, is apparently out of scope. Personally, I have a hard time imagining a compelling use-case outside of the standard library, and I doubt it’s being used “in the wild” (I checked several projects in the source-compatibility suite and found zero occurrences).

DictionaryLiteral is worthwhile. The SQLite library GRDB uses DictionaryLiteral in order to build database rows (which may have duplicated column names, and whose column ordering is important). This is mostly useful for tests:

    let row = try Row.fetchOne(db, "SELECT 1 AS a, 2 AS a")!
    XCTAssertEqual(row, ["a": 1, "a": 2])

Gwendal

Chris Lattner's wrote:

why is maintaining duplicate keys a feature?

Since it is immutable, why not sort the keys in the initializer, allowing an efficient binary search to look up values?

I really wish both duplicated keys and key ordering would be preserved, since both are needed for the above sample code.

Should those features be lost, the sky wouldn't fall, that's sure. But we'd have to write something much less easy to wrote and read:

XCTAssertEqual(row.map { $0 }, [("a", 1), ("a", 2)])

Gwendal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

46

I'm not sure a valid use case by a third party makes it hold its weight for inclusion in the stdlib.

You're definitely right, and that's why I wrote with the most humble tone I could.

Yet, the design of the stdlib *requires* some speculation about use cases, and speculation is *helped* by the exposition of actual uses. I'm not sure readers of the mailing list had any idea of the use cases of the current DictionaryLiteral, and maybe I helped a little.

Reproducing its feature set is extremely trivial, and would probably allow you to hint the implementation details better for your use case.

Please define "trivial”.

In case anybody would wonder, in the line below the `row` variable is of type Row which happens to adopt ExpressibleByDictionaryLiteral. It is not of type DictionaryLiteral. The use case here is the ability to express a row with a dictionary literal that accepts duplicated keys and preserves ordering:

  XCTAssertEqual(row, ["a": 1, "a": 2])

That’s great! In this case you aren’t using the DictionaryLiteral type, but a “dictionary literal”, which no one is suggesting we remove. If I’m understanding what you wrote, this is another case where the terrible name is making it super hard to discuss what we’re talking about. “Dictionary literals” and the ExpressibleByDictionaryLiteral protocol are safe!

···

On Jan 9, 2018, at 11:00 AM, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:

Le 9 janv. 2018 à 17:16, Zach Waldowski via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

I don't see how anything could better fit this use case than the current DictionaryLiteral. This is not *my* use case, but the use case of anyone that wants to model database rows beyond the traditional (and ill-advised) dictionary.

Some other users may come with other use cases that may also help the stdlib designers choose the best solution.

Gwendal

Zach
zach@waldowski.me <mailto:zach@waldowski.me>

On Tue, Jan 9, 2018, at 2:12 AM, Gwendal Roué via swift-evolution wrote:

Le 9 janv. 2018 à 08:06, Gwendal Roué via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

Le 9 janv. 2018 à 06:40, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

The ulterior question of whether preserving “DictionaryLiteral” is worthwhile, is apparently out of scope. Personally, I have a hard time imagining a compelling use-case outside of the standard library, and I doubt it’s being used “in the wild” (I checked several projects in the source-compatibility suite and found zero occurrences).

DictionaryLiteral is worthwhile. The SQLite library GRDB uses DictionaryLiteral in order to build database rows (which may have duplicated column names, and whose column ordering is important). This is mostly useful for tests:

    let row = try Row.fetchOne(db, "SELECT 1 AS a, 2 AS a")!
    XCTAssertEqual(row, ["a": 1, "a": 2])

Gwendal

Chris Lattner's wrote:

why is maintaining duplicate keys a feature?

Since it is immutable, why not sort the keys in the initializer, allowing an efficient binary search to look up values?

I really wish both duplicated keys and key ordering would be preserved, since both are needed for the above sample code.

Should those features be lost, the sky wouldn't fall, that's sure. But we'd have to write something much less easy to wrote and read:

XCTAssertEqual(row.map { $0 }, [("a", 1), ("a", 2)])

Gwendal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

1 Like
47

I'm not sure a valid use case by a third party makes it hold its weight for inclusion in the stdlib.

You're definitely right, and that's why I wrote with the most humble tone I could.

Yet, the design of the stdlib *requires* some speculation about use cases, and speculation is *helped* by the exposition of actual uses. I'm not sure readers of the mailing list had any idea of the use cases of the current DictionaryLiteral, and maybe I helped a little.

Reproducing its feature set is extremely trivial, and would probably allow you to hint the implementation details better for your use case.

Please define "trivial”.

In case anybody would wonder, in the line below the `row` variable is of type Row which happens to adopt ExpressibleByDictionaryLiteral. It is not of type DictionaryLiteral. The use case here is the ability to express a row with a dictionary literal that accepts duplicated keys and preserves ordering:

  XCTAssertEqual(row, ["a": 1, "a": 2])

That’s great! In this case you aren’t using the DictionaryLiteral type, but a “dictionary literal”, which no one is suggesting we remove.

You're right ! I was almost sure that ExpressibleByDictionaryLiteral initializer would eat a DictionaryLiteral, but it doesn't!

  extension Row : ExpressibleByDictionaryLiteral {
    init(dictionaryLiteral elements: (String, DatabaseValueConvertible?)...) { ... }
  }

This code has been written too long ago. I have been mislead, I'd like to apologize.

If I’m understanding what you wrote, this is another case where the terrible name is making it super hard to discuss what we’re talking about. “Dictionary literals” and the ExpressibleByDictionaryLiteral protocol are safe!

Yes, I have been bitten hard by the names! Time for bikeshedding indeed :-)

Gwendal

···

Le 9 janv. 2018 à 18:18, Nate Cook <natecook@apple.com> a écrit :

On Jan 9, 2018, at 11:00 AM, Gwendal Roué via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Le 9 janv. 2018 à 17:16, Zach Waldowski via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

48

Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur. I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.

Sounds good.

It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.

Right.

In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS. Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.

Right. Please keep in mind that both approaches also eliminate all the overlay dylibs for those apps, and both approaches put the vast majority of the stdlib code into the OS as well. The ‘deprecated’ dylib is probably going to be comparatively small.

With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.

Right.

- One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app. We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.

This happens with both models. Refer back to the dyld optimization WWDC talk by Nick and Louis 2-3 years ago. The big problem with Swift for startup time is that it adds half a dozen (or more) dylibs to your app bundle. In the talk they make it clear that adding a single dylib is not a big deal, it is adding a bunch of dylibs that is the problem, particularly if they have interdependencies between them.

Neither approach presents this performance problem.

Further, if it were important to solve this startup time problem, it is straight-forward to solve it for apps that do want to deploy backward (which will be almost all of them in NMOS). You’d do this by merging all the dylibs into a single one in the app bundle instead of leaving them to be independently resolved at load time.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.

Agreed.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle

In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib. That solution would have the following characteristics:

- Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.

- Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs. This would be a benefit in the code size for the OS, but not for the app itself.

The code size is small so it doesn’t matter much either way, but this actually is an advantage for the average app. Part of the point of this is that most apps don’t use this stuff, so they wouldn’t include the dylib at all. There is no cost for them in either launch time or size.

To pick on Mirrors, the one example someone came up with is an API used by the swift on server community. I haven’t heard of anyone using them in an iOS app (but of course I’m sure there is someone somewhere doing it :-)

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application. They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

The launch time comes from dyld, not code signing, but yes they would pay a very small cost. Again, I’d strongly recommend brushing up on nick + louis' WWDC talk that discusses this.

More to the point though, it is *good* that there is some (small) pressure for people to stop using APIs that are deprecated in Swift 5 and earlier. This is a one time opportunity. :-)

- Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib. This factorization can only happen once.

Correct, though Xiaodi’s point about gradually easing new APIs in with a similar approach is really really interesting and could be profound.

- There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs. It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.

Sure, but it also greatly eliminates the pressure to achieve perfection for a lot of API that simply isn’t important enough to fret that much about. This pays for itself in schedule time.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around. We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs. At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.

Right: source compatibility isn’t a forever thing, even for existing Objective-C APIs vended by apple. Giving people (e.g.) 3 years to migrate and remove them would probably be fine. The key thing is that you never break *already shipped apps*, you only make more work for people who are actively maintaining/updating their app.

With these points in mind, both scenarios are (by construction) very similar.

I disagree, because I think you’re misunderstanding the performance impact of adding exactly one dylib to an app bundle. How many dylibs does Swift 4 typically add to an app? It’s probably up to 8 or more.

1 dylib isn’t a problem, check with Nick and Louis.

I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).

I agree. The major motivation from my side isn’t the code size reduction, it is that we’ll have a better and brighter future with more clarity and less weird cruft left over - like the “DictionaryLiteral” type that sparked this whole thread. Have you looked at it? It’s crazy. :-)

-Chris

···

On Jan 12, 2018, at 4:43 PM, Ted Kremenek <kremenek@apple.com> wrote:

49

One point to bring up is that what you call libswiftCoreDeprecated.dylib
can be the place that future APIs live until they’re sufficiently mature.

I highly doubt that additions to the Swift standard library past 5.0 will
all be fully baked on arrival, and having everything carved into stone the
moment it’s added seems unwise.

Against this consideration the benefit of enabling a more stable set and a
less stable set of APIs seems to outweigh the start-up time consideration.
Yes, the implication is that most Swift apps will have to incur an
increased runtime cost.

···

On Fri, Jan 12, 2018 at 18:43 Ted Kremenek via swift-evolution < swift-evolution@swift.org> wrote:

Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a
different tactic to explain my reasoning — which may be wrong — by
explaining how I see things top down with the tradeoffs they incur. I’m
going to say a bunch of things I know *you* know, but others are on this
thread and I’ll state things for end-to-end clarity.

It seems to me that we are talking about two possible scenarios: (1) the
status quo of keeping everything in libswiftCore.dylib or (2) splitting
libswiftCore.dylib into two dylibs, one which includes some of the
“deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits
we will see with both scenarios, and see where the mismatch in our “talking
past each other” is happening.

In both cases (A) and (B), with ABI stability the Standard Library has the
option to ship in the OS. Thus applications using Swift on (say) iOS would
no longer need to include libswiftCore.dylib in the app when running on an
OS that shipped with the Standard Library.

With that in mind, here are the tradeoffs as I see between scenarios (A)
and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer
have *any* of the currently libswift*.dylib’s embedded inside the
application bundle because they would just use the one in the OS.

- One benefit of using libswift*.dylibs in the OS as opposed to those
embedded inside the app bundle is that there is a non-neglible startup time
improvement, as the code signing verification that happens when an app
launches would be faster as it would not need to verify each of these
dylibs that were previously embedded in the app. We should consider this
the new baseline for app startup time for an app running on an OS with an
ABI stable Standard Library.

- We continue to support the deprecated APIs for some time, possibly
indefinitely, even when better alternatives come around.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in
the app bundle

In the second scenario, we split out the deprecated APIs into a separate
dylib, say libswiftCoreDeprecated.dylib. That solution would have the
following characteristics:

- Like scenario (A), app bundles would not need to embed
libswiftCore.dylib when running on an OS that had an ABI-stable Standard
Library.

- Compared to scenario (A), the OS shipping the Standard Library would
have a slightly smaller libswiftCore.dylib that didn’t carry the bits for
the deprecated APIs. This would be a benefit in the code size for the OS,
but not for the app itself.

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib
(e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside
their app bundle. Compared to the new baseline I mentioned in (A), such
apps would have a launch time regression once they started using any API in
the libSwiftCoreDeprecated.dylib because code signing would need to verify
the dylib, just like it does today with the libswiftCore.dylib that is
embedded in every application. They would also be slightly larger because
the app bundle has the burden of hosting libswiftCoreDeprecated.dylib,
instead of compared to scenario (A) where the implementations of the
deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

- Because of binary compatibility concerns, after Swift 5 we would *never*
be able to “take out” any further APIs from libswiftCore.dylib and put them
in libswiftCoreDeprecated.dylib. This factorization can only happen once.

- There is some slight additional complexity added to the compiler and
build system to support this spit of the Standard Library into multiple
dylibs. It’s not a huge concern but it does exist and it is not free both
in terms of implementing and maintaining.

- We continue to support the deprecated APIs for some time, possibly
indefinitely, even when better alternatives come around. We *may* be able
to completely sunset these APIs by having a future version of the Swift
compiler simply refuse to build projects that use the deprecated (now
obsoleted) APIs. At that point, these apps need to move to using the newer
API alternatives, or not upgrade to using a new Swift compiler.

With these points in mind, both scenarios are (by construction) very
similar. Scenario (B) potentially provides a bit more flexibility in
outright sunsetting deprecated APIs down the road — but only the APIs we
deem today should be on that path. In the future when we decide to
deprecate APIs in the Standard Library we will need to follow the same kind
of deprecation-and-obsoletion path as other binary frameworks in macOS/iOS,
which may mean keeping API implementations around for a very long time just
for the purpose of binary compatibility.

If you agree with me up to this point, to me this boils down to a tradeoff
of some slightly increased opportunity to sunset some APIs in the future
for a slight reduction in size of the Standard Library today (as it ships
in the OS). Most apps that don’t use these APIs won’t care either way,
because even if we go with scenario B if they don’t use one of the
deprecated APIs they will essentially be in scenario A (and thus get the
benefit of using a Standard Library from the OS). In scenario B, those
apps that use the deprecated APIs will be slightly punished with slightly
increased app bundle size and startup time. Further, these APIs really
can’t be deprecated right now (in terms of marking them deprecated in the
module) until we have actual alternatives, so for the users that really
want to use Mirrors today (which I agree may be small) that is the main
tool they have to obtain that functionality. Such users will pay a cost —
albeit probably pretty minor — in scenario B.

I have not done the measurements of the impact in code size to the
Standard Library of removing Mirrors, but I hypothesize it is relatively
modest (I will look at verifying this hypothesis).

Do these observations align with what you are proposing, or am I missing a
critical detail?

Ted

On Jan 11, 2018, at 11:56 AM, Chris Lattner <clattner@nondot.org> wrote:

On Jan 11, 2018, at 12:01 AM, Ted Kremenek <kremenek@apple.com> wrote:

The nice thing about this is that only people who use these things would
have to pay the cost, and you can directly message this by deprecating all
the stuff in it. Think about it as an overlay for the Swift standard
library :-)

Hi Chris,

Even if we split the APIs into two sets, with an “ABI unstable” subset, we
are still going to “carry round these ’suboptimal APIs’” because clients
are using them today and those APIs — without an active plan to remove them
— *are* a part of the Standard Library. We’ve past the point in Swift
where we can just rip them out (you said so yourself in a follow up email).

Who is “we” and how do you define carry?

I think you’re saying that they will remain part of the swift project (in
the “SwiftDeprecated” module) for effectively ever. If so, I agree, and
that’s the point. The point is that the code can continue to exist and
service the few people who need it (and will become even fewer over the
years) without causing *harm* to the Swift ecosystem over the near or long
term. That’s the feature.

It feels to me that the main advantage of the overlay is a solution to
remove out pieces of the Standard Library that will be used in practice
less, and thus clients don’t “pay the cost” when they don’t use them.

Yes, exactly.

But I think we should be honest about what that “cost” actually is, as
these APIs aren’t going away right now. These suboptimal APIs will stay
around at least until better ones are created, and even then some existing
clients will want to rely on the existing suboptimal APIs anyway even if
others want to take advantage of newer, better APIs and idioms.

I’m not sure what you mean. I’m talking about two things specifically:

1) APIs that are already deprecated and have replacements. The deprecated
version goes in this library, avoid carrying them around forever.

2) APIs that are used by extremely narrow audiences, which were added
without much consideration and that have not been redesigned since Swift
1. Mirror’s are an example of this. I’m arguing that these should be
moved to this SwiftDeprecated module - but not actually deprecated until a
replacement exists.

I don’t see how this is bad for anyone, and I think this provides a
significant process improvement compared to your current approach, which
forces you to lock down APIs now without careful consideration just because
they are not “important enough”.

The main cost I see here that we save by having an “overlay for the
Standard library" is code size.

I don’t understand this point.

On macOS/iOS it is true that applications embed a copy of the Standard
Library within them and thus pay a real cost in the Standard Library’s
actually payload size in their own app. However, the only reason that is
necessary today is because we don’t have ABI stability. With ABI
stability, other options for shipping the Standard Library — such as
shipping it in the OS — become possible, and thus the burden of shouldering
that “cost” of code size shifts around. Then the tradeoffs of having an
overlay for the Standard Library are different. In a world with ABI
stability, is it better to have a Standard Library "overlay" that only some
clients use, and embed within their application bundle, than just eat that
cost in the Standard Library to be shared by potentially multiple clients
installed on a system? I think it really comes down to the numbers. In
that case, are we really just talking about Mirrors? I haven’t run the
numbers here, but my intuition tells me we are talking about a relatively
small code size impact overall.

This comment doesn’t make a lot of sense to me, perhaps we’re talking
across each other. The point of the design that I’m advocating for is that
all apps (that deploy to iOS-next or later) get a immediate improvement
from abi stability: the vast majority of the standard library is not put
into their app bundle. Further, the vastly most common apps - those that
are not using mirrors or deprecated APIs, also do not include the
SwiftDeprecated module.

Only the tiny minority of apps that do use these APIs would have a copy of
SwiftDeprecated in their app dylib, but they have a path to fix that, and
they would still see a huge improvement from the bulk of the standard
library being out of their app bundle.

There’s also other potential performance implications from doing this
split.

This doesn’t make sense to me. We’re not talking about taking comonly
used apis here. Mirrors are not performance sensitive at all, and neither
are deprecated wrappers of renamed APIs.

Being able to have the Standard Library be in the OS has potential major
implications on the startup time of Swift applications.

It is great for app developers to have an incentive to stop using
deprecated APIs! :-)

Can you elaborate a bit more on the specific “cost” factors you are using
when proposing to have a Standard Library overlay?

I think it is possible that you’re reading too much into the “overlay"
word here, so ignore that. The design I’m proposing is really simple:

1. The vast majority of the standard library goes into the OS. This
includes all the string, array, and other stuff people use.
2. The deprecated wrappers and obscure APIs go into a new module, and that
module doesn’t go into the OS. Only people that use it pay to cost.

This isn’t particularly complicated, if give you new ways to manage the
process of stdlib ABI stability, and I don’t see the downside.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

1 Like
50

One major drawback of splitting Coe library, is that despite having a stable ABI, you break the possibility to include binary frameworks inside an application unless they both link on the same version of libswiftCoreDeprecated.dylib (assuming libswiftCoreDeprecated.dylib has not a stable ABI, else there is no point having it in the first place).

···

Le 13 janv. 2018 à 01:43, Ted Kremenek via swift-evolution <swift-evolution@swift.org> a écrit :

Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur. I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.

It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.

In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS. Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.

With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.

- One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app. We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle

In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib. That solution would have the following characteristics:

- Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.

- Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs. This would be a benefit in the code size for the OS, but not for the app itself.

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application. They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

- Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib. This factorization can only happen once.

- There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs. It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around. We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs. At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.

With these points in mind, both scenarios are (by construction) very similar. Scenario (B) potentially provides a bit more flexibility in outright sunsetting deprecated APIs down the road — but only the APIs we deem today should be on that path. In the future when we decide to deprecate APIs in the Standard Library we will need to follow the same kind of deprecation-and-obsoletion path as other binary frameworks in macOS/iOS, which may mean keeping API implementations around for a very long time just for the purpose of binary compatibility.

If you agree with me up to this point, to me this boils down to a tradeoff of some slightly increased opportunity to sunset some APIs in the future for a slight reduction in size of the Standard Library today (as it ships in the OS). Most apps that don’t use these APIs won’t care either way, because even if we go with scenario B if they don’t use one of the deprecated APIs they will essentially be in scenario A (and thus get the benefit of using a Standard Library from the OS). In scenario B, those apps that use the deprecated APIs will be slightly punished with slightly increased app bundle size and startup time. Further, these APIs really can’t be deprecated right now (in terms of marking them deprecated in the module) until we have actual alternatives, so for the users that really want to use Mirrors today (which I agree may be small) that is the main tool they have to obtain that functionality. Such users will pay a cost — albeit probably pretty minor — in scenario B.

I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).

Do these observations align with what you are proposing, or am I missing a critical detail?

Ted

On Jan 11, 2018, at 11:56 AM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:

On Jan 11, 2018, at 12:01 AM, Ted Kremenek <kremenek@apple.com <mailto:kremenek@apple.com>> wrote:

The nice thing about this is that only people who use these things would have to pay the cost, and you can directly message this by deprecating all the stuff in it. Think about it as an overlay for the Swift standard library :-)

Hi Chris,

Even if we split the APIs into two sets, with an “ABI unstable” subset, we are still going to “carry round these ’suboptimal APIs’” because clients are using them today and those APIs — without an active plan to remove them — *are* a part of the Standard Library. We’ve past the point in Swift where we can just rip them out (you said so yourself in a follow up email).

Who is “we” and how do you define carry?

I think you’re saying that they will remain part of the swift project (in the “SwiftDeprecated” module) for effectively ever. If so, I agree, and that’s the point. The point is that the code can continue to exist and service the few people who need it (and will become even fewer over the years) without causing *harm* to the Swift ecosystem over the near or long term. That’s the feature.

It feels to me that the main advantage of the overlay is a solution to remove out pieces of the Standard Library that will be used in practice less, and thus clients don’t “pay the cost” when they don’t use them.

Yes, exactly.

But I think we should be honest about what that “cost” actually is, as these APIs aren’t going away right now. These suboptimal APIs will stay around at least until better ones are created, and even then some existing clients will want to rely on the existing suboptimal APIs anyway even if others want to take advantage of newer, better APIs and idioms.

I’m not sure what you mean. I’m talking about two things specifically:

1) APIs that are already deprecated and have replacements. The deprecated version goes in this library, avoid carrying them around forever.

2) APIs that are used by extremely narrow audiences, which were added without much consideration and that have not been redesigned since Swift 1. Mirror’s are an example of this. I’m arguing that these should be moved to this SwiftDeprecated module - but not actually deprecated until a replacement exists.

I don’t see how this is bad for anyone, and I think this provides a significant process improvement compared to your current approach, which forces you to lock down APIs now without careful consideration just because they are not “important enough”.

The main cost I see here that we save by having an “overlay for the Standard library" is code size.

I don’t understand this point.

On macOS/iOS it is true that applications embed a copy of the Standard Library within them and thus pay a real cost in the Standard Library’s actually payload size in their own app. However, the only reason that is necessary today is because we don’t have ABI stability. With ABI stability, other options for shipping the Standard Library — such as shipping it in the OS — become possible, and thus the burden of shouldering that “cost” of code size shifts around. Then the tradeoffs of having an overlay for the Standard Library are different. In a world with ABI stability, is it better to have a Standard Library "overlay" that only some clients use, and embed within their application bundle, than just eat that cost in the Standard Library to be shared by potentially multiple clients installed on a system? I think it really comes down to the numbers. In that case, are we really just talking about Mirrors? I haven’t run the numbers here, but my intuition tells me we are talking about a relatively small code size impact overall.

This comment doesn’t make a lot of sense to me, perhaps we’re talking across each other. The point of the design that I’m advocating for is that all apps (that deploy to iOS-next or later) get a immediate improvement from abi stability: the vast majority of the standard library is not put into their app bundle. Further, the vastly most common apps - those that are not using mirrors or deprecated APIs, also do not include the SwiftDeprecated module.

Only the tiny minority of apps that do use these APIs would have a copy of SwiftDeprecated in their app dylib, but they have a path to fix that, and they would still see a huge improvement from the bulk of the standard library being out of their app bundle.

There’s also other potential performance implications from doing this split.

This doesn’t make sense to me. We’re not talking about taking comonly used apis here. Mirrors are not performance sensitive at all, and neither are deprecated wrappers of renamed APIs.

Being able to have the Standard Library be in the OS has potential major implications on the startup time of Swift applications.

It is great for app developers to have an incentive to stop using deprecated APIs! :-)

Can you elaborate a bit more on the specific “cost” factors you are using when proposing to have a Standard Library overlay?

I think it is possible that you’re reading too much into the “overlay" word here, so ignore that. The design I’m proposing is really simple:

1. The vast majority of the standard library goes into the OS. This includes all the string, array, and other stuff people use.
2. The deprecated wrappers and obscure APIs go into a new module, and that module doesn’t go into the OS. Only people that use it pay to cost.

This isn’t particularly complicated, if give you new ways to manage the process of stdlib ABI stability, and I don’t see the downside.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

51

Pardon my lack of knowledge in this area, but is there not also a 3rd option available?

(C) Split libswiftCore.dylib into two dylibs, both at the OS level. The *Deprecated.dylib would only be included when the application binary was compiled using a special -using-deprecated flag that signifies this app needs at least one of the deprecated APIs? Then the code signing verification you mention wouldn’t be a part of the apps that use deprecated APIs.

If this is not possible because all OS-level .dylibs MUST be included for whatever reason then just disregard me, as I’m not an expert on this area of Swift.

···

On Jan 12, 2018, at 5:43 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur. I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.

It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.

In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS. Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.

With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.

- One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app. We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle

In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib. That solution would have the following characteristics:

- Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.

- Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs. This would be a benefit in the code size for the OS, but not for the app itself.

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application. They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

- Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib. This factorization can only happen once.

- There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs. It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around. We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs. At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.

With these points in mind, both scenarios are (by construction) very similar. Scenario (B) potentially provides a bit more flexibility in outright sunsetting deprecated APIs down the road — but only the APIs we deem today should be on that path. In the future when we decide to deprecate APIs in the Standard Library we will need to follow the same kind of deprecation-and-obsoletion path as other binary frameworks in macOS/iOS, which may mean keeping API implementations around for a very long time just for the purpose of binary compatibility.

If you agree with me up to this point, to me this boils down to a tradeoff of some slightly increased opportunity to sunset some APIs in the future for a slight reduction in size of the Standard Library today (as it ships in the OS). Most apps that don’t use these APIs won’t care either way, because even if we go with scenario B if they don’t use one of the deprecated APIs they will essentially be in scenario A (and thus get the benefit of using a Standard Library from the OS). In scenario B, those apps that use the deprecated APIs will be slightly punished with slightly increased app bundle size and startup time. Further, these APIs really can’t be deprecated right now (in terms of marking them deprecated in the module) until we have actual alternatives, so for the users that really want to use Mirrors today (which I agree may be small) that is the main tool they have to obtain that functionality. Such users will pay a cost — albeit probably pretty minor — in scenario B.

I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).

Do these observations align with what you are proposing, or am I missing a critical detail?

Ted

On Jan 11, 2018, at 11:56 AM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:

On Jan 11, 2018, at 12:01 AM, Ted Kremenek <kremenek@apple.com <mailto:kremenek@apple.com>> wrote:

The nice thing about this is that only people who use these things would have to pay the cost, and you can directly message this by deprecating all the stuff in it. Think about it as an overlay for the Swift standard library :-)

Hi Chris,

Even if we split the APIs into two sets, with an “ABI unstable” subset, we are still going to “carry round these ’suboptimal APIs’” because clients are using them today and those APIs — without an active plan to remove them — *are* a part of the Standard Library. We’ve past the point in Swift where we can just rip them out (you said so yourself in a follow up email).

Who is “we” and how do you define carry?

I think you’re saying that they will remain part of the swift project (in the “SwiftDeprecated” module) for effectively ever. If so, I agree, and that’s the point. The point is that the code can continue to exist and service the few people who need it (and will become even fewer over the years) without causing *harm* to the Swift ecosystem over the near or long term. That’s the feature.

It feels to me that the main advantage of the overlay is a solution to remove out pieces of the Standard Library that will be used in practice less, and thus clients don’t “pay the cost” when they don’t use them.

Yes, exactly.

But I think we should be honest about what that “cost” actually is, as these APIs aren’t going away right now. These suboptimal APIs will stay around at least until better ones are created, and even then some existing clients will want to rely on the existing suboptimal APIs anyway even if others want to take advantage of newer, better APIs and idioms.

I’m not sure what you mean. I’m talking about two things specifically:

1) APIs that are already deprecated and have replacements. The deprecated version goes in this library, avoid carrying them around forever.

2) APIs that are used by extremely narrow audiences, which were added without much consideration and that have not been redesigned since Swift 1. Mirror’s are an example of this. I’m arguing that these should be moved to this SwiftDeprecated module - but not actually deprecated until a replacement exists.

I don’t see how this is bad for anyone, and I think this provides a significant process improvement compared to your current approach, which forces you to lock down APIs now without careful consideration just because they are not “important enough”.

The main cost I see here that we save by having an “overlay for the Standard library" is code size.

I don’t understand this point.

On macOS/iOS it is true that applications embed a copy of the Standard Library within them and thus pay a real cost in the Standard Library’s actually payload size in their own app. However, the only reason that is necessary today is because we don’t have ABI stability. With ABI stability, other options for shipping the Standard Library — such as shipping it in the OS — become possible, and thus the burden of shouldering that “cost” of code size shifts around. Then the tradeoffs of having an overlay for the Standard Library are different. In a world with ABI stability, is it better to have a Standard Library "overlay" that only some clients use, and embed within their application bundle, than just eat that cost in the Standard Library to be shared by potentially multiple clients installed on a system? I think it really comes down to the numbers. In that case, are we really just talking about Mirrors? I haven’t run the numbers here, but my intuition tells me we are talking about a relatively small code size impact overall.

This comment doesn’t make a lot of sense to me, perhaps we’re talking across each other. The point of the design that I’m advocating for is that all apps (that deploy to iOS-next or later) get a immediate improvement from abi stability: the vast majority of the standard library is not put into their app bundle. Further, the vastly most common apps - those that are not using mirrors or deprecated APIs, also do not include the SwiftDeprecated module.

Only the tiny minority of apps that do use these APIs would have a copy of SwiftDeprecated in their app dylib, but they have a path to fix that, and they would still see a huge improvement from the bulk of the standard library being out of their app bundle.

There’s also other potential performance implications from doing this split.

This doesn’t make sense to me. We’re not talking about taking comonly used apis here. Mirrors are not performance sensitive at all, and neither are deprecated wrappers of renamed APIs.

Being able to have the Standard Library be in the OS has potential major implications on the startup time of Swift applications.

It is great for app developers to have an incentive to stop using deprecated APIs! :-)

Can you elaborate a bit more on the specific “cost” factors you are using when proposing to have a Standard Library overlay?

I think it is possible that you’re reading too much into the “overlay" word here, so ignore that. The design I’m proposing is really simple:

1. The vast majority of the standard library goes into the OS. This includes all the string, array, and other stuff people use.
2. The deprecated wrappers and obscure APIs go into a new module, and that module doesn’t go into the OS. Only people that use it pay to cost.

This isn’t particularly complicated, if give you new ways to manage the process of stdlib ABI stability, and I don’t see the downside.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

52

Hi Xiaodi,

It’s an interesting suggestion. My inclination is that seeding prototype APIs would be better done via source package managers, and not as binary frameworks. But they are two different approaches to the same problem and it is an interesting idea.

I’d like to first focus, however, on the utility of having such a separate dylib for the purposes of sunsetting APIs.

Ted

···

On Jan 12, 2018, at 6:02 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

One point to bring up is that what you call libswiftCoreDeprecated.dylib can be the place that future APIs live until they’re sufficiently mature.

I highly doubt that additions to the Swift standard library past 5.0 will all be fully baked on arrival, and having everything carved into stone the moment it’s added seems unwise.

Against this consideration the benefit of enabling a more stable set and a less stable set of APIs seems to outweigh the start-up time consideration. Yes, the implication is that most Swift apps will have to incur an increased runtime cost.
On Fri, Jan 12, 2018 at 18:43 Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur. I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.

It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.

In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS. Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.

With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.

- One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app. We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle

In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib. That solution would have the following characteristics:

- Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.

- Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs. This would be a benefit in the code size for the OS, but not for the app itself.

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application. They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

- Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib. This factorization can only happen once.

- There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs. It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around. We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs. At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.

With these points in mind, both scenarios are (by construction) very similar. Scenario (B) potentially provides a bit more flexibility in outright sunsetting deprecated APIs down the road — but only the APIs we deem today should be on that path. In the future when we decide to deprecate APIs in the Standard Library we will need to follow the same kind of deprecation-and-obsoletion path as other binary frameworks in macOS/iOS, which may mean keeping API implementations around for a very long time just for the purpose of binary compatibility.

If you agree with me up to this point, to me this boils down to a tradeoff of some slightly increased opportunity to sunset some APIs in the future for a slight reduction in size of the Standard Library today (as it ships in the OS). Most apps that don’t use these APIs won’t care either way, because even if we go with scenario B if they don’t use one of the deprecated APIs they will essentially be in scenario A (and thus get the benefit of using a Standard Library from the OS). In scenario B, those apps that use the deprecated APIs will be slightly punished with slightly increased app bundle size and startup time. Further, these APIs really can’t be deprecated right now (in terms of marking them deprecated in the module) until we have actual alternatives, so for the users that really want to use Mirrors today (which I agree may be small) that is the main tool they have to obtain that functionality. Such users will pay a cost — albeit probably pretty minor — in scenario B.

I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).

Do these observations align with what you are proposing, or am I missing a critical detail?

Ted

On Jan 11, 2018, at 11:56 AM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:

On Jan 11, 2018, at 12:01 AM, Ted Kremenek <kremenek@apple.com <mailto:kremenek@apple.com>> wrote:

The nice thing about this is that only people who use these things would have to pay the cost, and you can directly message this by deprecating all the stuff in it. Think about it as an overlay for the Swift standard library :-)

Hi Chris,

Even if we split the APIs into two sets, with an “ABI unstable” subset, we are still going to “carry round these ’suboptimal APIs’” because clients are using them today and those APIs — without an active plan to remove them — *are* a part of the Standard Library. We’ve past the point in Swift where we can just rip them out (you said so yourself in a follow up email).

Who is “we” and how do you define carry?

I think you’re saying that they will remain part of the swift project (in the “SwiftDeprecated” module) for effectively ever. If so, I agree, and that’s the point. The point is that the code can continue to exist and service the few people who need it (and will become even fewer over the years) without causing *harm* to the Swift ecosystem over the near or long term. That’s the feature.

It feels to me that the main advantage of the overlay is a solution to remove out pieces of the Standard Library that will be used in practice less, and thus clients don’t “pay the cost” when they don’t use them.

Yes, exactly.

But I think we should be honest about what that “cost” actually is, as these APIs aren’t going away right now. These suboptimal APIs will stay around at least until better ones are created, and even then some existing clients will want to rely on the existing suboptimal APIs anyway even if others want to take advantage of newer, better APIs and idioms.

I’m not sure what you mean. I’m talking about two things specifically:

1) APIs that are already deprecated and have replacements. The deprecated version goes in this library, avoid carrying them around forever.

2) APIs that are used by extremely narrow audiences, which were added without much consideration and that have not been redesigned since Swift 1. Mirror’s are an example of this. I’m arguing that these should be moved to this SwiftDeprecated module - but not actually deprecated until a replacement exists.

I don’t see how this is bad for anyone, and I think this provides a significant process improvement compared to your current approach, which forces you to lock down APIs now without careful consideration just because they are not “important enough”.

The main cost I see here that we save by having an “overlay for the Standard library" is code size.

I don’t understand this point.

On macOS/iOS it is true that applications embed a copy of the Standard Library within them and thus pay a real cost in the Standard Library’s actually payload size in their own app. However, the only reason that is necessary today is because we don’t have ABI stability. With ABI stability, other options for shipping the Standard Library — such as shipping it in the OS — become possible, and thus the burden of shouldering that “cost” of code size shifts around. Then the tradeoffs of having an overlay for the Standard Library are different. In a world with ABI stability, is it better to have a Standard Library "overlay" that only some clients use, and embed within their application bundle, than just eat that cost in the Standard Library to be shared by potentially multiple clients installed on a system? I think it really comes down to the numbers. In that case, are we really just talking about Mirrors? I haven’t run the numbers here, but my intuition tells me we are talking about a relatively small code size impact overall.

This comment doesn’t make a lot of sense to me, perhaps we’re talking across each other. The point of the design that I’m advocating for is that all apps (that deploy to iOS-next or later) get a immediate improvement from abi stability: the vast majority of the standard library is not put into their app bundle. Further, the vastly most common apps - those that are not using mirrors or deprecated APIs, also do not include the SwiftDeprecated module.

Only the tiny minority of apps that do use these APIs would have a copy of SwiftDeprecated in their app dylib, but they have a path to fix that, and they would still see a huge improvement from the bulk of the standard library being out of their app bundle.

There’s also other potential performance implications from doing this split.

This doesn’t make sense to me. We’re not talking about taking comonly used apis here. Mirrors are not performance sensitive at all, and neither are deprecated wrappers of renamed APIs.

Being able to have the Standard Library be in the OS has potential major implications on the startup time of Swift applications.

It is great for app developers to have an incentive to stop using deprecated APIs! :-)

Can you elaborate a bit more on the specific “cost” factors you are using when proposing to have a Standard Library overlay?

I think it is possible that you’re reading too much into the “overlay" word here, so ignore that. The design I’m proposing is really simple:

1. The vast majority of the standard library goes into the OS. This includes all the string, array, and other stuff people use.
2. The deprecated wrappers and obscure APIs go into a new module, and that module doesn’t go into the OS. Only people that use it pay to cost.

This isn’t particularly complicated, if give you new ways to manage the process of stdlib ABI stability, and I don’t see the downside.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

53

Why is that a problem? Despite referring to it as the “unstable” half, if we just put deprecated stuff in it, it would still be stable and should work for that.

You are right that things would be more complicated if it is used as a “staging ground” for features that eventually make it into core though.

-Chris

···

On Jan 13, 2018, at 4:58 AM, Jean-Daniel <mailing@xenonium.com> wrote:

One major drawback of splitting Coe library, is that despite having a stable ABI, you break the possibility to include binary frameworks inside an application unless they both link on the same version of libswiftCoreDeprecated.dylib (assuming libswiftCoreDeprecated.dylib has not a stable ABI, else there is no point having it in the first place).

Le 13 janv. 2018 à 01:43, Ted Kremenek via swift-evolution <swift-evolution@swift.org> a écrit :

Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur. I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.

It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.

In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS. Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.

With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.

- One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app. We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle

In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib. That solution would have the following characteristics:

- Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.

- Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs. This would be a benefit in the code size for the OS, but not for the app itself.

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application. They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

- Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib. This factorization can only happen once.

- There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs. It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around. We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs. At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.

With these points in mind, both scenarios are (by construction) very similar. Scenario (B) potentially provides a bit more flexibility in outright sunsetting deprecated APIs down the road — but only the APIs we deem today should be on that path. In the future when we decide to deprecate APIs in the Standard Library we will need to follow the same kind of deprecation-and-obsoletion path as other binary frameworks in macOS/iOS, which may mean keeping API implementations around for a very long time just for the purpose of binary compatibility.

If you agree with me up to this point, to me this boils down to a tradeoff of some slightly increased opportunity to sunset some APIs in the future for a slight reduction in size of the Standard Library today (as it ships in the OS). Most apps that don’t use these APIs won’t care either way, because even if we go with scenario B if they don’t use one of the deprecated APIs they will essentially be in scenario A (and thus get the benefit of using a Standard Library from the OS). In scenario B, those apps that use the deprecated APIs will be slightly punished with slightly increased app bundle size and startup time. Further, these APIs really can’t be deprecated right now (in terms of marking them deprecated in the module) until we have actual alternatives, so for the users that really want to use Mirrors today (which I agree may be small) that is the main tool they have to obtain that functionality. Such users will pay a cost — albeit probably pretty minor — in scenario B.

I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).

Do these observations align with what you are proposing, or am I missing a critical detail?

Ted

On Jan 11, 2018, at 11:56 AM, Chris Lattner <clattner@nondot.org> wrote:

On Jan 11, 2018, at 12:01 AM, Ted Kremenek <kremenek@apple.com> wrote:
The nice thing about this is that only people who use these things would have to pay the cost, and you can directly message this by deprecating all the stuff in it. Think about it as an overlay for the Swift standard library :-)

Hi Chris,

Even if we split the APIs into two sets, with an “ABI unstable” subset, we are still going to “carry round these ’suboptimal APIs’” because clients are using them today and those APIs — without an active plan to remove them — *are* a part of the Standard Library. We’ve past the point in Swift where we can just rip them out (you said so yourself in a follow up email).

Who is “we” and how do you define carry?

I think you’re saying that they will remain part of the swift project (in the “SwiftDeprecated” module) for effectively ever. If so, I agree, and that’s the point. The point is that the code can continue to exist and service the few people who need it (and will become even fewer over the years) without causing *harm* to the Swift ecosystem over the near or long term. That’s the feature.

It feels to me that the main advantage of the overlay is a solution to remove out pieces of the Standard Library that will be used in practice less, and thus clients don’t “pay the cost” when they don’t use them.

Yes, exactly.

But I think we should be honest about what that “cost” actually is, as these APIs aren’t going away right now. These suboptimal APIs will stay around at least until better ones are created, and even then some existing clients will want to rely on the existing suboptimal APIs anyway even if others want to take advantage of newer, better APIs and idioms.

I’m not sure what you mean. I’m talking about two things specifically:

1) APIs that are already deprecated and have replacements. The deprecated version goes in this library, avoid carrying them around forever.

2) APIs that are used by extremely narrow audiences, which were added without much consideration and that have not been redesigned since Swift 1. Mirror’s are an example of this. I’m arguing that these should be moved to this SwiftDeprecated module - but not actually deprecated until a replacement exists.

I don’t see how this is bad for anyone, and I think this provides a significant process improvement compared to your current approach, which forces you to lock down APIs now without careful consideration just because they are not “important enough”.

The main cost I see here that we save by having an “overlay for the Standard library" is code size.

I don’t understand this point.

On macOS/iOS it is true that applications embed a copy of the Standard Library within them and thus pay a real cost in the Standard Library’s actually payload size in their own app. However, the only reason that is necessary today is because we don’t have ABI stability. With ABI stability, other options for shipping the Standard Library — such as shipping it in the OS — become possible, and thus the burden of shouldering that “cost” of code size shifts around. Then the tradeoffs of having an overlay for the Standard Library are different. In a world with ABI stability, is it better to have a Standard Library "overlay" that only some clients use, and embed within their application bundle, than just eat that cost in the Standard Library to be shared by potentially multiple clients installed on a system? I think it really comes down to the numbers. In that case, are we really just talking about Mirrors? I haven’t run the numbers here, but my intuition tells me we are talking about a relatively small code size impact overall.

This comment doesn’t make a lot of sense to me, perhaps we’re talking across each other. The point of the design that I’m advocating for is that all apps (that deploy to iOS-next or later) get a immediate improvement from abi stability: the vast majority of the standard library is not put into their app bundle. Further, the vastly most common apps - those that are not using mirrors or deprecated APIs, also do not include the SwiftDeprecated module.

Only the tiny minority of apps that do use these APIs would have a copy of SwiftDeprecated in their app dylib, but they have a path to fix that, and they would still see a huge improvement from the bulk of the standard library being out of their app bundle.

There’s also other potential performance implications from doing this split.

This doesn’t make sense to me. We’re not talking about taking comonly used apis here. Mirrors are not performance sensitive at all, and neither are deprecated wrappers of renamed APIs.

Being able to have the Standard Library be in the OS has potential major implications on the startup time of Swift applications.

It is great for app developers to have an incentive to stop using deprecated APIs! :-)

Can you elaborate a bit more on the specific “cost” factors you are using when proposing to have a Standard Library overlay?

I think it is possible that you’re reading too much into the “overlay" word here, so ignore that. The design I’m proposing is really simple:

1. The vast majority of the standard library goes into the OS. This includes all the string, array, and other stuff people use.
2. The deprecated wrappers and obscure APIs go into a new module, and that module doesn’t go into the OS. Only people that use it pay to cost.

This isn’t particularly complicated, if give you new ways to manage the process of stdlib ABI stability, and I don’t see the downside.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

54

I agree. The major motivation from my side isn’t the code size reduction, it is that we’ll have a better and brighter future with more clarity and less weird cruft left over - like the “DictionaryLiteral” type that sparked this whole thread. Have you looked at it? It’s crazy. :-)

+1

One of the things that I like about Swift (vis-à-vis C++, Java, JavaScript etc) is that the Core team is not afraid to fix ‘historical mistakes’.
I think, Chris’ idea of a separate unstable/deprecated dylib provides an excellent platform for this practice to continue after ABI stability.

-g.

55

Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur. I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.

Sounds good.

It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.

Right.

In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS. Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.

Right. Please keep in mind that both approaches also eliminate all the overlay dylibs for those apps, and both approaches put the vast majority of the stdlib code into the OS as well. The ‘deprecated’ dylib is probably going to be comparatively small.

With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.

Right.

- One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app. We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.

This happens with both models. Refer back to the dyld optimization WWDC talk by Nick and Louis 2-3 years ago. The big problem with Swift for startup time is that it adds half a dozen (or more) dylibs to your app bundle. In the talk they make it clear that adding a single dylib is not a big deal, it is adding a bunch of dylibs that is the problem, particularly if they have interdependencies between them.

Right — it is the large volume of dylibs that is the problem.

Neither approach presents this performance problem.

Agreed — in practice neither scenario is much different.

Further, if it were important to solve this startup time problem, it is straight-forward to solve it for apps that do want to deploy backward (which will be almost all of them in NMOS). You’d do this by merging all the dylibs into a single one in the app bundle instead of leaving them to be independently resolved at load time.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.

Agreed.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle

In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib. That solution would have the following characteristics:

- Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.

- Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs. This would be a benefit in the code size for the OS, but not for the app itself.

The code size is small so it doesn’t matter much either way,

Agreed.

but this actually is an advantage for the average app. Part of the point of this is that most apps don’t use this stuff, so they wouldn’t include the dylib at all. There is no cost for them in either launch time or size.

To pick on Mirrors, the one example someone came up with is an API used by the swift on server community. I haven’t heard of anyone using them in an iOS app (but of course I’m sure there is someone somewhere doing it :-)

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application. They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

The launch time comes from dyld, not code signing, but yes they would pay a very small cost.

Yes, you’re right it is dyld. For some reason I was thinking about code signing.

Again, I’d strongly recommend brushing up on nick + louis' WWDC talk that discusses this.

More to the point though, it is *good* that there is some (small) pressure for people to stop using APIs that are deprecated in Swift 5 and earlier. This is a one time opportunity. :-)

Perhaps. It feels (to me) like a rather minor opportunity considering we need to still keep these APIs around and maintain them for some (perhaps long) period of time.

- Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib. This factorization can only happen once.

Correct, though Xiaodi’s point about gradually easing new APIs in with a similar approach is really really interesting and could be profound.

Absolutely — it’s an interesting idea, but also orthogonal to this one. I’d prefer to separate that into a different discussion as that one involves a different set of concerns and parties interested in that discussion.

- There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs. It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.

Sure, but it also greatly eliminates the pressure to achieve perfection for a lot of API that simply isn’t important enough to fret that much about. This pays for itself in schedule time.

To be honest I don’t see how it helps *that* much, and we may end up just disagreeing here on the optics. Since we have to keep these APIs around in some way, regardless of whether or not we are happy with them, I don’t see how it helps eliminate pressure. I’m not that greatly concerned about burning a small bit of space into the libswiftCore.dylib for some APIs we hope to effectively sunset in the (unforeseeable) future. OTOH, there are plenty of things with ABI stability that I am concerned get done, which is why efforts are focused on those.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around. We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs. At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.

Right: source compatibility isn’t a forever thing, even for existing Objective-C APIs vended by apple. Giving people (e.g.) 3 years to migrate and remove them would probably be fine. The key thing is that you never break *already shipped apps*, you only make more work for people who are actively maintaining/updating their app.

Agreed.

With these points in mind, both scenarios are (by construction) very similar.

I disagree, because I think you’re misunderstanding the performance impact of adding exactly one dylib to an app bundle. How many dylibs does Swift 4 typically add to an app? It’s probably up to 8 or more.

I agree that the launch time cost of a single dynamic library is small. My concern then is on the impact on people time of doing this factorization, and adding the needed logic to the build system and compiler to support it. It’s not insurmountable, but feels like a “nice to have” compared to other tasks which are clearly quite urgent for ABI stability. If we run out of urgent things to do, I can see considering doing this.

1 dylib isn’t a problem, check with Nick and Louis.

I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).

I agree. The major motivation from my side isn’t the code size reduction, it is that we’ll have a better and brighter future with more clarity and less weird cruft left over - like the “DictionaryLiteral” type that sparked this whole thread. Have you looked at it? It’s crazy. :-)

I think a bright future is still possible in either scenario. From what I can tell, if we don’t do anything, in the worst case there is some baggage left in libswiftCore.dylib for the purposes of binary compatibility with older apps. We still are left with the opportunity to deprecate those APIs in the future even if we can’t rip those symbols out now.

···

On Jan 12, 2018, at 11:23 PM, Chris Lattner <clattner@nondot.org> wrote:
On Jan 12, 2018, at 4:43 PM, Ted Kremenek <kremenek@apple.com> wrote:

56

This. I asked that already and they not understand me very well.

I see no reason for each app to have its own version of "deprecated lib".

Also we can keep the libswiftCore_4.1.dylib, libswiftCore_5.0.dylib,
libswiftCore_5.1.dylib etc on the SO too. The default target must be
libswiftCore.dylib, but if need to use a specific feature on
libswiftCore_4.1.dylib why not.

···

Em seg, 15 de jan de 2018 às 16:54, Jacob Williams via swift-evolution < swift-evolution@swift.org> escreveu:

Pardon my lack of knowledge in this area, but is there not also a 3rd
option available?

(C) Split libswiftCore.dylib into two dylibs, both at the OS level. The
*Deprecated.dylib would only be included when the application binary was
compiled using a special -using-deprecated flag that signifies this app
needs at least one of the deprecated APIs? Then the code signing
verification you mention wouldn’t be a part of the apps that use deprecated
APIs.

If this is not possible because all OS-level .dylibs MUST be included for
whatever reason then just disregard me, as I’m not an expert on this area
of Swift.

On Jan 12, 2018, at 5:43 PM, Ted Kremenek via swift-evolution < > swift-evolution@swift.org> wrote:

Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a
different tactic to explain my reasoning — which may be wrong — by
explaining how I see things top down with the tradeoffs they incur. I’m
going to say a bunch of things I know *you* know, but others are on this
thread and I’ll state things for end-to-end clarity.

It seems to me that we are talking about two possible scenarios: (1) the
status quo of keeping everything in libswiftCore.dylib or (2) splitting
libswiftCore.dylib into two dylibs, one which includes some of the
“deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits
we will see with both scenarios, and see where the mismatch in our “talking
past each other” is happening.

In both cases (A) and (B), with ABI stability the Standard Library has the
option to ship in the OS. Thus applications using Swift on (say) iOS would
no longer need to include libswiftCore.dylib in the app when running on an
OS that shipped with the Standard Library.

With that in mind, here are the tradeoffs as I see between scenarios (A)
and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer
have *any* of the currently libswift*.dylib’s embedded inside the
application bundle because they would just use the one in the OS.

- One benefit of using libswift*.dylibs in the OS as opposed to those
embedded inside the app bundle is that there is a non-neglible startup time
improvement, as the code signing verification that happens when an app
launches would be faster as it would not need to verify each of these
dylibs that were previously embedded in the app. We should consider this
the new baseline for app startup time for an app running on an OS with an
ABI stable Standard Library.

- We continue to support the deprecated APIs for some time, possibly
indefinitely, even when better alternatives come around.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in
the app bundle

In the second scenario, we split out the deprecated APIs into a separate
dylib, say libswiftCoreDeprecated.dylib. That solution would have the
following characteristics:

- Like scenario (A), app bundles would not need to embed
libswiftCore.dylib when running on an OS that had an ABI-stable Standard
Library.

- Compared to scenario (A), the OS shipping the Standard Library would
have a slightly smaller libswiftCore.dylib that didn’t carry the bits for
the deprecated APIs. This would be a benefit in the code size for the OS,
but not for the app itself.

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib
(e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside
their app bundle. Compared to the new baseline I mentioned in (A), such
apps would have a launch time regression once they started using any API in
the libSwiftCoreDeprecated.dylib because code signing would need to verify
the dylib, just like it does today with the libswiftCore.dylib that is
embedded in every application. They would also be slightly larger because
the app bundle has the burden of hosting libswiftCoreDeprecated.dylib,
instead of compared to scenario (A) where the implementations of the
deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

- Because of binary compatibility concerns, after Swift 5 we would *never*
be able to “take out” any further APIs from libswiftCore.dylib and put them
in libswiftCoreDeprecated.dylib. This factorization can only happen once.

- There is some slight additional complexity added to the compiler and
build system to support this spit of the Standard Library into multiple
dylibs. It’s not a huge concern but it does exist and it is not free both
in terms of implementing and maintaining.

- We continue to support the deprecated APIs for some time, possibly
indefinitely, even when better alternatives come around. We *may* be able
to completely sunset these APIs by having a future version of the Swift
compiler simply refuse to build projects that use the deprecated (now
obsoleted) APIs. At that point, these apps need to move to using the newer
API alternatives, or not upgrade to using a new Swift compiler.

With these points in mind, both scenarios are (by construction) very
similar. Scenario (B) potentially provides a bit more flexibility in
outright sunsetting deprecated APIs down the road — but only the APIs we
deem today should be on that path. In the future when we decide to
deprecate APIs in the Standard Library we will need to follow the same kind
of deprecation-and-obsoletion path as other binary frameworks in macOS/iOS,
which may mean keeping API implementations around for a very long time just
for the purpose of binary compatibility.

If you agree with me up to this point, to me this boils down to a tradeoff
of some slightly increased opportunity to sunset some APIs in the future
for a slight reduction in size of the Standard Library today (as it ships
in the OS). Most apps that don’t use these APIs won’t care either way,
because even if we go with scenario B if they don’t use one of the
deprecated APIs they will essentially be in scenario A (and thus get the
benefit of using a Standard Library from the OS). In scenario B, those
apps that use the deprecated APIs will be slightly punished with slightly
increased app bundle size and startup time. Further, these APIs really
can’t be deprecated right now (in terms of marking them deprecated in the
module) until we have actual alternatives, so for the users that really
want to use Mirrors today (which I agree may be small) that is the main
tool they have to obtain that functionality. Such users will pay a cost —
albeit probably pretty minor — in scenario B.

I have not done the measurements of the impact in code size to the
Standard Library of removing Mirrors, but I hypothesize it is relatively
modest (I will look at verifying this hypothesis).

Do these observations align with what you are proposing, or am I missing a
critical detail?

Ted

On Jan 11, 2018, at 11:56 AM, Chris Lattner <clattner@nondot.org> wrote:

On Jan 11, 2018, at 12:01 AM, Ted Kremenek <kremenek@apple.com> wrote:

The nice thing about this is that only people who use these things would
have to pay the cost, and you can directly message this by deprecating all
the stuff in it. Think about it as an overlay for the Swift standard
library :-)

Hi Chris,

Even if we split the APIs into two sets, with an “ABI unstable” subset, we
are still going to “carry round these ’suboptimal APIs’” because clients
are using them today and those APIs — without an active plan to remove them
— *are* a part of the Standard Library. We’ve past the point in Swift
where we can just rip them out (you said so yourself in a follow up email).

Who is “we” and how do you define carry?

I think you’re saying that they will remain part of the swift project (in
the “SwiftDeprecated” module) for effectively ever. If so, I agree, and
that’s the point. The point is that the code can continue to exist and
service the few people who need it (and will become even fewer over the
years) without causing *harm* to the Swift ecosystem over the near or long
term. That’s the feature.

It feels to me that the main advantage of the overlay is a solution to
remove out pieces of the Standard Library that will be used in practice
less, and thus clients don’t “pay the cost” when they don’t use them.

Yes, exactly.

But I think we should be honest about what that “cost” actually is, as
these APIs aren’t going away right now. These suboptimal APIs will stay
around at least until better ones are created, and even then some existing
clients will want to rely on the existing suboptimal APIs anyway even if
others want to take advantage of newer, better APIs and idioms.

I’m not sure what you mean. I’m talking about two things specifically:

1) APIs that are already deprecated and have replacements. The deprecated
version goes in this library, avoid carrying them around forever.

2) APIs that are used by extremely narrow audiences, which were added
without much consideration and that have not been redesigned since Swift
1. Mirror’s are an example of this. I’m arguing that these should be
moved to this SwiftDeprecated module - but not actually deprecated until a
replacement exists.

I don’t see how this is bad for anyone, and I think this provides a
significant process improvement compared to your current approach, which
forces you to lock down APIs now without careful consideration just because
they are not “important enough”.

The main cost I see here that we save by having an “overlay for the
Standard library" is code size.

I don’t understand this point.

On macOS/iOS it is true that applications embed a copy of the Standard
Library within them and thus pay a real cost in the Standard Library’s
actually payload size in their own app. However, the only reason that is
necessary today is because we don’t have ABI stability. With ABI
stability, other options for shipping the Standard Library — such as
shipping it in the OS — become possible, and thus the burden of shouldering
that “cost” of code size shifts around. Then the tradeoffs of having an
overlay for the Standard Library are different. In a world with ABI
stability, is it better to have a Standard Library "overlay" that only some
clients use, and embed within their application bundle, than just eat that
cost in the Standard Library to be shared by potentially multiple clients
installed on a system? I think it really comes down to the numbers. In
that case, are we really just talking about Mirrors? I haven’t run the
numbers here, but my intuition tells me we are talking about a relatively
small code size impact overall.

This comment doesn’t make a lot of sense to me, perhaps we’re talking
across each other. The point of the design that I’m advocating for is that
all apps (that deploy to iOS-next or later) get a immediate improvement
from abi stability: the vast majority of the standard library is not put
into their app bundle. Further, the vastly most common apps - those that
are not using mirrors or deprecated APIs, also do not include the
SwiftDeprecated module.

Only the tiny minority of apps that do use these APIs would have a copy of
SwiftDeprecated in their app dylib, but they have a path to fix that, and
they would still see a huge improvement from the bulk of the standard
library being out of their app bundle.

There’s also other potential performance implications from doing this
split.

This doesn’t make sense to me. We’re not talking about taking comonly
used apis here. Mirrors are not performance sensitive at all, and neither
are deprecated wrappers of renamed APIs.

Being able to have the Standard Library be in the OS has potential major
implications on the startup time of Swift applications.

It is great for app developers to have an incentive to stop using
deprecated APIs! :-)

Can you elaborate a bit more on the specific “cost” factors you are using
when proposing to have a Standard Library overlay?

I think it is possible that you’re reading too much into the “overlay"
word here, so ignore that. The design I’m proposing is really simple:

1. The vast majority of the standard library goes into the OS. This
includes all the string, array, and other stuff people use.
2. The deprecated wrappers and obscure APIs go into a new module, and that
module doesn’t go into the OS. Only people that use it pay to cost.

This isn’t particularly complicated, if give you new ways to manage the
process of stdlib ABI stability, and I don’t see the downside.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

57

Alternatively, from a different perspective, rather than adding a new
ABI-unstable library that gets bundled with apps, another way to look at it
is that we already *have* an ABI-unstable library that gets bundled with
apps. So we can instead think about introducing a new library with a stable
ABI, which can be distributed with the OS.

That way, as each piece of the existing standard library gets finalized, we
can move it into the new ABI-stable library. Some pieces (eg. Mirror) will
never make the jump and can be phased out eventually. And when future
proposals make additions to the standard library, they can begin life in
the original, existing, non-ABI-stable library.

The net effect is the same as what we have been discussing, but a shift in
viewpoint makes it clear that the ABI-stable library is the new thing we
are adding, and the existing standard library can continue to serve a
valuable purpose going forward.

Nevin

···

On Sun, Jan 14, 2018 at 9:04 PM, Ted Kremenek via swift-evolution < swift-evolution@swift.org> wrote:

> On Jan 12, 2018, at 11:23 PM, Chris Lattner <clattner@nondot.org> wrote:
>
> On Jan 12, 2018, at 4:43 PM, Ted Kremenek <kremenek@apple.com> wrote:
>> Hi Chris,
>>
>> Instead of responding to each of your point bit-by-bit, I’ll try a
different tactic to explain my reasoning — which may be wrong — by
explaining how I see things top down with the tradeoffs they incur. I’m
going to say a bunch of things I know *you* know, but others are on this
thread and I’ll state things for end-to-end clarity.
>
> Sounds good.
>
>> It seems to me that we are talking about two possible scenarios: (1)
the status quo of keeping everything in libswiftCore.dylib or (2) splitting
libswiftCore.dylib into two dylibs, one which includes some of the
“deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits
we will see with both scenarios, and see where the mismatch in our “talking
past each other” is happening.
>
> Right.
>
>> In both cases (A) and (B), with ABI stability the Standard Library has
the option to ship in the OS. Thus applications using Swift on (say) iOS
would no longer need to include libswiftCore.dylib in the app when running
on an OS that shipped with the Standard Library.
>
> Right. Please keep in mind that both approaches also eliminate all the
overlay dylibs for those apps, and both approaches put the vast majority of
the stdlib code into the OS as well. The ‘deprecated’ dylib is probably
going to be comparatively small.
>
>> With that in mind, here are the tradeoffs as I see between scenarios
(A) and (B):
>>
>> (A) Status quo: Keep shipping everything in libswiftCore.dylib
>>
>> - Applications running on an OS with the Standard Library would no
longer have *any* of the currently libswift*.dylib’s embedded inside the
application bundle because they would just use the one in the OS.
>
> Right.
>
>> - One benefit of using libswift*.dylibs in the OS as opposed to those
embedded inside the app bundle is that there is a non-neglible startup time
improvement, as the code signing verification that happens when an app
launches would be faster as it would not need to verify each of these
dylibs that were previously embedded in the app. We should consider this
the new baseline for app startup time for an app running on an OS with an
ABI stable Standard Library.
>
> This happens with both models. Refer back to the dyld optimization WWDC
talk by Nick and Louis 2-3 years ago. The big problem with Swift for
startup time is that it adds half a dozen (or more) dylibs to your app
bundle. In the talk they make it clear that adding a single dylib is not a
big deal, it is adding a bunch of dylibs that is the problem, particularly
if they have interdependencies between them.

Right — it is the large volume of dylibs that is the problem.

>
> Neither approach presents this performance problem.

Agreed — in practice neither scenario is much different.

>
> Further, if it were important to solve this startup time problem, it is
straight-forward to solve it for apps that do want to deploy backward
(which will be almost all of them in NMOS). You’d do this by merging all
the dylibs into a single one in the app bundle instead of leaving them to
be independently resolved at load time.
>
>> - We continue to support the deprecated APIs for some time, possibly
indefinitely, even when better alternatives come around.
>
> Agreed.
>
>> (B) Split libswiftCore.dylib into two dylibs, one that gets embedded in
the app bundle
>>
>> In the second scenario, we split out the deprecated APIs into a
separate dylib, say libswiftCoreDeprecated.dylib. That solution would have
the following characteristics:
>>
>> - Like scenario (A), app bundles would not need to embed
libswiftCore.dylib when running on an OS that had an ABI-stable Standard
Library.
>>
>> - Compared to scenario (A), the OS shipping the Standard Library would
have a slightly smaller libswiftCore.dylib that didn’t carry the bits for
the deprecated APIs. This would be a benefit in the code size for the OS,
but not for the app itself.
>
> The code size is small so it doesn’t matter much either way,

Agreed.

> but this actually is an advantage for the average app. Part of the
point of this is that most apps don’t use this stuff, so they wouldn’t
include the dylib at all. There is no cost for them in either launch time
or size.
>
> To pick on Mirrors, the one example someone came up with is an API used
by the swift on server community. I haven’t heard of anyone using them in
an iOS app (but of course I’m sure there is someone somewhere doing it :-)
>
>> - Any app using a deprecated API we put into
libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed
libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new
baseline I mentioned in (A), such apps would have a launch time regression
once they started using any API in the libSwiftCoreDeprecated.dylib because
code signing would need to verify the dylib, just like it does today with
the libswiftCore.dylib that is embedded in every application. They would
also be slightly larger because the app bundle has the burden of hosting
libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the
implementations of the deprecated APIs are hosted by the libswiftCore.dylib
that ships in the OS.
>
> The launch time comes from dyld, not code signing, but yes they would
pay a very small cost.

Yes, you’re right it is dyld. For some reason I was thinking about code
signing.

> Again, I’d strongly recommend brushing up on nick + louis' WWDC talk
that discusses this.
>
> More to the point though, it is *good* that there is some (small)
pressure for people to stop using APIs that are deprecated in Swift 5 and
earlier. This is a one time opportunity. :-)

Perhaps. It feels (to me) like a rather minor opportunity considering we
need to still keep these APIs around and maintain them for some (perhaps
long) period of time.

>
>> - Because of binary compatibility concerns, after Swift 5 we would
*never* be able to “take out” any further APIs from libswiftCore.dylib and
put them in libswiftCoreDeprecated.dylib. This factorization can only
happen once.
>
> Correct, though Xiaodi’s point about gradually easing new APIs in with a
similar approach is really really interesting and could be profound.

Absolutely — it’s an interesting idea, but also orthogonal to this one.
I’d prefer to separate that into a different discussion as that one
involves a different set of concerns and parties interested in that
discussion.

>
>> - There is some slight additional complexity added to the compiler and
build system to support this spit of the Standard Library into multiple
dylibs. It’s not a huge concern but it does exist and it is not free both
in terms of implementing and maintaining.
>
> Sure, but it also greatly eliminates the pressure to achieve perfection
for a lot of API that simply isn’t important enough to fret that much
about. This pays for itself in schedule time.

To be honest I don’t see how it helps *that* much, and we may end up just
disagreeing here on the optics. Since we have to keep these APIs around in
some way, regardless of whether or not we are happy with them, I don’t see
how it helps eliminate pressure. I’m not that greatly concerned about
burning a small bit of space into the libswiftCore.dylib for some APIs we
hope to effectively sunset in the (unforeseeable) future. OTOH, there are
plenty of things with ABI stability that I am concerned get done, which is
why efforts are focused on those.

>
>> - We continue to support the deprecated APIs for some time, possibly
indefinitely, even when better alternatives come around. We *may* be able
to completely sunset these APIs by having a future version of the Swift
compiler simply refuse to build projects that use the deprecated (now
obsoleted) APIs. At that point, these apps need to move to using the newer
API alternatives, or not upgrade to using a new Swift compiler.
>
> Right: source compatibility isn’t a forever thing, even for existing
Objective-C APIs vended by apple. Giving people (e.g.) 3 years to migrate
and remove them would probably be fine. The key thing is that you never
break *already shipped apps*, you only make more work for people who are
actively maintaining/updating their app.

Agreed.

>
>> With these points in mind, both scenarios are (by construction) very
similar.
>
> I disagree, because I think you’re misunderstanding the performance
impact of adding exactly one dylib to an app bundle. How many dylibs does
Swift 4 typically add to an app? It’s probably up to 8 or more.

I agree that the launch time cost of a single dynamic library is small.
My concern then is on the impact on people time of doing this
factorization, and adding the needed logic to the build system and compiler
to support it. It’s not insurmountable, but feels like a “nice to have”
compared to other tasks which are clearly quite urgent for ABI stability.
If we run out of urgent things to do, I can see considering doing this.

>
> 1 dylib isn’t a problem, check with Nick and Louis.
>
>> I have not done the measurements of the impact in code size to the
Standard Library of removing Mirrors, but I hypothesize it is relatively
modest (I will look at verifying this hypothesis).
>
> I agree. The major motivation from my side isn’t the code size
reduction, it is that we’ll have a better and brighter future with more
clarity and less weird cruft left over - like the “DictionaryLiteral” type
that sparked this whole thread. Have you looked at it? It’s crazy. :-)

I think a bright future is still possible in either scenario. From what I
can tell, if we don’t do anything, in the worst case there is some baggage
left in libswiftCore.dylib for the purposes of binary compatibility with
older apps. We still are left with the opportunity to deprecate those APIs
in the future even if we can’t rip those symbols out now.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

1 Like
58

For some OS vendors it may be feasible to have multiple Swift runtimes installed on the system to be used by different apps. That, however, is not an option if you actually want the libraries in the OS — in the case of macOS and iOS the OS frameworks — to actually use Swift themselves. At the end of the day the frameworks in the framework stack have to be using the same Swift runtime in order to have Swift APIs, pass Swift objects around that are managed by the runtime, etc.

The Swift runtime was designed to actually allow the coexistence of two Swift runtimes: one for apps built before ABI stability that include a copy of the Swift runtime for their own use, and one for ABI stability that can be used both by the OS frameworks as well as apps built with the same version of Swift (or later) that had ABI stability. In the case of runtime coexistence (which is to not break older Swift apps that aren’t rebuilt with a newer compiler) essentially the two runtimes are designed to essentially ignore each other.

···

On Jan 15, 2018, at 11:48 AM, Wallacy via swift-evolution <swift-evolution@swift.org> wrote:

This. I asked that already and they not understand me very well.

I see no reason for each app to have its own version of "deprecated lib".

Also we can keep the libswiftCore_4.1.dylib, libswiftCore_5.0.dylib, libswiftCore_5.1.dylib etc on the SO too. The default target must be libswiftCore.dylib, but if need to use a specific feature on libswiftCore_4.1.dylib why not.

Em seg, 15 de jan de 2018 às 16:54, Jacob Williams via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> escreveu:
Pardon my lack of knowledge in this area, but is there not also a 3rd option available?

(C) Split libswiftCore.dylib into two dylibs, both at the OS level. The *Deprecated.dylib would only be included when the application binary was compiled using a special -using-deprecated flag that signifies this app needs at least one of the deprecated APIs? Then the code signing verification you mention wouldn’t be a part of the apps that use deprecated APIs.

If this is not possible because all OS-level .dylibs MUST be included for whatever reason then just disregard me, as I’m not an expert on this area of Swift.

On Jan 12, 2018, at 5:43 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi Chris,

Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur. I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.

It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.

In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS. Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.

With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):

(A) Status quo: Keep shipping everything in libswiftCore.dylib

- Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.

- One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app. We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.

(B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle

In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib. That solution would have the following characteristics:

- Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.

- Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs. This would be a benefit in the code size for the OS, but not for the app itself.

- Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application. They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.

- Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib. This factorization can only happen once.

- There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs. It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.

- We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around. We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs. At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.

With these points in mind, both scenarios are (by construction) very similar. Scenario (B) potentially provides a bit more flexibility in outright sunsetting deprecated APIs down the road — but only the APIs we deem today should be on that path. In the future when we decide to deprecate APIs in the Standard Library we will need to follow the same kind of deprecation-and-obsoletion path as other binary frameworks in macOS/iOS, which may mean keeping API implementations around for a very long time just for the purpose of binary compatibility.

If you agree with me up to this point, to me this boils down to a tradeoff of some slightly increased opportunity to sunset some APIs in the future for a slight reduction in size of the Standard Library today (as it ships in the OS). Most apps that don’t use these APIs won’t care either way, because even if we go with scenario B if they don’t use one of the deprecated APIs they will essentially be in scenario A (and thus get the benefit of using a Standard Library from the OS). In scenario B, those apps that use the deprecated APIs will be slightly punished with slightly increased app bundle size and startup time. Further, these APIs really can’t be deprecated right now (in terms of marking them deprecated in the module) until we have actual alternatives, so for the users that really want to use Mirrors today (which I agree may be small) that is the main tool they have to obtain that functionality. Such users will pay a cost — albeit probably pretty minor — in scenario B.

I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).

Do these observations align with what you are proposing, or am I missing a critical detail?

Ted

On Jan 11, 2018, at 11:56 AM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:

On Jan 11, 2018, at 12:01 AM, Ted Kremenek <kremenek@apple.com <mailto:kremenek@apple.com>> wrote:

The nice thing about this is that only people who use these things would have to pay the cost, and you can directly message this by deprecating all the stuff in it. Think about it as an overlay for the Swift standard library :-)

Hi Chris,

Even if we split the APIs into two sets, with an “ABI unstable” subset, we are still going to “carry round these ’suboptimal APIs’” because clients are using them today and those APIs — without an active plan to remove them — *are* a part of the Standard Library. We’ve past the point in Swift where we can just rip them out (you said so yourself in a follow up email).

Who is “we” and how do you define carry?

I think you’re saying that they will remain part of the swift project (in the “SwiftDeprecated” module) for effectively ever. If so, I agree, and that’s the point. The point is that the code can continue to exist and service the few people who need it (and will become even fewer over the years) without causing *harm* to the Swift ecosystem over the near or long term. That’s the feature.

It feels to me that the main advantage of the overlay is a solution to remove out pieces of the Standard Library that will be used in practice less, and thus clients don’t “pay the cost” when they don’t use them.

Yes, exactly.

But I think we should be honest about what that “cost” actually is, as these APIs aren’t going away right now. These suboptimal APIs will stay around at least until better ones are created, and even then some existing clients will want to rely on the existing suboptimal APIs anyway even if others want to take advantage of newer, better APIs and idioms.

I’m not sure what you mean. I’m talking about two things specifically:

1) APIs that are already deprecated and have replacements. The deprecated version goes in this library, avoid carrying them around forever.

2) APIs that are used by extremely narrow audiences, which were added without much consideration and that have not been redesigned since Swift 1. Mirror’s are an example of this. I’m arguing that these should be moved to this SwiftDeprecated module - but not actually deprecated until a replacement exists.

I don’t see how this is bad for anyone, and I think this provides a significant process improvement compared to your current approach, which forces you to lock down APIs now without careful consideration just because they are not “important enough”.

The main cost I see here that we save by having an “overlay for the Standard library" is code size.

I don’t understand this point.

On macOS/iOS it is true that applications embed a copy of the Standard Library within them and thus pay a real cost in the Standard Library’s actually payload size in their own app. However, the only reason that is necessary today is because we don’t have ABI stability. With ABI stability, other options for shipping the Standard Library — such as shipping it in the OS — become possible, and thus the burden of shouldering that “cost” of code size shifts around. Then the tradeoffs of having an overlay for the Standard Library are different. In a world with ABI stability, is it better to have a Standard Library "overlay" that only some clients use, and embed within their application bundle, than just eat that cost in the Standard Library to be shared by potentially multiple clients installed on a system? I think it really comes down to the numbers. In that case, are we really just talking about Mirrors? I haven’t run the numbers here, but my intuition tells me we are talking about a relatively small code size impact overall.

This comment doesn’t make a lot of sense to me, perhaps we’re talking across each other. The point of the design that I’m advocating for is that all apps (that deploy to iOS-next or later) get a immediate improvement from abi stability: the vast majority of the standard library is not put into their app bundle. Further, the vastly most common apps - those that are not using mirrors or deprecated APIs, also do not include the SwiftDeprecated module.

Only the tiny minority of apps that do use these APIs would have a copy of SwiftDeprecated in their app dylib, but they have a path to fix that, and they would still see a huge improvement from the bulk of the standard library being out of their app bundle.

There’s also other potential performance implications from doing this split.

This doesn’t make sense to me. We’re not talking about taking comonly used apis here. Mirrors are not performance sensitive at all, and neither are deprecated wrappers of renamed APIs.

Being able to have the Standard Library be in the OS has potential major implications on the startup time of Swift applications.

It is great for app developers to have an incentive to stop using deprecated APIs! :-)

Can you elaborate a bit more on the specific “cost” factors you are using when proposing to have a Standard Library overlay?

I think it is possible that you’re reading too much into the “overlay" word here, so ignore that. The design I’m proposing is really simple:

1. The vast majority of the standard library goes into the OS. This includes all the string, array, and other stuff people use.
2. The deprecated wrappers and obscure APIs go into a new module, and that module doesn’t go into the OS. Only people that use it pay to cost.

This isn’t particularly complicated, if give you new ways to manage the process of stdlib ABI stability, and I don’t see the downside.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

59

Hi Nevin,

I think this is an interesting perspective. For me it begets the question of how an API “gets finalized”. Currently anything that goes through the Swift Evolution proposal process that gets ratified is considered an official change to the language and Standard Library. That feels like “finalized” to me, unless we want to introduce some new affordances to the process that allow us to introduce new APIs that are clearly going through a trial. I think that’s a very interested idea (which is one Xiaodi proposed), but that’s not something we have today.

On this thread we are talking about APIs that have been in the Standard Library for a while, and largely predate the Swift Evolution process. What makes them less finalized versus other APIs with similar heritage? These APIs are in use, albeit Chris points out that they are (most likely) uncommonly used. A core deliverable of ABI stability, however, is that the majority of the Standard Library is in a binary stable form. The only APIs I think we can consider candidates for a “deprecated” dylib that gets embedded in the app are “leaf” APIs that are uncommonly used by clients and not relied upon by other parts of the Standard Library that are commonly used by clients. Practically speaking, however, I’m not convinced that set is so large to add the complexity/engineering cost/time of creating this separate dylib right now. Those APIs can’t get ripped out right now — after all they’ve been in there for a while and we’re not deprecating them because there aren’t alternatives that exist yet anyway. Thus these APIs need to continue to exist, and the benefits to me to create this separate dylib for those old “warty” APIs that we may — one day — want to replace seems marginal.

I do think the idea of having a way to add *new* ABI-unstable APIs is very interesting, but I think the root of this discussion comes from talking about APIs that have been around for a while but feel like potential warts as we approach ABI stability. I hate to say it, but there will be warts regardless. There’s a lot to do for ABI stability — it’s making good progress — but not every last change we may have wanted to do will get all the attention we may have originally hoped it would. Getting ABI stability done for Swift 5 is about prioritization and best effort, which includes accepting that not everything is going to be perfect.

Ted

···

On Jan 15, 2018, at 1:04 PM, Nevin Brackett-Rozinsky <nevin.brackettrozinsky@gmail.com> wrote:

Alternatively, from a different perspective, rather than adding a new ABI-unstable library that gets bundled with apps, another way to look at it is that we already *have* an ABI-unstable library that gets bundled with apps. So we can instead think about introducing a new library with a stable ABI, which can be distributed with the OS.

That way, as each piece of the existing standard library gets finalized, we can move it into the new ABI-stable library. Some pieces (eg. Mirror) will never make the jump and can be phased out eventually. And when future proposals make additions to the standard library, they can begin life in the original, existing, non-ABI-stable library.

The net effect is the same as what we have been discussing, but a shift in viewpoint makes it clear that the ABI-stable library is the new thing we are adding, and the existing standard library can continue to serve a valuable purpose going forward.

Nevin

On Sun, Jan 14, 2018 at 9:04 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> On Jan 12, 2018, at 11:23 PM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:
>
> On Jan 12, 2018, at 4:43 PM, Ted Kremenek <kremenek@apple.com <mailto:kremenek@apple.com>> wrote:
>> Hi Chris,
>>
>> Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur. I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.
>
> Sounds good.
>
>> It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.
>
> Right.
>
>> In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS. Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.
>
> Right. Please keep in mind that both approaches also eliminate all the overlay dylibs for those apps, and both approaches put the vast majority of the stdlib code into the OS as well. The ‘deprecated’ dylib is probably going to be comparatively small.
>
>> With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):
>>
>> (A) Status quo: Keep shipping everything in libswiftCore.dylib
>>
>> - Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.
>
> Right.
>
>> - One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app. We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.
>
> This happens with both models. Refer back to the dyld optimization WWDC talk by Nick and Louis 2-3 years ago. The big problem with Swift for startup time is that it adds half a dozen (or more) dylibs to your app bundle. In the talk they make it clear that adding a single dylib is not a big deal, it is adding a bunch of dylibs that is the problem, particularly if they have interdependencies between them.

Right — it is the large volume of dylibs that is the problem.

>
> Neither approach presents this performance problem.

Agreed — in practice neither scenario is much different.

>
> Further, if it were important to solve this startup time problem, it is straight-forward to solve it for apps that do want to deploy backward (which will be almost all of them in NMOS). You’d do this by merging all the dylibs into a single one in the app bundle instead of leaving them to be independently resolved at load time.
>
>> - We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.
>
> Agreed.
>
>> (B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle
>>
>> In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib. That solution would have the following characteristics:
>>
>> - Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.
>>
>> - Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs. This would be a benefit in the code size for the OS, but not for the app itself.
>
> The code size is small so it doesn’t matter much either way,

Agreed.

> but this actually is an advantage for the average app. Part of the point of this is that most apps don’t use this stuff, so they wouldn’t include the dylib at all. There is no cost for them in either launch time or size.
>
> To pick on Mirrors, the one example someone came up with is an API used by the swift on server community. I haven’t heard of anyone using them in an iOS app (but of course I’m sure there is someone somewhere doing it :-)
>
>> - Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application. They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.
>
> The launch time comes from dyld, not code signing, but yes they would pay a very small cost.

Yes, you’re right it is dyld. For some reason I was thinking about code signing.

> Again, I’d strongly recommend brushing up on nick + louis' WWDC talk that discusses this.
>
> More to the point though, it is *good* that there is some (small) pressure for people to stop using APIs that are deprecated in Swift 5 and earlier. This is a one time opportunity. :-)

Perhaps. It feels (to me) like a rather minor opportunity considering we need to still keep these APIs around and maintain them for some (perhaps long) period of time.

>
>> - Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib. This factorization can only happen once.
>
> Correct, though Xiaodi’s point about gradually easing new APIs in with a similar approach is really really interesting and could be profound.

Absolutely — it’s an interesting idea, but also orthogonal to this one. I’d prefer to separate that into a different discussion as that one involves a different set of concerns and parties interested in that discussion.

>
>> - There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs. It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.
>
> Sure, but it also greatly eliminates the pressure to achieve perfection for a lot of API that simply isn’t important enough to fret that much about. This pays for itself in schedule time.

To be honest I don’t see how it helps *that* much, and we may end up just disagreeing here on the optics. Since we have to keep these APIs around in some way, regardless of whether or not we are happy with them, I don’t see how it helps eliminate pressure. I’m not that greatly concerned about burning a small bit of space into the libswiftCore.dylib for some APIs we hope to effectively sunset in the (unforeseeable) future. OTOH, there are plenty of things with ABI stability that I am concerned get done, which is why efforts are focused on those.

>
>> - We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around. We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs. At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.
>
> Right: source compatibility isn’t a forever thing, even for existing Objective-C APIs vended by apple. Giving people (e.g.) 3 years to migrate and remove them would probably be fine. The key thing is that you never break *already shipped apps*, you only make more work for people who are actively maintaining/updating their app.

Agreed.

>
>> With these points in mind, both scenarios are (by construction) very similar.
>
> I disagree, because I think you’re misunderstanding the performance impact of adding exactly one dylib to an app bundle. How many dylibs does Swift 4 typically add to an app? It’s probably up to 8 or more.

I agree that the launch time cost of a single dynamic library is small. My concern then is on the impact on people time of doing this factorization, and adding the needed logic to the build system and compiler to support it. It’s not insurmountable, but feels like a “nice to have” compared to other tasks which are clearly quite urgent for ABI stability. If we run out of urgent things to do, I can see considering doing this.

>
> 1 dylib isn’t a problem, check with Nick and Louis.
>
>> I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).
>
> I agree. The major motivation from my side isn’t the code size reduction, it is that we’ll have a better and brighter future with more clarity and less weird cruft left over - like the “DictionaryLiteral” type that sparked this whole thread. Have you looked at it? It’s crazy. :-)

I think a bright future is still possible in either scenario. From what I can tell, if we don’t do anything, in the worst case there is some baggage left in libswiftCore.dylib for the purposes of binary compatibility with older apps. We still are left with the opportunity to deprecate those APIs in the future even if we can’t rip those symbols out now.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

60

I should have been more clear: APIs that we add through Swift Evolution
would continue to be permanent additions to the language just as they are
today. I was not suggesting otherwise. However, the underlying
behind-the-scenes ABI-related aspects of their implementations would
initially be malleable, so the optimization wizards on Swift Dev could have
a chance to do their thing.

For example, the String ABI thread on Swift Dev has discussion about
various aspects of the String type’s implementation, which do not affect
API but do need to be settled before ABI stability. Similar decisions may
need to be made about other types, and this idea of Xiaodi’s and Chris’s
would keep that option open.

Nevin

···

On Tue, Jan 16, 2018 at 1:39 AM, Ted Kremenek <kremenek@apple.com> wrote:

Hi Nevin,

I think this is an interesting perspective. For me it begets the question
of how an API “gets finalized”. Currently anything that goes through the
Swift Evolution proposal process that gets ratified is considered an
official change to the language and Standard Library. That feels like
“finalized” to me, unless we want to introduce some new affordances to the
process that allow us to introduce new APIs that are clearly going through
a trial. I think that’s a very interested idea (which is one Xiaodi
proposed), but that’s not something we have today.

On this thread we are talking about APIs that have been in the Standard
Library for a while, and largely predate the Swift Evolution process. What
makes them less finalized versus other APIs with similar heritage? These
APIs are in use, albeit Chris points out that they are (most likely)
uncommonly used. A core deliverable of ABI stability, however, is that the
majority of the Standard Library is in a binary stable form. The only APIs
I think we can consider candidates for a “deprecated” dylib that gets
embedded in the app are “leaf” APIs that are uncommonly used by clients and
not relied upon by other parts of the Standard Library that are commonly
used by clients. Practically speaking, however, I’m not convinced that set
is so large to add the complexity/engineering cost/time of creating this
separate dylib right now. Those APIs can’t get ripped out right now —
after all they’ve been in there for a while and we’re not deprecating them
because there aren’t alternatives that exist yet anyway. Thus these APIs
need to continue to exist, and the benefits to me to create this separate
dylib for those old “warty” APIs that we may — one day — want to replace
seems marginal.

I do think the idea of having a way to add *new* ABI-unstable APIs is very
interesting, but I think the root of this discussion comes from talking
about APIs that have been around for a while but feel like potential warts
as we approach ABI stability. I hate to say it, but there will be warts
regardless. There’s a lot to do for ABI stability — it’s making good
progress — but not every last change we may have wanted to do will get all
the attention we may have originally hoped it would. Getting ABI stability
done for Swift 5 is about prioritization and best effort, which includes
accepting that not everything is going to be perfect.

Ted

On Jan 15, 2018, at 1:04 PM, Nevin Brackett-Rozinsky < > nevin.brackettrozinsky@gmail.com> wrote:

Alternatively, from a different perspective, rather than adding a new
ABI-unstable library that gets bundled with apps, another way to look at it
is that we already *have* an ABI-unstable library that gets bundled with
apps. So we can instead think about introducing a new library with a stable
ABI, which can be distributed with the OS.

That way, as each piece of the existing standard library gets finalized,
we can move it into the new ABI-stable library. Some pieces (eg. Mirror)
will never make the jump and can be phased out eventually. And when future
proposals make additions to the standard library, they can begin life in
the original, existing, non-ABI-stable library.

The net effect is the same as what we have been discussing, but a shift in
viewpoint makes it clear that the ABI-stable library is the new thing we
are adding, and the existing standard library can continue to serve a
valuable purpose going forward.

Nevin

On Sun, Jan 14, 2018 at 9:04 PM, Ted Kremenek via swift-evolution < > swift-evolution@swift.org> wrote:

> On Jan 12, 2018, at 11:23 PM, Chris Lattner <clattner@nondot.org> >> wrote:
>
> On Jan 12, 2018, at 4:43 PM, Ted Kremenek <kremenek@apple.com> wrote:
>> Hi Chris,
>>
>> Instead of responding to each of your point bit-by-bit, I’ll try a
different tactic to explain my reasoning — which may be wrong — by
explaining how I see things top down with the tradeoffs they incur. I’m
going to say a bunch of things I know *you* know, but others are on this
thread and I’ll state things for end-to-end clarity.
>
> Sounds good.
>
>> It seems to me that we are talking about two possible scenarios: (1)
the status quo of keeping everything in libswiftCore.dylib or (2) splitting
libswiftCore.dylib into two dylibs, one which includes some of the
“deprecated” APIs. I’ll enumerate what I think are the tradeoffs/benefits
we will see with both scenarios, and see where the mismatch in our “talking
past each other” is happening.
>
> Right.
>
>> In both cases (A) and (B), with ABI stability the Standard Library has
the option to ship in the OS. Thus applications using Swift on (say) iOS
would no longer need to include libswiftCore.dylib in the app when running
on an OS that shipped with the Standard Library.
>
> Right. Please keep in mind that both approaches also eliminate all the
overlay dylibs for those apps, and both approaches put the vast majority of
the stdlib code into the OS as well. The ‘deprecated’ dylib is probably
going to be comparatively small.
>
>> With that in mind, here are the tradeoffs as I see between scenarios
(A) and (B):
>>
>> (A) Status quo: Keep shipping everything in libswiftCore.dylib
>>
>> - Applications running on an OS with the Standard Library would no
longer have *any* of the currently libswift*.dylib’s embedded inside the
application bundle because they would just use the one in the OS.
>
> Right.
>
>> - One benefit of using libswift*.dylibs in the OS as opposed to those
embedded inside the app bundle is that there is a non-neglible startup time
improvement, as the code signing verification that happens when an app
launches would be faster as it would not need to verify each of these
dylibs that were previously embedded in the app. We should consider this
the new baseline for app startup time for an app running on an OS with an
ABI stable Standard Library.
>
> This happens with both models. Refer back to the dyld optimization
WWDC talk by Nick and Louis 2-3 years ago. The big problem with Swift for
startup time is that it adds half a dozen (or more) dylibs to your app
bundle. In the talk they make it clear that adding a single dylib is not a
big deal, it is adding a bunch of dylibs that is the problem, particularly
if they have interdependencies between them.

Right — it is the large volume of dylibs that is the problem.

>
> Neither approach presents this performance problem.

Agreed — in practice neither scenario is much different.

>
> Further, if it were important to solve this startup time problem, it is
straight-forward to solve it for apps that do want to deploy backward
(which will be almost all of them in NMOS). You’d do this by merging all
the dylibs into a single one in the app bundle instead of leaving them to
be independently resolved at load time.
>
>> - We continue to support the deprecated APIs for some time, possibly
indefinitely, even when better alternatives come around.
>
> Agreed.
>
>> (B) Split libswiftCore.dylib into two dylibs, one that gets embedded
in the app bundle
>>
>> In the second scenario, we split out the deprecated APIs into a
separate dylib, say libswiftCoreDeprecated.dylib. That solution would have
the following characteristics:
>>
>> - Like scenario (A), app bundles would not need to embed
libswiftCore.dylib when running on an OS that had an ABI-stable Standard
Library.
>>
>> - Compared to scenario (A), the OS shipping the Standard Library would
have a slightly smaller libswiftCore.dylib that didn’t carry the bits for
the deprecated APIs. This would be a benefit in the code size for the OS,
but not for the app itself.
>
> The code size is small so it doesn’t matter much either way,

Agreed.

> but this actually is an advantage for the average app. Part of the
point of this is that most apps don’t use this stuff, so they wouldn’t
include the dylib at all. There is no cost for them in either launch time
or size.
>
> To pick on Mirrors, the one example someone came up with is an API used
by the swift on server community. I haven’t heard of anyone using them in
an iOS app (but of course I’m sure there is someone somewhere doing it :-)
>
>> - Any app using a deprecated API we put into
libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed
libswiftCoreDeprecated.dylib inside their app bundle. Compared to the new
baseline I mentioned in (A), such apps would have a launch time regression
once they started using any API in the libSwiftCoreDeprecated.dylib because
code signing would need to verify the dylib, just like it does today with
the libswiftCore.dylib that is embedded in every application. They would
also be slightly larger because the app bundle has the burden of hosting
libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the
implementations of the deprecated APIs are hosted by the libswiftCore.dylib
that ships in the OS.
>
> The launch time comes from dyld, not code signing, but yes they would
pay a very small cost.

Yes, you’re right it is dyld. For some reason I was thinking about code
signing.

> Again, I’d strongly recommend brushing up on nick + louis' WWDC talk
that discusses this.
>
> More to the point though, it is *good* that there is some (small)
pressure for people to stop using APIs that are deprecated in Swift 5 and
earlier. This is a one time opportunity. :-)

Perhaps. It feels (to me) like a rather minor opportunity considering we
need to still keep these APIs around and maintain them for some (perhaps
long) period of time.

>
>> - Because of binary compatibility concerns, after Swift 5 we would
*never* be able to “take out” any further APIs from libswiftCore.dylib and
put them in libswiftCoreDeprecated.dylib. This factorization can only
happen once.
>
> Correct, though Xiaodi’s point about gradually easing new APIs in with
a similar approach is really really interesting and could be profound.

Absolutely — it’s an interesting idea, but also orthogonal to this one.
I’d prefer to separate that into a different discussion as that one
involves a different set of concerns and parties interested in that
discussion.

>
>> - There is some slight additional complexity added to the compiler and
build system to support this spit of the Standard Library into multiple
dylibs. It’s not a huge concern but it does exist and it is not free both
in terms of implementing and maintaining.
>
> Sure, but it also greatly eliminates the pressure to achieve perfection
for a lot of API that simply isn’t important enough to fret that much
about. This pays for itself in schedule time.

To be honest I don’t see how it helps *that* much, and we may end up just
disagreeing here on the optics. Since we have to keep these APIs around in
some way, regardless of whether or not we are happy with them, I don’t see
how it helps eliminate pressure. I’m not that greatly concerned about
burning a small bit of space into the libswiftCore.dylib for some APIs we
hope to effectively sunset in the (unforeseeable) future. OTOH, there are
plenty of things with ABI stability that I am concerned get done, which is
why efforts are focused on those.

>
>> - We continue to support the deprecated APIs for some time, possibly
indefinitely, even when better alternatives come around. We *may* be able
to completely sunset these APIs by having a future version of the Swift
compiler simply refuse to build projects that use the deprecated (now
obsoleted) APIs. At that point, these apps need to move to using the newer
API alternatives, or not upgrade to using a new Swift compiler.
>
> Right: source compatibility isn’t a forever thing, even for existing
Objective-C APIs vended by apple. Giving people (e.g.) 3 years to migrate
and remove them would probably be fine. The key thing is that you never
break *already shipped apps*, you only make more work for people who are
actively maintaining/updating their app.

Agreed.

>
>> With these points in mind, both scenarios are (by construction) very
similar.
>
> I disagree, because I think you’re misunderstanding the performance
impact of adding exactly one dylib to an app bundle. How many dylibs does
Swift 4 typically add to an app? It’s probably up to 8 or more.

I agree that the launch time cost of a single dynamic library is small.
My concern then is on the impact on people time of doing this
factorization, and adding the needed logic to the build system and compiler
to support it. It’s not insurmountable, but feels like a “nice to have”
compared to other tasks which are clearly quite urgent for ABI stability.
If we run out of urgent things to do, I can see considering doing this.

>
> 1 dylib isn’t a problem, check with Nick and Louis.
>
>> I have not done the measurements of the impact in code size to the
Standard Library of removing Mirrors, but I hypothesize it is relatively
modest (I will look at verifying this hypothesis).
>
> I agree. The major motivation from my side isn’t the code size
reduction, it is that we’ll have a better and brighter future with more
clarity and less weird cruft left over - like the “DictionaryLiteral” type
that sparked this whole thread. Have you looked at it? It’s crazy. :-)

I think a bright future is still possible in either scenario. From what
I can tell, if we don’t do anything, in the worst case there is some
baggage left in libswiftCore.dylib for the purposes of binary compatibility
with older apps. We still are left with the opportunity to deprecate those
APIs in the future even if we can’t rip those symbols out now.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution