Possibility to enable `any` in Embedded Swift

MikeIsaev · 2024-06-17T10:38:03.988Z

Is it possible to optionally enable any and other features in Embedded Swift? I'm using it with WebAssembly for web development, and in this case, I completely understand and appreciate that String and its related functionalities have been disabled to minimize the compiled binary size. Having adapted JavaScriptKit to work in Embedded one, my 'hello world' app is 140Kb uncompressed, and 48Kb when gzipped. While these results are impressive, I find myself really missing the any type, which is essential for fully functional protocols, Codable, and result builders. Without it, Swift feels very limited. I wouldn't mind if the 'hello world' binary were about 1Mb but included working any.

What is the overhead (in percentage) that the any type adds to the final binary? If someone could guide me, I'm ready to work on enabling any and related things.

photovirus · 2024-06-17T22:36:47.033Z

I'd be happy to see any as a flag. Embedded platforms are surely demanding, but not any platform is resource-constrained to that extent, and web seems to be the place where binary size matters, and memory consumption isn't as important.

aetherealtech · 2024-06-17T23:19:33.930Z

This is not a suggestion that existentials should never be supported in embedded Swift, but you might want to take some of the examples you've encountered where you reach for any and post a question on the forum asking how to un-existentialize it.

You might be surprised just how unnecessary existentials really are. Unless something's type is genuinely runtime variable (like the type of something is affected by user input) and open-ended (rather than from a compile time known list of possible types), the type information can be preserved (whether it should is a different issue).

Usually this only happens in library code that supports clients supplying their own subclasses or conforming types and it needs to store them in a container like an array. Although I think even then some kind of recursive enum (basically a linked list) could un-existentialize that. The only thing I can think of off the top of my head that can't be un-existentialized is reading a type name as a string from dynamic input and turning it into a type through reflection.

Un-existentializing usually involves gratuitous use of generics, to a degree that can feel excessive, "heavy" or non-scalable if you're not used to it. SwiftUI is a good example of what this looks like. After a while I got used to it and now prefer it as a coding style. It's also taught me a ton about the language.

rauhul · 2024-06-18T00:14:52.308Z

A form of any is almost certainly something we want for Embedded Swift because it enables dynamic dispatch (without classes). However, the current implementation of any requires Reflection metadata which we cannot bring to Embedded Swift.

I personally have been waiting for more of the ownership features to land before tackling this limitation.

wadetregaskis · 2024-06-18T00:58:06.870Z

aetherealtech:

Usually this only happens in library code that supports clients supplying their own subclasses or conforming types and it needs to store them in a container like an array. Although I think even then some kind of recursive enum (basically a linked list) could un-existentialize that.

The practical downside there is the labour involved in manually creating, using, and maintaining that type union.

If, instead, that functionality were part of the language (so-called "sum types", so you could e.g. typealias Element = Foo | Bar) it might greatly reduce the need for true existentials (in general, not just for "Embedded" Swift).

Protocols are an existing but more limited tool for avoiding existentials, as well, only for the subset of cases where you care only about common functionality, not unique functionality. They can also be a bit heavy to apply (and the recent clamp-down on "retroactive" conformance further limits their applicability).

aetherealtech · 2024-06-18T12:05:59.515Z

I have some thoughts on that too but don't want to hijack this thread. I'll start another one.

let_zeppelin · 2024-06-18T13:26:14.301Z

If the types that are implementing the protocol are known, maybe you can help yourself with a macro that basically looks like this:

@Implemented(by: Red.self, Green.self, Blue.self)
protocol Color {
  var rgb: (Int, Int, Int) { get }
}

// `Implemented` macro expands to:
enum AnyColor: Color {
  case red(Red)
  case green(Green)
  case blue(Blue)

  var rgb: (Int, Int, Int) {
    switch self {
    case .red(let red):
      red.rgb
    case .green(let green):
      green.rgb
    case .blue(let blue):
      blue.rgb
    }
  }
}

Max_Desiatov · 2024-06-18T13:49:14.411Z

+1 to what was suggested above by multiple people. Developers are too frequently reaching for existentials. This is probably coming from attempts to follow patterns coming from other languages like Objective-C, where existentials is the only way do polymorphism. One may not realize the massive overhead to performance and binary size that existentials bring to the code.

In the vast majority of cases generics (esp. with some opaque types) and enums (augmented by macros) would do just fine. 1 MB binary may be ok for a "Hello, World!" demo, but when using a real world app your users will be very grateful for small binaries and better performance, which of course benefits the battery life of user's devices.

Another approach that could work in the embedded setting is explicit protocol witnesses. As any pattern, it should be used with caution, but it does have its own merits especially as it doesn't introduce the retroactive conformance problem into your code that so many protocol-heavy libraries tend to have.

Slava_Pestov · 2024-06-18T14:06:18.910Z

rauhul:

However, the current implementation of any requires Reflection metadata which we cannot bring to Embedded Swift.

I think in principle it doesn’t need any more runtime support than vtable dispatch. We’ll need to pre-specialize metadata records though (just like we pre-specialize vtables of generic classes).

wadetregaskis · 2024-06-18T14:28:00.226Z

Max_Desiatov:

One may not realize the massive overhead to performance and binary size that existentials bring to the code.

Although in fairness, Swift's generics tend to bring a pretty disturbing amount of overhead, too.

However, in Embedded Swift mode their most poor-performing aspects are apparently disabled (e.g. they are always specialised in Embedded Swift), so they might be safer to use there. Indeed, it might take people already familiar with non-Embedded Swift quite some time to really shift their mindset towards generics being safe to use in Embedded Swift.

filip-sakel · 2024-06-18T14:29:49.072Z

What do you mean by metadata-record prespecialization?

rauhul · 2024-06-18T16:23:44.539Z

wadetregaskis:

However, in Embedded Swift mode their most poor-performing aspects are apparently disabled (e.g. they are always specialised in Embedded Swift), so they might be safer to use there. Indeed, it might take people already familiar with non-Embedded Swift quite some time to really shift their mindset towards generics being safe to use in Embedded Swift.

Generics in embedded swift come with a different problem because they are monophorized. On one hand we get the benefits of specializing all uses, on the other hand we can end up duplicating a lot of code and hoping the llvm outliner recombines the common parts.

any or something similar could allow the programmer to opt into dynamic dispatch and a single common implementation saving on codesize (and potentially runtime performance).

Alejandro · 2024-06-18T16:55:30.450Z

Max_Desiatov:

One may not realize the massive overhead to performance and binary size that existentials bring to the code.

Performance yes, but existentials themselves are just a data structure and don't necessarily bring large swaths of binary size (I guess it takes more instructions to operate on them..?). They just store pointers to metadata if any Any and another pointer to a witness table for any Protocol. These structures are already emitted currently just by definition so an existential itself doesn't bring any more baggage in terms of needing to store data in a binary.

That said, in embedded contexts where these structures don't exist at all currently then yeah introducing specialized metadata records and protocol witness tables would come with some baggage, but some folks may be willing to pay for it. (These structures aren't super massive by themselves either, it's just the collection of tens to hundreds of them that make it expensive)