How to disable implicit Foundation imports?

I think this is the import in question and we should be able to make it implementation-only.

We will probably have to make any such change depending on the tools-version of the package being >5.7 in order to not break existing packages that might accidentally rely on Foundation being available implicitly.

2 Likes

Arguably, it’s a straight-up bug in user code to depend on a leaking implementation detail of SwiftPM for an implicit import of Foundation, which most users would appreciate having the opportunity to fix with explicit import Foundation. Well, not merely arguably—this entire thread is that argument actually being made. Debugging the status quo is much less intuitive than an error message telling you to import Foundation.

7 Likes

I wonder if an additional measure that may be viable, which wouldn't requiring import scanning, etc., is to vend this extension of Foundation as an overlay, i.e., only when both Foundation and the library are imported.

6 Likes

@NeoNacho thanks for tracking down the code in question.

Candidly, and at the sore risk of pushing the discussion in an unproductive direction (not at all my intention), but the reliance on Foundation for essentially very simple things like this (literally just a path string) makes my blood boil just a little bit. I should qualify that statement by saying my aversion is mostly because Foundation is a huge monolithic entity that only has partial (and varied!) support for 3rd party platforms, and has inherited a lot of Objective C strangeness, recreating APIs for many basic types in fundamentally non-Swifty ways, and implementing other, ostensibly useful functionalities in entirely non-type-safe ways. For all intents and purposes that I'm familiar with, if devs are able to to avoid Foundation – at least in non-Darwin code – they should.

For that reason, it really bugs me that Foundation takes so much room in Swift's collective consciousness, and I feel the community would be so much better off with more modular and "Swifty" alternatives. I gather the decision makers at Apple feel the same, at least to some degree, which is why there is a push for the numerous and ever-increasing number of open source SwiftPM packages that have been published over the last years.

What I'm trying to say is, in my humble opinion, the move away from Foundation as a legacy monolith can't happen quickly enough, and I would greatly appreciate essential tools like Swift Package Manager taking a stance to avoid essentially forcing its adoption like this.


Edit: As an attempt to make this post more productive and less "complainy": what if we had a standalone module for Bundle, that could be imported here instead of "the entire thing"? That module could also be @_export imported by Foundation for reasons of backwards compatibility

10 Likes

I greatly appreciate that you've moved so quickly to improve this situation @xwu, thank you!

2 Likes

this is not at all an unproductive direction, in fact i personally think it is a very productive thing to talk about (at least as a library author) and the only reason it is not talked about more is because someone still has to write the smaller, more modular frameworks to replace Foundation, and we do not always have time to take a 3 week detour to write a pure swift library so more often than not we just go down the Foundation route because it is the path of least resistance.

the issues with Foundation are well known and if anything i think the things you have mentioned are only touching on the tip of the iceberg.

again i do not think anyone disagrees with what you are saying but it is a very big task and “i hate Foundation” is necessary but not sufficient. in particular many of us pure swift library authors are not good at cocoapods or XCode so writing libraries that don’t import Foundation but only compile on amazon linux and only compile on this week’s nightly toolchain is not a step forward it is really just more of the same if we are being honest.

if we want to make progress it is important i think to focus less on “we need more foundationless libraries” because there actually are a lot more than people think but they don’t get adoption because going from linux to macOS/iOS/watchOS/tvOS is just as hard as the other way around and the ecosystem is still fragmented just in a different way.

I would appreciate a mod moving what is essentially a new thread here (starting with my post above) actually into a new thread. I think it could be quite productive to discuss potential solutions.

For me, I'd be happy if I could just import and link, say, the Foundation implementation of JSONDecoder without the kitchen sink coming along for the ride. In the JavaScript world the solution to that is known as tree shaking but, in both languages, this is easier said than done with automated tooling alone. As evidenced in this thread, that doesn't seem to work at all for Wasm, and I also haven't had any luck doing so on Android – i.e. it doesn't work where we need it the most.

I personally would be happy with a solution that modularises small parts of Foundation and reduces interdependencies between those modules; maybe in the future these could be moved out of the library entirely. The difficulty I see is that Foundation seems to be deliberately written in such a way that increases interdependencies between its parts – philosophically it appears to want to be "the platform", and as a codebase appears to implicitly reward dogfooding from within.

In the immediate-term my main concern is about that philosophy itself – Foundation being "the platform" probably makes sense on Darwin because the library is shipped with the OS and has years (decades?) of battle testing, so embracing it there is essentially low cost and all win. The issue is when that mentality leaks, for example via SwiftPM, into platforms (arguably all non-Darwin platforms?) where the opposite is true: the cost of linking Foundation is high, and the implementation itself is incomplete and often incompletely tested (-> buggy).

4 Likes

"At the beginning people complained that Foundation doesn't exist on non Apple platforms, now when it is finally there people doing their best to avoid it..." :sweat_smile:

Considering static linking only: It looks like you would be happy if strip did its job better - and it doesn't (*) - perhaps then this is the strip whom to blame. This is not Foundation specific - the same could be said about any large library that explodes the app binary size even if you use a single function in the library. Splitting things into micro packages to avoid app size issue works but feels like a roundabout / workaround solution.

(*) or maybe strip does work and without it the app size would be much larger? :thinking:

PS. I understand that you'd prefer 3MB to 26MB app, it does fell wrong and I am fully with you on that. However is this just that, i.e. "it feels wrong" or do you have some real practical issue with that, like you are shipping your app to your customers over 3G connection and it takes ages? To be honest I don't even know how many MBs is the app we are shipping, won't be surprised (or care much) if it is 5MB or 50MB.

7 times out of 10 i am installing apps when i am cellular data or spotty wifi or spotty wifi that displaces cellular data because of iOS’s fallback and if it takes too long i give up on the download and my friends are the same. and if it is a website and not an app store app i am even more likely to bounce.

so i totally understand why this is an existential problem for app developers.

2 Likes

Nobody ever said this. We wanted Swift native versions of some of the functionality in Foundation, and for it to be open source. Apple's eventual answer was swift-corelibs-foundation and... well... here we are.

4 Likes

Yes, I would say the vast majority of internet users don't have an Ethernet cable running to their device (especially if it's a phone or a tablet). You don't have to use 3G to experience this, just try downloading something at a hotel, a coffee shop, an airport, or at some residential area with heavy WiFi congestion. And I'm only speaking of 1st world countries that have a privilege of having something better than 3G available.

1 Like

I'm talking about a web app, which has much more ethereal "download" characteristics (i.e. it's not downloaded and then "done"), and is updated more often than a typical mobile app. A typical mobile app may well be ~50MB, but a typical web app tends to be much smaller than that.

Also, users don't make as much of a conscious choice to download a web app – there is no popup asking them if they want to download it –, so it doesn't feel good at an ethical level to regularly spam them with 10s of MBs of downloads that may or may not be cached.

Also, we (essentially have no choice but to) mirror our app in China, which comes tied with endless amounts of performance considerations. The connection between China and the rest of the world is tenuous at best, and people's internet connections within China are also not always as good as they are elsewhere.

I could continue, but in short, it matters a lot :slight_smile:

2 Likes

Interestingly to see people commenting upon the easy "why 3MB is better than 26MB" point yet nobody picked up upon why strip doesn't seem to be doing what it's supposed to do to keep the app size still around "3MB". Is it the case that "String.trimmingCharacters" indirectly touches the whole/most of the Foundation (perhaps unicode tables (1))? Or is strip inherently incapable / unreliable?

(1) and if it is indeed due to inclusion of unicode tables perhaps Foundation could have some alternative that won't explode the app size? bike shedding:

"string".nonUnicodeAwareTrimmingCharacters(in: .nonUnicodeAwareWhitespaces)
Somewhat relevant tangent.

All good reasons above why we need to keep app binaries small. Yet we (the industry as a whole and Apple in particular) have got ourselves in a situation when we increased all binaries by some 30-70% (my estimate, please correct if you have a better figure) by switching all apps to 64 bits, even if 99.99% (another estimate of mine) of apps don't utilise the benefits of being 64 bits. Isn't that a mistake?

3 Likes

As for Unicode tables, I've mentioned this before in other threads: after stdlib switched from ICU to its own tables, the situation became worse in terms of binary size for statically-linked apps. This is caused by Foundation not reusing the tables from stdlib and still relying on ICU. Thus statically-linked apps bundle two separate Unicode tables, one instance from stdlib, the other one from ICU.

7 Likes

I guess it is not yet capable, there's e.g.:

3 Likes

in your opinion, what are a few of the biggest obstacles preventing you from adopting pure-swift alternatives like swift-json, swift-png, or swift-url?

Ignoring those specific libraries, I really don't know. My guess is that Apple's leadership is unwilling to spend resources on such open source work and swift-corelibs-foundation was a political compromise within Apple that technically allowed Apple engineers to develop in the open because it was still controlled by Apple and could be used in the backend environments, even if it wasn't ideal for the language. This bootstrapped some open source functionality into Swift but also took energy away from native replacements, which is compounded by the lack of investment in Foundation from Apple more recently.

On the plus side it seems likely that the existence of swift-corelibs-foundation played some part in the recent open evolution proposals for URL and other Foundation improvements Apple is shipping this year.

But this is all speculation, which many really don't like here, so I'll leave it at that.

I don't think we should even be questioning OP's concerns about binary size - OP isn't using Foundation, so they shouldn't have to pay for it.

Whether OP's concerns apply to anybody else's projects, and how good/bad internet connections around the world may be, is not relevant to the original question.

7 Likes

this isn’t really helpful for those of us who make pure swift libraries because we are not part of Apple and not part of the corelibs project. so we can’t really make better libraries until we know what we specifically are doing wrong and not what Apple is doing wrong.

I've been trying to knock out the dependencies my Swift programs have on corelibs-foundation recently. The issue I've run into with using these foundation-less replacements is that a number of core dependencies already import Foundation. So then what's the point of depending on one of these libraries when I already have to "pay" for Foundation & everyone already knows Foundation? (the specific libraries I've come up against are Soto & async-http-client)

For me that's definitely the major issue. I do think that replacements for major currency types like Date and maybe Data (though some Collection<UInt8> is pretty good for that) would really benefit from inclusion in the standard library just because they become so much more useful when everyone can agree on the same type. Just like Result used to be.

Specific feedback on swift-json

This library looks really awesome and if I ever do manage to banish Foundation from my server components, it will probably be what I use for JSON decoding.

Encoding is obviously an important feature to make it useful for replacing Foundation and I see you have that on your roadmap, which is great.

The other thing I think would really help (just my 2 cents!) would be a very simple entry point into the package. It's a small thing, but just decoding a Decodable type (where I think most Swift programmers would want to start with a package like this) requires seeing the following types:

  • JSON
  • Grammar
  • JSON.Rule
  • JSON.Rule.Root

If I were the author of this package, I would add a convenience API for what I think is the most popular use:

extension JSON {
    static func decode<Data: Collection, Decoded: Decodable>(_ data: Data, as: Decoded.Type) throws -> Decoded { ... }
}

So then your getting started example could be:

import JSON
struct MyType: Codable { var x: Int }

let input = #"{ "x": 1 }"#
let object = try JSON.decode(input.utf8, as: MyType.self)

Then the section after that could get into more advanced things like annotating error messages and fancy grammar things. Basically I think when the community & tools have such a heavy bias towards Foundation you have to make it as simple as possible to use an alternative.

The WebURL getting started section is extremely focused by comparison, even though that library has tons of complex functionality:

import WebURL

var url = WebURL("https://github.com/karwa/swift-url")!
url.scheme   // "https"
url.hostname // "github.com"
url.path     // "/karwa/swift-url"

url.pathComponents.removeLast(2)
// "https://github.com/"

url.pathComponents += ["apple", "swift"]
// "https://github.com/apple/swift"
9 Likes