Will OrderedDictionary ever graduate?

i would not say that i am actively avoiding SPM, rather that SPM has certain limitations that make it unsuitable for a non-trivial swath of projects. i personally am not a user of XCode, but i have run into scaling problems using SPM for projects (on linux!) that have a large number of targets, because SPM only supports one manifest per project, so if you are committed to using SPM, you have to live with enormous (and arguably, unworkable) package manifests. it is hard to avoid the simple truth that once a project reaches a certain size, SPM can no longer keep up.

this, to me, suggests that SPMā€™s problems are multifaceted and could be very challenging to resolve, and i feel that preconditioning OrderedDictionary on improvements to SPM is needlessly coupling two things that ought to be independently addressable.

7 Likes

Do you see any way to accomplish what you want without subjecting the libraries to evolution and binary deployment concerns? I realize half of that isn't an issue on Linux, but we can't just pretend 90% of Swift's users won't be impact by deployment limitations. I have a hard time envisioning a good solution that doesn't involve some sort of language work, even if, say, Collection was included with Swift but not part of the default import. How can we get something like that to work while still allowing users to use the external version to get newer capabilities or bug fixes?

I agree that SPM has issues; I think Swift has come a long way, and we've sort of outgrown what SPM is currently able to provide.

I remember that the original goal was to have a minimal standard library. OrderedDictionary is certainly very, very useful, but is it really such a fundamental building block? And if it is, what exactly distinguishes it from Deque? Or Heap? Or BitArray? Or TreeDictionary, SortedDictionary, SparseSet, and all the other types in (or soon to be in) swift-collections?

Since OrderedDictionary currently exists in a package, it clearly does not need to be part of the standard library. I don't know if the original goal of a minimalist stdlib is still a goal today - but if it is, it obviously depends on us having an excellent and easy to use package manager to provide that which the standard library chooses not to.

When it comes to online coding platforms, I wonder if SwiftPM support in scripts would help. If you could just write:

@package("apple/swift-collections")
import Collections

And then have swift run script.swift do all the SPM work for you (downloading and linking the package), I think that would give an experience approximately as good as if it were a toolchain library.

Another thing to consider - any types that go in the standard library will become part of the OS on Apple platforms, meaning they will require @available guards. Yes, package resolution is annoying, but being able to use the data type immediately, everywhere, regardless of your deployment target, is rather nice compensation.

15 Likes

ā€¦ without restructuring the whole thing to use SPM instead.

Iā€™ve worked on multiple projects that use both SPM and Cocapods simultaneously. I donā€™t understand the ā€œinsteadā€ aspect of the comment here.

1 Like

I've never done that before, so I have no idea what it would entail.

It doesn't entail much, you need need to make sure none of your dependencies in the separate managers share any dependencies, otherwise you may run into duplicate symbols. If nothing else it will waste binary space with duplicates of any shared dependencies. I tend to use CocoaPods for large binary dependencies, as CocoaPods can still conditionally link them based on build configuration, leaving my debug builds much faster to iterate.

2 Likes

Off-topic tangent.

Going back fifteen to twenty years, when I was at Apple, "Core"-prefixed frameworks basically meant that their APIs were plain C. Often using CoreFoundation. As opposed to Objective-C frameworks.

That's clearly no longer the case (e.g. CoreAnimation).

This naming scheme was never used consistently - there were always plenty of frameworks without "Core" in the name that also just had C APIs. And as far as I recall there were never any special 'privileges' or status given to "Core" frameworks - the weren't imported implicitly, they didn't ship anywhere the rest of the frameworks didn't ship, it didn't have any direct bearing on whether they were included in Darwin or not, etc.

Sometimes they came in pairs - "CoreFoo" and "Foo", e.g. CoreFoundation and Foundation. With the idea being that one was the fast, efficient, powerful C version and the other was the Objective-C wrapper. But those lines blurred too, in time.

As far as I recall nobody ever enforced any of that, it was at best just organisational knowledge / convention. So I'd be amazed if there aren't many exceptions. Usually each framework was named by whichever specific person did "File > New Projectā€¦" in Xcode (or Project Builder).

"Core" didn't mean first or least, either - e.g. there was once the Symbolication framework, written in Objective-C, which was replaced with CoreSymbolication, written in C++ (with C API), because classes and message-passing (even with IMP-caching etc) were just way too slow and RAM-inefficient for that purpose. CoreSymbolication was hundreds or even thousands of times faster, and meant e.g. developers at Adobe could symbolicate time profiles of Photoshop in minutes instead of hours.

7 Likes

In other IDEs (e.g. IntelliJ) you can just start using a class (e.g. OrderedDictionary) in code, and the IDE automagically inserts the relevant import directive. I'd love to see Xcode do similar - add the import, do the SPM add, etc.

There are of course many conditions to handle - e.g. multiple packages that contain the same symbol name - but there's ways to address those. e.g. have the project pre-defined which package collections & registries it uses, in priority order. If all else fails prompt the user with a FixIt with multiple options, one for each candidate package (IntelliJ does this).

It's even hypothetically possible for Xcode to disambiguate between packages based on further contextual information, e.g. if you use not just OrderedDictionary but a specific method there-on which only exists in a specific OrderedDictionary package. Xcode has leeway here since it doesn't necessarily need to resolve the reference immediately, just before it does the next explicit compile.

And I'm sure that on the 'backend' this requires some work - e.g. some kind of big symbol index that Xcode can use to determine which package(s) contain what you seem to be referring to - but IntelliJ etc are existence proofs.

I believe Xcode 12 added such a feature (for symbols that were in any linked framework) but it was so overzealous (autocomplete would offer all system and linked symbols, allowing automatic import of random frameworks when users accidentally accepted the wrong result, and couldn't reverse the import when the usage was removed) that it was removed in Xcode 13. Auto import is essentially required for Java given how granular and verbose its imports can be. Swift doesn't really need to go there yet. In fact, it would be far more beneficial to filter the vast majority of completion results out entirely, like the random system C libraries you can get for any fuzzy input. Basically, the completion system needs a rework before it can support something like automatic imports while maintaining good DX.

4 Likes

Huh, I don't remember that. I must have overlooked it entirely.

IntelliJ also doesn't remove now-unused imports, which is annoying but doesn't invalidate its feature for adding them. And there are a lot of linters etc which can be used to fix those dead dependencies.

Sounds like Xcode's implementation just wasn't quite over the acceptability hump. I hope that doesn't discourage the Xcode team from trying again.

I think encouraging more modularisation is wise, from where Swift is now. I regularly see anti-dependency commentary in the Swift community, and consequently a lot of reinventing wheels. A lot of it stems from toolchain issues ("Resolving packagesā€¦" etc), but some is simply the friction of having to manage dependencies completely manually (though kudos to Xcode in recent versions for making adding package dependencies easier via the GUI).

Perhaps you can imagine how empowering it would be if you were coding away, in the flow, got to a point where you needed a heap, and you just write var heap = Heap() and it automagically pulls in a suitable implementation (perhaps based on your project & Xcode preferences, community ratings, GitHub stars, whatever).

That neatly moots a whole bunch of hard questions about what belongs in the standard library(ies) or not, by simply making the distinction much less important in practice. Not completely unimportant - see prior comments in this thread regarding some of the other trade-offs - but closer to.

It looks like it'll be increasingly important even with the "standard" libraries, given the move to break up Swift Foundation into many packages.

I thought it was added in Xcode 13 and removed in Xcode 14.

SwiftUI's Identifiable moved to the standard library.

Not really, it was simply removed from SwiftUI and placed in the standard library during the beta process. In doing so it was put through the evolution process and never shipped as part of a stable SwiftUI release. And it was never placed in an external library like swift-collections, which is what this thread was about.

1 Like