Possibility of more openly documenting Swift's internals?

My perception of the official documentation of the language (the Language Guide) is that it is quite high-level and only covers user-facing features. Beside some exceptions, it basically never mentions how most things work under the hood. This makes the initial introduction to the language generally concise, which is a good thing; however, beyond this level, there's basically zero to none resources (especially official) that would provide interested users with a deeper view into the internal mechanisms of the language features.

My personal story with this is that, being very careful about my code on the one hand, and having a genuine interest for low-level mechanisms on the other, this state of things left me feeling somewhat lost whenever I tried to get a good enough understanding for some features, in the end abandoning the idea of incorporating them into my code because I oftentimes just wasn't sure if they were doing what I was thinking they would do. A good example for me is async/await: it was clear for me from the docs (and overall exposure to programming trends) what async/await is essentially doing, but I never could grasp how exactly it was doing its thing until I allocated myself a cozy weekend of extensive research watching great in quality, but scarce in numbers, talks on Swift's inner workings like this one.

To perhaps generalise beyond my personal anecdotes, I think one would agree that developing a better intuition for the internal logic of a programming entity, be it a language feature or a data structure, leads to generally better code. The oder side of the coin could be that without sufficient intuition, one might unknowingly misuse those said feature.

Now, the previous paragraph might have been trying to present an obvious argument, but sometimes I had a feeling that some documentation tends to intentionally operate with generic "rules" as in "the use of X is discouraged" or "only use Y if you're sure" without pinpointing what exactly the problem could be, and I have a feeling that this could have something to do with the so-called Hyrum's law. Which makes me wonder—and this now refers to the question in the topic's header—is this a somewhat deliberate policy that if some sort of behaviour has not been declared guaranteed, then it shouldn't be exposed within any official documentation?

It is obviously not a non-goal to document those implementation details that genuinely are subject to change at any point; however going back to the async/await example, I would assume that it's most likely the case that the particular coroutine implementation will be there to for more than a minute, and just understanding how it works would improve the code of many by a lot.

So, would the Swift community be interested in expanding the docs to include some of the internals for educational purposes?

The obvious example to close off my message is Rust's Nomicon. I terribly wish Swift had something like this resource.

6 Likes

@Slava_Pestov has been writing things like this. Compiling Swift generics, Part I

3 Likes

I perhaps should mention that I'm aware of the existence of some more internal docs in the Swift's repo, but they are somewhat hard to find, don't have the "official" flavour and may not be kept as up-to-date as one may wish.

2 Likes

Oh indeed, I've read these documents, and I love them. They are actually a good example for my sentiment—have I not been here at the right time (or perhaps have not you referenced them here), I would have never discovered them, and that's a great pity.

There's a lot we can do to improve the documentation of the implementation, but I'm hesitant to do anything that implies that it's important for people to understand that implementation in order to program in Swift.

14 Likes

I'm actually utmost curious to hear why you think so.

Personally, Swift was my first language and is to this day my primary one. I also know Rust and C++ to some medium level.

However, there's a catch: I never had a reason to code in Rust or C++; I learned them exclusively in an attempt to understand Swift better. I learned C++ to read the compiler and runtime code because I couldn't get enough context from the documentation alone—and Rust to reinforce my understanding of the more strict ownership categories and low-level concepts.

This is... quite an extreme case, certainly, but it's to make the point that the best resource to some features of Swift for me was a whole different programming language. This is, of course, just a single example, but I hope we'd both agree that it's a suboptimal one.

I'd absolutely adore the world where language features in and of themselves wouldn't have required the knowledge of intrinsics, but then I wonder: how else can you make people understand them?

5 Likes

If I know how things work internally, I can probably write much better code. :slight_smile:

3 Likes

Well, I think this is actually a really important point. There's a difference between (1) understanding how language features work at a level that influences how you use them and (2) understanding how language features work at a level where you could plausibly reimplement that part of the compiler / runtime. The latter is what I think of as the "internals" of the implementation, and documentation of it is going to be directed at language implementors who need to make changes in that system, not at Swift programmers who want to understand how a high-level feature works. We can talk about how to provide more advanced documentation than TSPL — explanatory articles that would help programmers to understand the lower-level implications of using various language features. But we do have to draw the line between that kind of advanced user documentation and formal specifications / subsystem implementation documentation that's primarily of use to researchers and implementors; and we also have to be careful not to imply that programmers must understand these details in order to use Swift correctly.

21 Likes

I think that's a good point. As I understood I think the author was aiming mostly for (1), or rather I found myself to be in that boat. I have a similar experience where reading the basic language guide isn't enough to know how to use some language features properly. Most of the time it would have to be a combination of reading the language guide, looking at WWDC videos, reading a third party book - e.g. by objc.io - and reading questions on this forum that people ask to get a holistic view on how something works. And this is without trying to go into (2) implementation details (which sometimes does help though).

I see this as a proposal to try and centralize all of that information as an advanced extension to the basic language guide that goes more in depth as, for instance, WWDC videos would go.

8 Likes

It could be something like a ‘under the hood’ section of the documentation.

Yes, this should not be mixed up.

1 Like