Major evolution of the type system, including protocol improvements, etc

See e.g. "Swift 6 mode" discussed here Design Priorities for the Swift 6 Language Mode

2 Likes

The -swift-version compiler flag, #if swift(>= 5), etc.

1 Like

Yeah, I think this is the important point. There are undoubtedly pragmatic considerations that make it more difficult to contribute major features Swift as an individual who is not able to work on the compiler full-time (of which Apple employs many), but that's not due to a preference for contributions originating from Apple engineers all else being equal.

In fact, that characterization feels like it gets the causality entirely backward—it would be quite surprising if Apple's Swift engineers weren't, broadly, putting the bulk of their effort towards projects that were closely aligned with Apple's, the core team's, and language steering group's visions for the Swift language. Anything else would speak to major organizational dysfunction.

To try to get a bit back on-topic, @YoshihiroUnno, I suspect that each of the things you propose would be better served as its own thread dedicated to the topic rather than a series of posts within the same topic with the broad goal of foundational changes to the type system, which, as you can see, is bound to elicit broad discussion rather than focus it on the technical details of a specific proposal. :slightly_smiling_face:

10 Likes

Thanks for the discussion everyone.
This may sound verbose, but I'm not good at English, so I'm not sure how the discussion went.
If I have the opportunity to speak again, I will speak.
This idea presentation from me stopped halfway.

With estimated number of all developers being 30 million, one million for Swift (~3%) sounds about right.

1 Like

The overarching goal of my proposal is to redefine protocols more clearly as abstract types, making them easier to work with and taking full advantage of the capabilities of abstract types to play an active role in Swift programs. That's it.

This is an idea I would love to share with you.

I thought the existing protocol would need to be modified to make this idea a reality, but I realized there was a way to leave the existing protocol as is and add a protocol based on new concepts to Swift.
Thanks to everyone for their advice.

Still, I think the realization of this idea will be very big.
I would like you to continue to announce it in the future and evaluate whether it is worthy of realization.

In general, abstract types are types that have specific functional and interface requirements and do not specify their implementation.
Class types are powerful data types that combine the concepts of abstract types and inheritance.
So a class type is essentially a kind of abstract type.
And Swift uses this abstract nature of class types as if it were a given.

Protocols, on the other hand, also fit nicely into the definition of abstract types.
Protocols without an associated type have traditionally been used as abstract types in Swift.
For protocols with associated types, the interface requirements depend on the associated type.
Using this type of protocol as an abstract type would require an explicit value for each associated type, which was not possible for a long time, but has become possible with the advent of PAT.

Now we are very free to use protocols as abstract types.

However, I believe that protocol as an abstract type is currently limited to use as variables to store and manipulate data instances.
It might be the best we can do without breaking current protocol compatibility.

The new protocol is not limited to such uses.
Just as a class type is used as a source of inheritance when defining a new class type, a protocol as an abstract type can also be used as a source of inheritance when defining a new protocol.
This also applies to conforming protocols when defining new data types.
Also, the current protocol accepts only concrete data types for associated types, but the new one also accepts protocols as abstract types.

To illustrate how this is possible, take IteratorProtocol, Sequence, and Collection as examples.

Below are the current definitions of each (partial excerpt).

public protocol IteratorProtocol<Element> {
    associatedtype Element

}

public protocol Sequence<Element> {
    associatedtype Element 
        where Self.Element == Self.Iterator.Element

    associatedtype Iterator : IteratorProtocol

}

public protocol Collection<Element> : Sequence {
    associatedtype Element

    associatedtype Index : Comparable 
        where Self.Index == Self.Indices.Element, 
            Self.Indices.Element == Self.Indices.Index, 
            Self.Indices.Index == Self.SubSequence.Index

    associatedtype Iterator = IndexingIterator<Self>

    associatedtype SubSequence : Collection = Slice<Self>
        where Self.Element == Self.SubSequence.Element, 
            Self.SubSequence == Self.SubSequence.SubSequence

    associatedtype Indices : Collection = DefaultIndices<Self> 
        where Self.Indices == Self.Indices.SubSequence

}

On the other hand, below is the new definition that I propose.
It uses the new syntax I propose.
Read about them here(https://forums.swift.org/t/major-evolution-of-the-type-system-including-protocol-improvements-etc/65249/10 , https://forums.swift.org/t/major-evolution-of-the-type-system-including-protocol-improvements-etc/65249/7).

public protocol IteratorProtocol<
    Element: Any> {

}

public protocol Sequence<
    Element: Any, 
    Iterator: IteratorProtocol<Element: Element>> {
    
}

public protocol Collection<
    Element: Any, 
    Index: Comparable,
    SubSequence: Collection<Element: Element, SubSequence: SubSequence> 
        = Slice<...>,
    Indices: Collection<Element: Element, SubSequence: Indices> 
        = DefaultIndices<...>> 
    : Sequence<
        Element: Element, 
        Iterator: IndexingIterator<...>> {

}

First, notice the definition of Sequence.
As currently defined, the associated type Iterator accepts only concrete data types conforming to the IteratorProtocol.
On the other hand, in the new definition, the associated type Iterator accepts any type that satisfies the requirements of IteratorProtocol<Element: Element> as an abstract type, not limited to concrete data types.
This relaxes the requirements for the associated type Iterator.
This will also relax the requirements of the protocol Sequence itself.
Also note that syntactically, we don't need to write a where clause.

Next, look at Collection.
It inherits from Sequence.
In the new definition, the Sequence<Element: Element, Iterator: ...> part of the inheritance source is written in a similar (but slightly different) way to inheriting a class type with type arguments.
It is written in such a way that it inherits from an abstract type.
It should be written here as a complete protocol.
There is no need to write a where clause here either.

Talking about associated types, the current definition is that associated types are related in complex where clauses and I may not be interpreting them correctly.
However, I think that the new definition will be as written here.
I'm assuming this is correct, but I think it's pretty easy to understand what kind of requirements each associated type has.
And these, too, are relaxing the requirements as much as possible.
As a constraint on an associated type, we can write an incomplete protocol.

1 Like

I would like to explain my thoughts on the contents of the new protocol definition, but first, let's organize the current situation.
In current protocols, definitions are usually split into two definition blocks.
The first is a protocol definition block starting with "protocol" and the second is a protocol extension block starting with "extension".
The role of the protocol definition block is clear: it describes the protocol's interface requirements.
On the other hand, protocol extension blocks have two main roles.
The first is to define protocol-specific properties and methods that are absolutely necessary for the protocol to work. The second is to define default implementations of interface requirements.

Now let's talk about the new protocol.

The new protocol obsoletes the default implementation.

One reason is that if a class type chooses the default implementation, inherited class types cannot change the implementation, and we want to prevent that situation.
A workaround for this issue is to have a slightly renamed implementation that can be used arbitrarily instead of the default implementation that directly matches the requirements of the interface. However, this requires some experience.
Deprecating the default implementation makes this technique mandatory and common sense.

Programs proactively and explicitly indicate what kind of implementation they provide for every interface requirement if the default implementation is obsolete.
And all their implementations are guaranteed to be modifiable in inheriting class types.
I think this is rather desirable for developers. This is one of the big reasons.

The new protocol does away with the default implementation, but instead introduces a mandatory implementation. More on this later.

2 Likes

I'm sorry that the topic changed so much because I didn't think about the overall structure of the presentation well.

Creating a new type by inheriting a base type is called type inheritance.
Currently, Swift has class type inheritance and protocol inheritance, but it can be said that these are type inheritance based on a very different interpretation.
Swift's class types are designed around the so-called general concept of class inheritance. The idea is that what is created is both the type it inherits from and another type that adds new elements to it. The ability to change the implementation of existing elements is also a very important part of this concept.
On the other hand, the current protocol inheritance does not have this notion. I think it's because it's designed around a special definition of "a protocol doesn't conform to itself".

The new protocol will fully embrace the concept of class inheritance mentioned above. This is made possible by allowing protocols to be abstract types.

Also, off topic, new protocols will be able to have their own stored properties.

1 Like

I wrote the term "mandatory implementation" in a post a while back. I will explain this.

Interface requirements written in protocol definition blocks are typically expected to be implemented by conforming data types.
However, when protocol inheritance is used to create a new protocol, it is possible to give a specific implementation for the interface requirements of the inherited protocol for the convenience of the new protocol.
This is the "mandatory implementation".

An interface requirement that has a mandatory implementation cannot be given an implementation in a conforming data type. Also, any implementation that inherits from the data type it inherits from can no longer be changed or used.
A mandatory implementation can be changed in the protocol that inherits it.

1 Like

Since the new protocol does not require protocol extension blocks, the entire protocol is described in the protocol definition block according to the following rules.

  • Write the interface requirements as before, without the implementation content.
  • Describe protocol-specific elements (properties, methods, stored properties, etc.) with implementation details.
  • To give a mandatory implementation to an interface requirement for the first time, write it without "override". This is the same as when providing an implementation in a conforming data type.
  • Add "override" to override the mandatory implementation or the implementation of protocol-specific elements. This is similar to overriding in class types.
  • In order to use the implementation before overriding in the description of overriding the implementation, add "super."
  • No protocol qualifiers are required for interface requirements or protocol-specific elements to be defined. The same is true when they are used within the protocol.
  • In order for interface requirements and protocol-specific elements to be used within inherited protocols and data types, they must be accompanied by protocol qualifiers. The same is true for writing mandatory implementations of them or writing overrides of them.

The "required" attached to the required initializer is considered unnecessary. But on the other hand, it might be better to mark all interface requirements with "required".

Whether or not protocol qualifiers are required when defining, overriding, or using subscripts has not yet been considered.
Static interface requirements and static protocol-specific elements have not yet been considered.

Since the new protocol can have a namespace similar to other data types, it seems possible to define nested types.

1 Like

I will explain "Self".

In the current protocol, "Self" is used to mean "a data type conforming to that protocol". This is an essential language element for writing protocol definitions.
On the other hand, the new protocol does not give a role to "Self" in such roles. A possible role is as a predefined type alias that points to its own complete protocol with the same type parameters as itself.
So "Self" isn't essential in the new protocol, but I don't think it's a bad idea to introduce it for convenience.

Next, let's talk about "self."

In the current protocol, "self" represents a read-write variable that stores an instance of some data type conforming to that protocol. I think it's fair to say that a protocol is something that manipulates this variable through its own interface.
Even in the new protocol, "self" is the same in terms of language specifications.
Therefore, it seems appropriate for the new protocol to use "self" in the same way as the existing protocol.

Yoshihiro, I think I probably need to step in as a moderator here. I appreciate that you're enjoying thinking through this language design concept, but if there isn't really any path for adopting it into Swift, at some point it's just off-topic for these forums, which are not general language design forums. At the very least, I need set your expectations appropriately.

Please, write up this idea as a white paper about the general design of protocols, like a concrete language proposal would. Pick a syntax that you like and show some examples in that paper of what code would look like under your vision. Post that paper somewhere, perhaps to github or a personal blog, and feel free to link to it here. As we tried to convey before, it is very unlikely that Swift will adopt your proposal completely, but if we can see it laid out comprehensively, maybe there's still something we can learn from it. If nothing else, maybe some other language's designers will see your ideas and incorporate them into a different language; that's part of how progress happens in programming languages.

If you decide to take that path, I'd be happy to leave this thread open for us to discuss your paper when it's done. But I do have to ask you to stop writing an abstract design monologue on the forums, and I will close the thread if it continues.

12 Likes

Thank you for your advice.
I apologize for posting haphazardly without laying out the content of the proposal well.
I'm going to stop posting here and try to write a white paper.
I have to start by studying what a white paper looks like. Also, I'm not even sure I can write enough content with just my knowledge.
I would appreciate it if you could give me an example of the layout of the white paper I should write.

1 Like

A technical white paper is both a technical document and a persuasive essay: you want to lay out in detail what you see as the problems with the current situation, describe your suggested alternative, and make the case for why your idea is the best thing to do. It might look something like an evolution proposal, just without some of the extra structure about compatibility.

4 Likes

Thank you for letting me know.