[Proposal] Higher Kinded Types (Monads, Functors, etc.)

Looking beyond functional programming abstractions, higher-kinded types are a fancy way of saying "template template parameters". A textbook motivation for those from C++ land is Andrei Alexandrescu's "policy pattern", stuff like this:

protocol RefStoragePolicy: <*: class> {
  typealias Ref: class
  init(ref: Ref)
  var ref: Ref
}

struct Weak<T: class>: RefStoragePolicy { weak var ref: T }
struct Unowned<T: class>: RefStoragePolicy { unowned var ref: T }
struct Strong<T: class>: RefStoragePolicy { var ref: T }

class HeterogeneousRefConsumer<Storage: RefStoragePolicy> {
  func consumeRef<T: class>(ref: T) {
    let storage = Storage<T>(ref: ref)
    doStuffWith(storage)
  }
}

-Joe

···

On Dec 16, 2015, at 5:10 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 16, 2015, at 1:34 PM, Douglas Gregor <dgregor@apple.com> wrote:

With higher-kinded types, one could potentially [*] produce a collection of the same kind as Self but with the element type T. So, mapping a Set produces a Set, mapping an Array produces an Array, etc.

Yeah, but Set can’t follow the functor laws, so that map won’t apply to it. That’s why, while I agree that this kind of constraint is helpful and important for completing the generics picture, IMO it is probably less helpful/important than many people think.

A lot of the uses don’t necessarily require HKTs as you can get a lot of mileage out of a structure preserving map with code written against the specific type. For example, we already have structure preserving maps in the standard library for Array, Optional, and ImplicitlyUnwrappedOptional.

It would be nice if Set and Dictionary also had a structure preserving map today (mapping over values in the dictionary leaving the keys untouched). I have seen people pick on Set a bit in regards to this because it can't conform the the usual functor laws so I’ll offer an example of using it there. Imagine we have a set of people and we want to know the set of dates where at least one person has a birthday (maybe to provide an indicator in a UI). Structure preserving map is exactly what we need and is quite useful even when the semantics are a bit less strict than the mathematical / categorical definition of a Functor.

I think it’s fair to say that types which *implement* the usual HKT typeclasses / protocols are extremely useful when used in a concrete setting. Even if you can’t express the protocol or conformance directly in the language the concepts and signatures are valuable as a pattern.

Of course in a thread about HKTs I’m sure you’re looking for examples where the code needs to be generic over the structure. The most common examples I can think of are “do notation” as well as various methods of effect handling and effect composition. Swift is clearly taking a different (and better IMO) approach to effects at the language level.

However it doesn’t (yet anyway) offer all of the control over effects that could be useful. This may be an area where HKTs would be useful although I don’t have a concrete example ready at hand. I have been thinking about and experimenting with some architectural ideas for managing (iOS) application state, data flow, event flow, etc using inspiration from reader, writer, state, and continuation monads. I may eventually find examples here, although it is also quite possible that generic use of HKTs won't really necessary. It’s hard to say for sure as I haven’t spent enough time on this yet.

So I suppose where I land is that I know the patterns that can only be expressed in a language by HKTs are very useful and practical. The Swift library already implements these patterns in some areas and they are widely useful. The library should be consistent in application of the HKT patterns it implements (such as structure preserving map).

Because it is possible (theoretically) to express the patterns directly in the language it makes sense to eventually be able to do that. If / when that happens all the library would need to do is declare the corresponding protocols and conformances to enable them to be used generically.

I have an intuition that practical uses for Swift programmers (especially library implementers) would emerge but are hard to find without experimentation and experience. Because we cannot actually use them today we don’t spend too much time thinking about how to apply them in our immediate context so it’s a bit of a catch-22. Of course this is a pretty weak argument which is why I am in strong agreement that this is not an immediate or high priority. But I do hope we have the chance to experiment and find practical uses for HKTs in Swift someday.

Matthew

···

On Dec 16, 2015, at 6:26 PM, Dave Abrahams <dabrahams@apple.com> wrote:

On Dec 16, 2015, at 4:04 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

With higher-kinded types, one could potentially [*] produce a collection of the same kind as Self but with the element type T. So, mapping a Set produces a Set, mapping an Array produces an Array, etc.

I think structure preserving map is a great example and one I hope will eventually make it into the standard library someday.

How, specifically, would you use it? To which types would it apply, and how would the change benefit users?

I'm not sure I follow? Are you suggesting that having HKTs at all will encourage Swift programmers to use bad naming conventions? I don't think the two are related unless you're talking about people using a Monad library, in which case it will be the Monad library using and encouraging bad naming conventions, not HKTs themselves.

This is directed to the thread/discussion in general, not any specific
person: I think it was said before but it is probably worth
repeating: Whether the type system should allow HKT or not doesn't
necessarily have anything to do with the names and data structures of
the stdlib.

For example, a Mappable protocol (let's call it that rather
than Functor) is currently not even _possible_ to write, and I believe this
is the question to consider, i.e. whether Swift's type system should be
allowed to express types that are parameterized not only by concrete types
but also by parameterized types, for example if a return type could
be Self<T> or just Self.

It would be sad if the question about HKT turned into yet another fruitless
and meaningless functional vs imperative debate (see the talk *-Oriented
Programming by Graham Lee).

/Jens

···

On Thursday, December 17, 2015, Will Fancher via swift-evolution < swift-evolution@swift.org> wrote:

I'm not sure I follow? Are you suggesting that having HKTs at all will
encourage Swift programmers to use bad naming conventions? I don't think
the two are related unless you're talking about people using a Monad
library, in which case it will be the Monad library using and encouraging
bad naming conventions, not HKTs themselves.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <javascript:;>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
bitCycle AB | Smedjegatan 12 | 742 32 Östhammar | Sweden

Phone: +46-73-753 24 62
E-mail: jens@bitcycle.com

This is directed to the thread/discussion in general, not any specific person: I think it was said before but it is probably worth repeating: Whether the type system should allow HKT or not doesn't necessarily have anything to do with the names and data structures of the stdlib.

Well said! Could we steer the discussion in the right direction by identifying what exact features of type system are needed?

Some questions that come to mind:

1) Do we need ability to express covariance and contravariance of generic types?

2) Do we need a better mechanism for ad-hoc polymorphism, similar to Haskell's typeclasses?

3) How should type constraints (`where` clause) be extended?

Some of those questions are already discussed in other threads, so it’s mainly a matter of ensuring that the upcoming proposals are powerful enough.

Krzysztof

For example, a Mappable protocol (let's call it that rather than Functor) is currently not even _possible_ to write, and I believe this is the question to consider, i.e. whether Swift's type system should be allowed to express types that are parameterized not only by concrete types but also by parameterized types, for example if a return type could be Self<T> or just Self.

It would be sad if the question about HKT turned into yet another fruitless and meaningless functional vs imperative debate (see the talk *-Oriented Programming by Graham Lee).

/Jens

···

On Thursday, December 17, 2015, Will Fancher via swift-evolution <swift-evolution@swift.org> wrote:
I'm not sure I follow? Are you suggesting that having HKTs at all will encourage Swift programmers to use bad naming conventions? I don't think the two are related unless you're talking about people using a Monad library, in which case it will be the Monad library using and encouraging bad naming conventions, not HKTs themselves.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
bitCycle AB | Smedjegatan 12 | 742 32 Östhammar | Sweden

Phone: +46-73-753 24 62
E-mail: jens@bitcycle.com

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

There's a lot of passionate Swift people here, it's great to see, and I
think with some clarification we could reach a consensus.

Lots of interesting discussion here for different proposals:
* Supporting Monads, Functors in the Standard Library
* Renaming flatMap, map, etc. ...

I think the second would be really interesting as a separate discussion in
another proposal (ie. map/reduce is probably Term of Art
https://swift.org/documentation/api-design-guidelines.html\).

The current proposal is, to paraphrase:
* A more expressive type system through more flexible generics (HKT)

I want Swift and the Standard Library to be clean, consistent, expressive,
safe, and approachable. I believe that HKTs are a necessary step to achieve
all of these things.

Going back over the posts, for those talking about the proposal, this seems
to be the votes so far:

*Clear Votes for/against HKTs:*

   - Will Fancher +1
   - Al Skipp +1
   - Austin Zheng +1
   - Dave Abrahams -1
   - T.J. Usiyan +1
   - Greg Titus -1
   - Joe Groff +1
   - Jens Persson +1
   - Brent Royal-Gordon +1
   - Krzysztof Siejkowski + 1

*Neutral about HKTs (expressed opinions on needing concrete examples, or
FP):*

   - Matthew Johnson +0
   - Douglas Gregor +0
   - Andrey Tarantsov +0

Noteworthy is Dave and Douglas from Apple, who basically say that either
they need more examples of how this will benefit many users, or that they
don't think it will.

We have collected a few examples of where this would benefit. They are
mainly for generating DSLs, a DSL is by definition domain-specific, so it's
going to be niche. However, DSLs in general are applicable to most apps, so
perhaps there's something in that.

I think Krzysztof Siejkowski's three points are great, and we should look
at and support those other threads.

We could get a representative sample of Swift code from github. Then find
which of those benefit from higher kinded types:
* turning runtime bugs into compile-time errors
* reducing code-duplication, etc.

A good place to look is popular Swift libraries (top 5 on github), and see
if they (and by extension their users) would benefit from HKTs:

   - GitHub - Alamofire/Alamofire: Elegant HTTP Networking in Swift (possibly, DSL)
   - GitHub - SwiftyJSON/SwiftyJSON: The better way to deal with JSON data in Swift. (very likely, DSL / type
   constraints)
   - GitHub - SnapKit/SnapKit: A Swift Autolayout DSL for iOS & OS X (very likely, chained DSL)
   - GitHub - MengTo/Spring: A library to simplify iOS animations in Swift. (probably not in their DSL, maybe in
   their implementation)
   - GitHub - ochococo/Design-Patterns-In-Swift: 📖 Design Patterns implemented in Swift 5.0 (probably, but
   I'm biased)

Likewise there's two more cases:

   - GitHub - Quick/Quick: The Swift (and Objective-C) testing framework.
   - GitHub - Quick/Nimble: A Matcher Framework for Swift and Objective-C

They're testing frameworks, their implementation DSL could really benefit
from HKTs last time I looked at it. Likewise, I'm not 100% sure but HKTs
may be the building blocks upon which Swift's first convenient mocking
framework could be built. Testing, and testing DSLs are things that all
swift users can benefit from, the question is whether that needs HKTs.

···

On Fri, Dec 18, 2015 at 1:43 AM, Krzysztof Siejkowski via swift-evolution < swift-evolution@swift.org> wrote:

This is directed to the thread/discussion in general, not any specific
person: I think it was said before but it is probably worth
repeating: Whether the type system should allow HKT or not doesn't
necessarily have anything to do with the names and data structures of
the stdlib.

Well said! Could we steer the discussion in the right direction by
identifying what exact features of type system are needed?

Some questions that come to mind:

1) Do we need ability to express covariance and contravariance of generic
types?

2) Do we need a better mechanism for ad-hoc polymorphism, similar to
Haskell's typeclasses?

3) How should type constraints (`where` clause) be extended?

Some of those questions are already discussed in other threads, so it’s
mainly a matter of ensuring that the upcoming proposals are powerful enough.

Krzysztof

For example, a Mappable protocol (let's call it that rather
than Functor) is currently not even _possible_ to write, and I believe this
is the question to consider, i.e. whether Swift's type system should be
allowed to express types that are parameterized not only by concrete types
but also by parameterized types, for example if a return type could
be Self<T> or just Self.

It would be sad if the question about HKT turned into yet
another fruitless and meaningless functional vs imperative debate (see the
talk *-Oriented Programming by Graham Lee).

/Jens

On Thursday, December 17, 2015, Will Fancher via swift-evolution < > swift-evolution@swift.org> wrote:

I'm not sure I follow? Are you suggesting that having HKTs at all will
encourage Swift programmers to use bad naming conventions? I don't think
the two are related unless you're talking about people using a Monad
library, in which case it will be the Monad library using and encouraging
bad naming conventions, not HKTs themselves.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
bitCycle AB | Smedjegatan 12 | 742 32 Östhammar | Sweden
http://www.bitcycle.com/
Phone: +46-73-753 24 62
E-mail: jens@bitcycle.com

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Did anything become of this…. This seemed to die out with no proposal.

I would think having a common protocol for types that support monadic operations (regardless if they fulfill all the monadic operations) would generally be helpful — even if they were not implemented theoretically as a Functor / Monad separate protocol definitions.

Scala seems to just treat them as one set of operations (MonadOps : map, flatMap, filter, withFilter). `filter` copies the contents into a new set, `withFilter` is just an on-demand filter for any later functions such as map/flatmap (BTW, Is swift filter on-demand, copy, or a combination???). I do notice that several different implement of the Monad protocol have been done in different repositories - by third parties.

And yes, Optionals are monads - and as such can be used in comprehension clauses.

I know that for-comprehension has been stated as probably not a Swift 3 thing (Felix indicated this but I don’t know where it is stated) - but I am wondering if they are thinking it is larger than it actually is (being that it is just a more friendly way to state existing map/flatmap combinations). If the worry is `for` having dual purpose — but if that is the case maybe making a new keyword for comprehensions like `all`.

Even if the proposal were drawn up - and it was `Deferred` at least it would potentially be on the radar.

···

On 2015-12-18, at 1:01:21, André Videla via swift-evolution <swift-evolution@swift.org> wrote:

I'm extremely excited about HKT. Huge +1 for me as well.

Moreover, In addition to lots of code reuse, this would allow to extend the language with very expressive and useful syntax for monadic types.
Monadic comprehension are very useful and generic.
We could imagine something the for-comprehension in scala and use it with list comprehension as well as parser combinators.

I sincerely hope this feature will see the day.

Sent from my iPod
On 2015/12/17, at 17:44, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Big +1 for HKTs from me!

-Thorsten

Am 17.12.2015 um 13:10 schrieb Will Fancher via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>:

Optional.map and Array.map do different things, and unifying them seems harmful.

In terms of category theory, they're not different. They are both arrows from a to b in the Functor category. Obviously that's horribly abstract, so I'll put it this way: If you think of these map functions not as operations on Optional and Array, and instead think of them simply as ways to compose an arbitrary data structure, they are in fact doing the same thing. On an implementation level, and on a runtime level, they perform differently. But on a type level, a theoretical level, and a categorical level, they are the same. You merely need to accept that when using this abstraction without knowledge of the concrete type, you have no reason to make any assumptions about how the implementation and runtime will behave. At that point, all you need is to be sure that the Functor being passed in abides by the Functor laws, and that's just a matter of convention.

First, I consider myself a smart person with a very broad experience with non-functional languages, but reading this makes my mind hurt, a lot:

   typeclass Functor f where
       fmap :: (a -> b) -> f a -> f b

<snip>

This makes it possible to build functions which operate not just on Maybe, but on any Functor.

   fstrlen :: Functor f => f String -> f Int
   fstrlen fstr = fmap length faster

<snip>

   protocol Functor {
       typealias A
       func fmap<FB where FB ~= Self>(f: A -> FB.A) -> FB
   }

I understand what's going on here, but I never, ever want to see this code anywhere near (my) Swift.

HKTs that come from category theory tend to have this effect. They are hard to understand by reading the protocol definition. Admittedly, this is a big flaw with Monads and with Haskell. It takes a reasonable understanding of category theory for the purpose of this code to be obvious to the reader.

(I will point out that in those code examples, I used absolutely abysmal naming conventions. It could be made marginally more readable and understandable if the naming were better)

I'll take this time to make the point that just because you don't like the way the Functor protocol looks, doesn't mean Array and Optional aren't both Functors and Monads. As a matter of mathematics, if the Monad functions can exist on a type, and they would follow the Monad laws, that type is a Monad whether the implementor likes it or not. Of course it still needs manual implementing, but regardless, the type is theoretically a Monad. (This realization is the reason that the Java team decided to implement flatMap for Java 8's Optional class.)

I believe the way to convince the rest of us (and I would love to be convinced) is to provide a few real, “end-user” app-level use cases and explain the benefit in plain language. In particular, I don't understand the example by Jens Persson, although it seems like a valuable one.

I can (again) link to just a few of the abstract Monadic functions, all of which are very useful in practical applications.

Control.Monad

But perhaps it would also help to describe the architectural decisions that abiding by Monad imposes.

Being Monadic makes you think about data composition, which cleans up the way you design your code. Going back to the Futures example, I could implement future, and subsequently use it, like this:

    public class Promise<T> {
        private var handlers: [T -> ()] =
        private var completed: T? = nil
        
        public init() {
        }
        
        private func onComplete(handler: T -> ()) {
            if let completed = completed {
                handler(completed)
            } else {
                handlers.append(handler)
            }
        }
        
        public func complete(t: T) {
            completed = t
            for handler in handlers {
                handler(t)
            }
            handlers =
        }
        
        public var future: Future<T> {
            return Future(promise: self)
        }
    }

    public struct Future<T> {
        private let promise: Promise<T>
        
        private init(promise: Promise<T>) {
            self.promise = promise
        }
        
        public func onComplete(handler: T -> ()) {
            promise.onComplete(handler)
        }
    }

    public func useFutures() {
        downloadURLInFuture().onComplete { content in
            processContentAsync(content).onComplete { processed in
                processed.calculateSomethingAsync().onComplete {
                    ...
                        ...
                            print(finalProduct)
                        ...
                    ...
                }
            }
        }
    }

You can see how this resembles the infamous Node.js issue of callback hell, and how the nesting could quickly get out of hand. If only we had some form of composition... Arrows between Futures... Luckily, Future is a Monad (whether I like it or not!)

    // Monad

    public extension Future {
        public static func point<T>(t: T) -> Future<T> {
            let promise = Promise<T>()
            promise.complete(t)
            return promise.future
        }
        
        public func flatMap<U>(f: T -> Future<U>) -> Future<U> {
            let uPromise = Promise<U>()
            
            onComplete { t in
                f(t).onComplete { u in
                    uPromise.complete(u)
                }
            }
            
            return uPromise.future
        }
    }

Not only do I now get map and apply for free, but I also get a great method of composing Futures. The example from above can now be rewritten to be much more well composed.

    public func useFutures() {
        downloadURLInFuture()
            .flatMap(processContentAsync)
            .flatMap { $0.calculateSomethingAsync() }
            ...
            .onComplete { finalProduct in
                print(finalProduct)
            }
    }

The important thing here is that thinking about Monads helped me discover a better composition model. Now my code can be more readable and well composed.

And again, I'll mention the enormous world of functions and capabilities that can be gotten for free for the sake of even more well-composed code.

I don't want a flatMap on optionals (which, I believe, was thankfully removed in Swift 2).

And why on Earth not? The concrete implementation of it on Optional is very simple and easy to understand, and the method is incredibly useful. Personally, I use it all the time (it was not removed). And again, just because you don't like using flatMap doesn't mean Optional isn't a Monad.

Finally, I'd like to point out that it doesn't hurt any end users not familiar with Monads if Array implements a higher kinded Monad protocol. As far as they have to be concerned, Array just has these useful map and flatMap functions. Meanwhile, those of us who wish to abstract over Monads in general can write our abstract code and implement Monad on our various types which mathematically have to be Monads. It's a layer of robustness and improvement that unconcerned end users don't have to bother themselves with.

While I understand that the Swift team doesn't have the resources to implement everything that everyone wants, and that HKTs aren't very high on their list of priorities, it seems unreasonable to me that one could think of HKTs and Monads as somehow detrimental to the language.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I’d just like to point out that I’ve been asking for concrete examples of how Swift programmers would benefit from higher-kinded types in their daily development, but I’ve received links to ICFP papers and lists of Haskell abstractions (Monad, Functor, Applicative) that only a type-theorist can understand. At best, this is a public-relations problem for higher-kinded types, because the vast majority of Swift developers—and also of members of this list—won’t be able to readily translate the content behind those links into “how does this improve my Swift code?” At worst, it’s a signal that this feature might not be of practical relevance to Swift, either because it solves problems Swift programmers don’t have or because it requires a thorough reading of TaPL to understand.

  - Doug

···

On Dec 16, 2015, at 2:40 PM, T.J. Usiyan <griotspeak@gmail.com> wrote:

The ideal situation would involve being able to represent monad and being able to type alias the behavior that that would afford us 'for free' to specific named functions methods for our specific types, I think.

The concrete example of a HKT's usefullness that I am after is captured here with [Causal Commutative Arrows](http://cs.yale.edu/c2/images/uploads/ICFP-CCA.pdf\). I admit that the big win has to do with the compile time optimization afforded, but I will argue that the syntax's similarity to audio related pseudocode is just as big of a win. [These slides](http://static1.squarespace.com/static/53e04d59e4b0c0da377d20b1/t/541cb6eee4b0be37af62e23f/1411167982496/nichoiceICFP2014Presentation.pdf\) give a hint at what I am talking about. I agree that using "Monad" everywhere is not generally descriptive but I think that that is simply an issue of being able to document intent.

I mentioned several examples in passing, but I suppose I could flesh one or two out.

Futures are a concept used in a lot of web development frameworks as a method of asynchronously representing a value that doesn't exist, but eventually will. Naturally, you represent this with the type Future<T>. You make a Promise<T> object, then pass around and use the associated future values. Here's a very quick example code.

    public class Promise<T> {
        private var handlers: [T -> ()] =
        private var completed: T? = nil
        
        public init() {
        }
        
        private func onComplete(handler: T -> ()) {
            if let completed = completed {
                handler(completed)
            } else {
                handlers.append(handler)
            }
        }
        
        public func complete(t: T) {
            completed = t
            for handler in handlers {
                handler(t)
            }
            handlers =
        }
        
        public var future: Future<T> {
            return Future(promise: self)
        }
    }

    public struct Future<T> {
        private let promise: Promise<T>
        
        private init(promise: Promise<T>) {
            self.promise = promise
        }
        
        public func onComplete(handler: T -> ()) {
            promise.onComplete(handler)
        }
    }

Using this, you can create a promise, return its future, complete that promise asynchronously, and be sure that any number of arbitrary handlers definitely get called with the completed value.

    let future: Future<SomeType> = someFutureCalculation()
    future.onComplete { someValue in
        doThing(someValue)
    }

But what about transforming those values outside of the handler, in the context where you received the future? This is where Monad, Applicative and Functor come in.

    // Monad

    public extension Future: Monad {
        public static func point<T>(t: T) -> Future<T> {
            let promise = Promise<T>()
            promise.complete(t)
            return promise.future
        }
        
        public func flatMap<U>(f: T -> Future<U>) -> Future<U> {
            let uPromise = Promise<U>()
            
            onComplete { t in
                f(t).onComplete { u in
                    uPromise.complete(u)
                }
            }
            
            return uPromise.future
        }
    }

Now, we can do interesting manipulations like this

    let future: Future<SomeType> = someFutureCalculation()
    future.flatMap { someValue in
        return otherFutureCalc(someValue)
    }.onComplete { otherValue in
        print(otherValue)
    }

That alone is marginally useful. What's really useful is that if we have the "free" definitions of Functor and Applicative I described in a previous message, we can use those methods by just having implemented the Monad protocol.

    let future: Future<String> = someFutureCalculation()
    future.map(stringToInt).flatMap { i in
        return otherFuture(i)
    }.apply(functionFuture).onComplete { val in
        print(val)
    }

There's a lot going on here, and it comes from free Functor and Applicative definitions that currently aren't possible.

Additionally, if I have been given an array of Futures, I don't have to re-implement sequence to turn that into a Future of an array. I just get this for free.

    let array: [Future<A>] = ...
    let farr: Future<[A]> = sequence(array)

That's super useful.

Having Monads also means that if you would write code which deals with Futures, you can instead write code which deals with Monads or Functors. So this

    func changeFuture(future: Future<String>) -> Future<Int> {
        return future.map { s in
            // create Int from s
        }
    }

can become this

    func changeSomething<FString: Functor, FInt where FInt ~= FString, FInt.FunctorType == Int>
    (functor: FString) -> FInt {
        return functor.map { s in
            // create Int from s
        }
    }

Now, it works with anything, not just Futures.

The point of HKTs isn't to make a specific task easier. The point is that it makes it possible for any types which conform to a HKT protocol to use all the same functions. Thus, you can write a host of functions that operate on HKTs instead of rewriting them for specialized types. In the case of futures, I'd like to remind you that this is all the code it took to make Futures capable of the countless Functor / Monad functions

    public extension Future: Monad {
        public static func point<T>(t: T) -> Future<T> {
            let promise = Promise<T>()
            promise.complete(t)
            return promise.future
        }
        
        public func flatMap<U>(f: T -> Future<U>) -> Future<U> {
            let uPromise = Promise<U>()
            
            onComplete { t in
                f(t).onComplete { u in
                    uPromise.complete(u)
                }
            }
            
            return uPromise.future
        }
    }

Those few lines open up a world of possibilities on the Future type so that I can avoid boilerplate and repetition.

The concrete example in those links is a DSL for audio related programming.
I thought that I presented the point in that paragraph but I will try to
summarize.

Using Arrows, which are a specific HKT, allows us to write audio code which
resembles the description of certain audio operations more closely than
typical procedural code. This, all by itself, is a big win. Certain
optimizations are proven safe if we use a controlled subset of arrows.
These optimizations allow the code–which tends to represent many loops–to
be rewritten as one loop accepting a ball of coalesced state. This
optimized code is exceptionally fast and provides the second win.

TLDR, Audio signal flow code benefits greatly from specific HKT.

···

On Wed, Dec 16, 2015 at 6:49 PM, Douglas Gregor <dgregor@apple.com> wrote:

On Dec 16, 2015, at 2:40 PM, T.J. Usiyan <griotspeak@gmail.com> wrote:

The ideal situation would involve being able to represent monad and being
able to type alias the behavior that that would afford us 'for free' to
specific named functions methods for our specific types, I think.

The concrete example of a HKT's usefullness that I am after is captured
here with [Causal Commutative Arrows](
http://cs.yale.edu/c2/images/uploads/ICFP-CCA.pdf\). I admit that the big
win has to do with the compile time optimization afforded, but I will argue
that the syntax's similarity to audio related pseudocode is just as big of
a win. [These slides](
http://static1.squarespace.com/static/53e04d59e4b0c0da377d20b1/t/541cb6eee4b0be37af62e23f/1411167982496/nichoiceICFP2014Presentation.pdf\)
give a hint at what I am talking about. I agree that using "Monad"
everywhere is not generally descriptive but I think that that is simply an
issue of being able to document intent.

I’d just like to point out that I’ve been asking for concrete examples of
how Swift programmers would benefit from higher-kinded types in their daily
development, but I’ve received links to ICFP papers and lists of Haskell
abstractions (Monad, Functor, Applicative) that only a type-theorist can
understand. At best, this is a public-relations problem for higher-kinded
types, because the vast majority of Swift developers—and also of members of
this list—won’t be able to readily translate the content behind those links
into “how does this improve my Swift code?” At worst, it’s a signal that
this feature might not be of practical relevance to Swift, either because
it solves problems Swift programmers don’t have or because it requires a
thorough reading of TaPL to understand.

- Doug

Do these papers contain examples of generic use of HKTs that wouldn’t be possible with a little bit of copy and paste boilerplate? Or do they mostly contain use of concrete types that implement some typeclasses?

···

On Dec 16, 2015, at 4:40 PM, T.J. Usiyan via swift-evolution <swift-evolution@swift.org> wrote:

The ideal situation would involve being able to represent monad and being able to type alias the behavior that that would afford us 'for free' to specific named functions methods for our specific types, I think.

The concrete example of a HKT's usefullness that I am after is captured here with [Causal Commutative Arrows](http://cs.yale.edu/c2/images/uploads/ICFP-CCA.pdf\). I admit that the big win has to do with the compile time optimization afforded, but I will argue that the syntax's similarity to audio related pseudocode is just as big of a win. [These slides](http://static1.squarespace.com/static/53e04d59e4b0c0da377d20b1/t/541cb6eee4b0be37af62e23f/1411167982496/nichoiceICFP2014Presentation.pdf\) give a hint at what I am talking about. I agree that using "Monad" everywhere is not generally descriptive but I think that that is simply an issue of being able to document intent.

TJ

On Wed, Dec 16, 2015 at 5:11 PM, Will Fancher via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Admittedly, the majority of my reasoning for higher kinded types comes from wanting Monads to be an available abstraction. I've been writing a lot of generic code, and finding that a lot of my types are Monads, which leads me to repeat many functions for each of these types, when those functions could be abstracted.

The fact is, most generic types can be Functors, and a lot of them can be Monads. Therefore, having generics at all suggests that the language will have a lot of these kinds of types. The standard library doesn't have many. Optionals and the various collection protocols make up the majority of them. So developers who can rely mostly on the standard library using concrete types probably have everything they need in this area. But creating one's own generic types will very often lead to reimplementing the same Monad / Applicative / Functor utility functions. I can't tell you how many times I've copy-pasted my sequence function and changed only the type signature. Futures, Eithers, and Streams are three examples of types I've been implementing as Monads, which has lead to a lot of repetition.

It is my personal opinion that Monads come up often enough that having proper support for them would be worthwhile. But I suppose the average Swift programmer may not run into these things as often as I do, so it makes sense that HKTs are low on the Swift team's priorities.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Looking beyond functional programming abstractions, higher-kinded types are a fancy way of saying "template template parameters”.

Thanks for bringing that up. I almost mentioned it. I didn’t because I know the Swift team wants to avoid a lot of what has been done with templates and am not sure exactly where you draw the line. :)

···

A textbook motivation for those from C++ land is Andrei Alexandrescu's "policy pattern", stuff like this:

protocol RefStoragePolicy: <*: class> {
typealias Ref: class
init(ref: Ref)
var ref: Ref
}

struct Weak<T: class>: RefStoragePolicy { weak var ref: T }
struct Unowned<T: class>: RefStoragePolicy { unowned var ref: T }
struct Strong<T: class>: RefStoragePolicy { var ref: T }

class HeterogeneousRefConsumer<Storage: RefStoragePolicy> {
func consumeRef<T: class>(ref: T) {
   let storage = Storage<T>(ref: ref)
   doStuffWith(storage)
}
}

-Joe
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I am actually +1 on adding HKTs eventually. I would really like to see them in Swift.

I also think fleshing out the existing generics features is a much more immediate concern. It's pretty clear we won't see every generics feature we can imagine in Swift 3 so we need to choose. I would prefer to see the existing features fully fleshed out and limitations removed before we start introducing more sophisticated features into the type system.

Collecting a list of libraries you think might benefit from HKTs is a good start for providing more concrete motivation in the context of Swift. I think a good next step would be to imagine how those libraries might look different if Swift had HKTs. The API should ideally be comfortable for users whether they understand HKTs or not (or even know they exist).

How would the implementations be better and the interfaces more composable and / or safer? For the DSL examples, how would the HKT approach be superior to other approaches? Answering these questions with hypothetical but concrete examples will do a lot to change the conversation and maybe also increase the priority.

Matthew

···

Sent from my iPad

On Dec 17, 2015, at 9:40 AM, Andrew Bennett via swift-evolution <swift-evolution@swift.org> wrote:

There's a lot of passionate Swift people here, it's great to see, and I think with some clarification we could reach a consensus.

Lots of interesting discussion here for different proposals:
* Supporting Monads, Functors in the Standard Library
* Renaming flatMap, map, etc. ...

I think the second would be really interesting as a separate discussion in another proposal (ie. map/reduce is probably Term of Art https://swift.org/documentation/api-design-guidelines.html\).

The current proposal is, to paraphrase:
* A more expressive type system through more flexible generics (HKT)

I want Swift and the Standard Library to be clean, consistent, expressive, safe, and approachable. I believe that HKTs are a necessary step to achieve all of these things.

Going back over the posts, for those talking about the proposal, this seems to be the votes so far:

Clear Votes for/against HKTs:
Will Fancher +1
Al Skipp +1
Austin Zheng +1
Dave Abrahams -1
T.J. Usiyan +1
Greg Titus -1
Joe Groff +1
Jens Persson +1
Brent Royal-Gordon +1
Krzysztof Siejkowski + 1
Neutral about HKTs (expressed opinions on needing concrete examples, or FP):
Matthew Johnson +0
Douglas Gregor +0
Andrey Tarantsov +0
Noteworthy is Dave and Douglas from Apple, who basically say that either they need more examples of how this will benefit many users, or that they don't think it will.

We have collected a few examples of where this would benefit. They are mainly for generating DSLs, a DSL is by definition domain-specific, so it's going to be niche. However, DSLs in general are applicable to most apps, so perhaps there's something in that.

I think Krzysztof Siejkowski's three points are great, and we should look at and support those other threads.

We could get a representative sample of Swift code from github. Then find which of those benefit from higher kinded types:
* turning runtime bugs into compile-time errors
* reducing code-duplication, etc.

A good place to look is popular Swift libraries (top 5 on github), and see if they (and by extension their users) would benefit from HKTs:
GitHub - Alamofire/Alamofire: Elegant HTTP Networking in Swift (possibly, DSL)
GitHub - SwiftyJSON/SwiftyJSON: The better way to deal with JSON data in Swift. (very likely, DSL / type constraints)
GitHub - SnapKit/SnapKit: A Swift Autolayout DSL for iOS & OS X (very likely, chained DSL)
GitHub - MengTo/Spring: A library to simplify iOS animations in Swift. (probably not in their DSL, maybe in their implementation)
GitHub - ochococo/Design-Patterns-In-Swift: 📖 Design Patterns implemented in Swift 5.0 (probably, but I'm biased)
Likewise there's two more cases:
GitHub - Quick/Quick: The Swift (and Objective-C) testing framework.
GitHub - Quick/Nimble: A Matcher Framework for Swift and Objective-C
They're testing frameworks, their implementation DSL could really benefit from HKTs last time I looked at it. Likewise, I'm not 100% sure but HKTs may be the building blocks upon which Swift's first convenient mocking framework could be built. Testing, and testing DSLs are things that all swift users can benefit from, the question is whether that needs HKTs.

On Fri, Dec 18, 2015 at 1:43 AM, Krzysztof Siejkowski via swift-evolution <swift-evolution@swift.org> wrote:

This is directed to the thread/discussion in general, not any specific person: I think it was said before but it is probably worth repeating: Whether the type system should allow HKT or not doesn't necessarily have anything to do with the names and data structures of the stdlib.

Well said! Could we steer the discussion in the right direction by identifying what exact features of type system are needed?

Some questions that come to mind:

1) Do we need ability to express covariance and contravariance of generic types?

2) Do we need a better mechanism for ad-hoc polymorphism, similar to Haskell's typeclasses?

3) How should type constraints (`where` clause) be extended?

Some of those questions are already discussed in other threads, so it’s mainly a matter of ensuring that the upcoming proposals are powerful enough.

Krzysztof

For example, a Mappable protocol (let's call it that rather than Functor) is currently not even _possible_ to write, and I believe this is the question to consider, i.e. whether Swift's type system should be allowed to express types that are parameterized not only by concrete types but also by parameterized types, for example if a return type could be Self<T> or just Self.

It would be sad if the question about HKT turned into yet another fruitless and meaningless functional vs imperative debate (see the talk *-Oriented Programming by Graham Lee).

/Jens

On Thursday, December 17, 2015, Will Fancher via swift-evolution <swift-evolution@swift.org> wrote:
I'm not sure I follow? Are you suggesting that having HKTs at all will encourage Swift programmers to use bad naming conventions? I don't think the two are related unless you're talking about people using a Monad library, in which case it will be the Monad library using and encouraging bad naming conventions, not HKTs themselves.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
bitCycle AB | Smedjegatan 12 | 742 32 Östhammar | Sweden
http://www.bitcycle.com/
Phone: +46-73-753 24 62
E-mail: jens@bitcycle.com

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

If the Swift core team decides that a feature is out of scope for Swift 3, the proposal will be deferred and will not come up for a public review. Discussion, design, and even out-of-tree implementation can proceed, but you can expect that the core team won’t be deeply involved because they have to focus on making Swift 3 happen.

One natural question that pops out of this is, “what if the feature was effectively ‘perfect’, with a detailed design proposal, complete implementation, updated standard library, examples showing how other external projects could benefit from this feature, etc?” Even then, the time it takes to carefully review a design—especially one as wide-ranging as higher-kinded types—for its interactions with the rest of the language, runtime, standard library, and Swift ecosystem is significant, and we’ve found that in many cases our time is more constrained by design review than it is even by implementation concerns.

  - Doug

···

On Dec 16, 2015, at 5:18 PM, Will Fancher <willfancher38@gmail.com> wrote:

while I agree that this kind of constraint is helpful and important for completing the generics picture, IMO it is probably less helpful/important than many people think.

While I would love to see HKTs in Swift as soon as possible, and I personally would really appreciate a good Monad setup, I have to say I agree with this. It'd be great to have, but it's not as important as anything else on Swift 3's todo list.

So this brings me to a question: If the community were to write HKTs into the language and submit a pull request, what is the likelihood that it would be accepted, despite the fact that the Swift team feels it is out of scope for their own Swift 3 efforts?

I am in favor of HKT eventually, but agree that they are lower priority for
now. I do think some of these kinds of abstractions and thinking will be
more fluent for the next generation of programmers, but Swift has already
harvested the major wins in that arena by embracing enough of the power
from functional programming styles to allow us significant productivity and
expressivity wins. HKT allows more code sharing but does not prevent us
from making the practical types we need, like Future/Promise (
GitHub - bignerdranch/Deferred: Work with values that haven't been determined yet. for a in-the-wild implementation
that I’m involved with).

Step Christopher
Big Nerd Ranch, LLC
schristopher@bignerdranch.com

···

On Thu, Dec 17, 2015 at 10:40 AM, Andrew Bennett via swift-evolution < swift-evolution@swift.org> wrote:

There's a lot of passionate Swift people here, it's great to see, and I
think with some clarification we could reach a consensus.

Lots of interesting discussion here for different proposals:
* Supporting Monads, Functors in the Standard Library
* Renaming flatMap, map, etc. ...

I think the second would be really interesting as a separate discussion in
another proposal (ie. map/reduce is probably Term of Art
https://swift.org/documentation/api-design-guidelines.html\).

The current proposal is, to paraphrase:
* A more expressive type system through more flexible generics (HKT)

I want Swift and the Standard Library to be clean, consistent, expressive,
safe, and approachable. I believe that HKTs are a necessary step to achieve
all of these things.

Going back over the posts, for those talking about the proposal, this
seems to be the votes so far:

*Clear Votes for/against HKTs:*

   - Will Fancher +1
   - Al Skipp +1
   - Austin Zheng +1
   - Dave Abrahams -1
   - T.J. Usiyan +1
   - Greg Titus -1
   - Joe Groff +1
   - Jens Persson +1
   - Brent Royal-Gordon +1
   - Krzysztof Siejkowski + 1

*Neutral about HKTs (expressed opinions on needing concrete examples, or
FP):*

   - Matthew Johnson +0
   - Douglas Gregor +0
   - Andrey Tarantsov +0

Noteworthy is Dave and Douglas from Apple, who basically say that either
they need more examples of how this will benefit many users, or that they
don't think it will.

We have collected a few examples of where this would benefit. They are
mainly for generating DSLs, a DSL is by definition domain-specific, so it's
going to be niche. However, DSLs in general are applicable to most apps, so
perhaps there's something in that.

I think Krzysztof Siejkowski's three points are great, and we should look
at and support those other threads.

We could get a representative sample of Swift code from github. Then find
which of those benefit from higher kinded types:
* turning runtime bugs into compile-time errors
* reducing code-duplication, etc.

A good place to look is popular Swift libraries (top 5 on github), and see
if they (and by extension their users) would benefit from HKTs:

   - GitHub - Alamofire/Alamofire: Elegant HTTP Networking in Swift (possibly, DSL)
   - GitHub - SwiftyJSON/SwiftyJSON: The better way to deal with JSON data in Swift. (very likely, DSL / type
   constraints)
   - GitHub - SnapKit/SnapKit: A Swift Autolayout DSL for iOS & OS X (very likely, chained DSL)
   - GitHub - MengTo/Spring: A library to simplify iOS animations in Swift. (probably not in their DSL, maybe
   in their implementation)
   - GitHub - ochococo/Design-Patterns-In-Swift: 📖 Design Patterns implemented in Swift 5.0 (probably, but
   I'm biased)

Likewise there's two more cases:

   - GitHub - Quick/Quick: The Swift (and Objective-C) testing framework.
   - GitHub - Quick/Nimble: A Matcher Framework for Swift and Objective-C

They're testing frameworks, their implementation DSL could really benefit
from HKTs last time I looked at it. Likewise, I'm not 100% sure but HKTs
may be the building blocks upon which Swift's first convenient mocking
framework could be built. Testing, and testing DSLs are things that all
swift users can benefit from, the question is whether that needs HKTs.

On Fri, Dec 18, 2015 at 1:43 AM, Krzysztof Siejkowski via swift-evolution > <swift-evolution@swift.org> wrote:

This is directed to the thread/discussion in general, not any specific
person: I think it was said before but it is probably worth
repeating: Whether the type system should allow HKT or not doesn't
necessarily have anything to do with the names and data structures of
the stdlib.

Well said! Could we steer the discussion in the right direction by
identifying what exact features of type system are needed?

Some questions that come to mind:

1) Do we need ability to express covariance and contravariance of generic
types?

2) Do we need a better mechanism for ad-hoc polymorphism, similar to
Haskell's typeclasses?

3) How should type constraints (`where` clause) be extended?

Some of those questions are already discussed in other threads, so it’s
mainly a matter of ensuring that the upcoming proposals are powerful enough.

Krzysztof

For example, a Mappable protocol (let's call it that rather
than Functor) is currently not even _possible_ to write, and I believe this
is the question to consider, i.e. whether Swift's type system should be
allowed to express types that are parameterized not only by concrete types
but also by parameterized types, for example if a return type could
be Self<T> or just Self.

It would be sad if the question about HKT turned into yet
another fruitless and meaningless functional vs imperative debate (see the
talk *-Oriented Programming by Graham Lee).

/Jens

On Thursday, December 17, 2015, Will Fancher via swift-evolution < >> swift-evolution@swift.org> wrote:

I'm not sure I follow? Are you suggesting that having HKTs at all will
encourage Swift programmers to use bad naming conventions? I don't think
the two are related unless you're talking about people using a Monad
library, in which case it will be the Monad library using and encouraging
bad naming conventions, not HKTs themselves.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
bitCycle AB | Smedjegatan 12 | 742 32 Östhammar | Sweden
http://www.bitcycle.com/
Phone: +46-73-753 24 62
E-mail: jens@bitcycle.com

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I'm +0 as well, to be clear. I only wanted to steer the discussion away from functional abstractions and toward more concrete applications of HKT that come up in practice in industrial languages like C++.

-Joe

···

On Dec 17, 2015, at 7:40 AM, Andrew Bennett via swift-evolution <swift-evolution@swift.org> wrote:

Joe Groff +1

I feel that people voting +1 here all have some specific use case in mind. But unlike in other proposals, here, for the love of god, I cannot imagine what it is.

  • Brent Royal-Gordon +1

Can each of you guys just write a bit of code that you want to be made possible, something that is substantially better than alternatives?

Really, it may seem obvious to you, but it's a complete mystery to me and others like me.

Well, for one thing, I’ve already earned 605 reputation on Stack Overflow because of this particular weakness in the type system. I’d like that to stop. <What's the cleanest way of applying map() to a dictionary in Swift? - Stack Overflow;

···

--
Brent Royal-Gordon
Architechies

An excellent summary, Andrew.

I feel that people voting +1 here all have some specific use case in mind. But unlike in other proposals, here, for the love of god, I cannot imagine what it is.

Will Fancher +1
Al Skipp +1
Austin Zheng +1
T.J. Usiyan +1
Joe Groff +1
Jens Persson +1
Brent Royal-Gordon +1
Krzysztof Siejkowski + 1

Can each of you guys just write a bit of code that you want to be made possible, something that is substantially better than alternatives?

Really, it may seem obvious to you, but it's a complete mystery to me and others like me.

A good place to look is popular Swift libraries (top 5 on github), and see if they (and by extension their users) would benefit from HKTs:
GitHub - Alamofire/Alamofire: Elegant HTTP Networking in Swift (possibly, DSL)
GitHub - SwiftyJSON/SwiftyJSON: The better way to deal with JSON data in Swift. (very likely, DSL / type constraints)
GitHub - SnapKit/SnapKit: A Swift Autolayout DSL for iOS & OS X (very likely, chained DSL)
GitHub - MengTo/Spring: A library to simplify iOS animations in Swift. (probably not in their DSL, maybe in their implementation)
GitHub - ochococo/Design-Patterns-In-Swift: 📖 Design Patterns implemented in Swift 5.0 (probably, but I'm biased)
Likewise there's two more cases:
GitHub - Quick/Quick: The Swift (and Objective-C) testing framework.
GitHub - Quick/Nimble: A Matcher Framework for Swift and Objective-C

Would *love* this, DSLs especially.

Common, don't just vote +1 — give examples!

A.

Sorry Joe, I misinterpreted your position - I may have interpreted
supportive of the discourse with supportive of the proposal. Also sorry I
left you off the Apple people list!

···

On Friday, 18 December 2015, Joe Groff <jgroff@apple.com> wrote:

On Dec 17, 2015, at 7:40 AM, Andrew Bennett via swift-evolution < > swift-evolution@swift.org > <javascript:_e(%7B%7D,'cvml','swift-evolution@swift.org');>> wrote:

   - Joe Groff +1

I'm +0 as well, to be clear. I only wanted to steer the discussion away
from functional abstractions and toward more concrete applications of HKT
that come up in practice in industrial languages like C++.

-Joe

It never died, the proposal draft is still published for you to comment on before we submit it. We'd like to have at least a partial implementation to go along with the change because it is so massive, so it's going to take longer than your average proposal:

~Robert Widmann

2016/01/21 10:36、Craig Cruden via swift-evolution <swift-evolution@swift.org> のメッセージ:

···

Did anything become of this…. This seemed to die out with no proposal.

I would think having a common protocol for types that support monadic operations (regardless if they fulfill all the monadic operations) would generally be helpful — even if they were not implemented theoretically as a Functor / Monad separate protocol definitions.

Scala seems to just treat them as one set of operations (MonadOps : map, flatMap, filter, withFilter). `filter` copies the contents into a new set, `withFilter` is just an on-demand filter for any later functions such as map/flatmap (BTW, Is swift filter on-demand, copy, or a combination???). I do notice that several different implement of the Monad protocol have been done in different repositories - by third parties.

And yes, Optionals are monads - and as such can be used in comprehension clauses.

I know that for-comprehension has been stated as probably not a Swift 3 thing (Felix indicated this but I don’t know where it is stated) - but I am wondering if they are thinking it is larger than it actually is (being that it is just a more friendly way to state existing map/flatmap combinations). If the worry is `for` having dual purpose — but if that is the case maybe making a new keyword for comprehensions like `all`.

Even if the proposal were drawn up - and it was `Deferred` at least it would potentially be on the radar.

On 2015-12-18, at 1:01:21, André Videla via swift-evolution <swift-evolution@swift.org> wrote:

I'm extremely excited about HKT. Huge +1 for me as well.

Moreover, In addition to lots of code reuse, this would allow to extend the language with very expressive and useful syntax for monadic types.
Monadic comprehension are very useful and generic.
We could imagine something the for-comprehension in scala and use it with list comprehension as well as parser combinators.

I sincerely hope this feature will see the day.

Sent from my iPod
On 2015/12/17, at 17:44, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:

Big +1 for HKTs from me!

-Thorsten

Am 17.12.2015 um 13:10 schrieb Will Fancher via swift-evolution <swift-evolution@swift.org>:

Optional.map and Array.map do different things, and unifying them seems harmful.

In terms of category theory, they're not different. They are both arrows from a to b in the Functor category. Obviously that's horribly abstract, so I'll put it this way: If you think of these map functions not as operations on Optional and Array, and instead think of them simply as ways to compose an arbitrary data structure, they are in fact doing the same thing. On an implementation level, and on a runtime level, they perform differently. But on a type level, a theoretical level, and a categorical level, they are the same. You merely need to accept that when using this abstraction without knowledge of the concrete type, you have no reason to make any assumptions about how the implementation and runtime will behave. At that point, all you need is to be sure that the Functor being passed in abides by the Functor laws, and that's just a matter of convention.

First, I consider myself a smart person with a very broad experience with non-functional languages, but reading this makes my mind hurt, a lot:

   typeclass Functor f where
       fmap :: (a -> b) -> f a -> f b

<snip>

This makes it possible to build functions which operate not just on Maybe, but on any Functor.

   fstrlen :: Functor f => f String -> f Int
   fstrlen fstr = fmap length faster

<snip>

   protocol Functor {
       typealias A
       func fmap<FB where FB ~= Self>(f: A -> FB.A) -> FB
   }

I understand what's going on here, but I never, ever want to see this code anywhere near (my) Swift.

HKTs that come from category theory tend to have this effect. They are hard to understand by reading the protocol definition. Admittedly, this is a big flaw with Monads and with Haskell. It takes a reasonable understanding of category theory for the purpose of this code to be obvious to the reader.

(I will point out that in those code examples, I used absolutely abysmal naming conventions. It could be made marginally more readable and understandable if the naming were better)

I'll take this time to make the point that just because you don't like the way the Functor protocol looks, doesn't mean Array and Optional aren't both Functors and Monads. As a matter of mathematics, if the Monad functions can exist on a type, and they would follow the Monad laws, that type is a Monad whether the implementor likes it or not. Of course it still needs manual implementing, but regardless, the type is theoretically a Monad. (This realization is the reason that the Java team decided to implement flatMap for Java 8's Optional class.)

I believe the way to convince the rest of us (and I would love to be convinced) is to provide a few real, “end-user” app-level use cases and explain the benefit in plain language. In particular, I don't understand the example by Jens Persson, although it seems like a valuable one.

I can (again) link to just a few of the abstract Monadic functions, all of which are very useful in practical applications.

Control.Monad

But perhaps it would also help to describe the architectural decisions that abiding by Monad imposes.

Being Monadic makes you think about data composition, which cleans up the way you design your code. Going back to the Futures example, I could implement future, and subsequently use it, like this:

    public class Promise<T> {
        private var handlers: [T -> ()] =
        private var completed: T? = nil
        
        public init() {
        }
        
        private func onComplete(handler: T -> ()) {
            if let completed = completed {
                handler(completed)
            } else {
                handlers.append(handler)
            }
        }
        
        public func complete(t: T) {
            completed = t
            for handler in handlers {
                handler(t)
            }
            handlers =
        }
        
        public var future: Future<T> {
            return Future(promise: self)
        }
    }

    public struct Future<T> {
        private let promise: Promise<T>
        
        private init(promise: Promise<T>) {
            self.promise = promise
        }
        
        public func onComplete(handler: T -> ()) {
            promise.onComplete(handler)
        }
    }

    public func useFutures() {
        downloadURLInFuture().onComplete { content in
            processContentAsync(content).onComplete { processed in
                processed.calculateSomethingAsync().onComplete {
                    ...
                        ...
                            print(finalProduct)
                        ...
                    ...
                }
            }
        }
    }

You can see how this resembles the infamous Node.js issue of callback hell, and how the nesting could quickly get out of hand. If only we had some form of composition... Arrows between Futures... Luckily, Future is a Monad (whether I like it or not!)

    // Monad

    public extension Future {
        public static func point<T>(t: T) -> Future<T> {
            let promise = Promise<T>()
            promise.complete(t)
            return promise.future
        }
        
        public func flatMap<U>(f: T -> Future<U>) -> Future<U> {
            let uPromise = Promise<U>()
            
            onComplete { t in
                f(t).onComplete { u in
                    uPromise.complete(u)
                }
            }
            
            return uPromise.future
        }
    }

Not only do I now get map and apply for free, but I also get a great method of composing Futures. The example from above can now be rewritten to be much more well composed.

    public func useFutures() {
        downloadURLInFuture()
            .flatMap(processContentAsync)
            .flatMap { $0.calculateSomethingAsync() }
            ...
            .onComplete { finalProduct in
                print(finalProduct)
            }
    }

The important thing here is that thinking about Monads helped me discover a better composition model. Now my code can be more readable and well composed.

And again, I'll mention the enormous world of functions and capabilities that can be gotten for free for the sake of even more well-composed code.

I don't want a flatMap on optionals (which, I believe, was thankfully removed in Swift 2).

And why on Earth not? The concrete implementation of it on Optional is very simple and easy to understand, and the method is incredibly useful. Personally, I use it all the time (it was not removed). And again, just because you don't like using flatMap doesn't mean Optional isn't a Monad.

Finally, I'd like to point out that it doesn't hurt any end users not familiar with Monads if Array implements a higher kinded Monad protocol. As far as they have to be concerned, Array just has these useful map and flatMap functions. Meanwhile, those of us who wish to abstract over Monads in general can write our abstract code and implement Monad on our various types which mathematically have to be Monads. It's a layer of robustness and improvement that unconcerned end users don't have to bother themselves with.

While I understand that the Swift team doesn't have the resources to implement everything that everyone wants, and that HKTs aren't very high on their list of priorities, it seems unreasonable to me that one could think of HKTs and Monads as somehow detrimental to the language.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution