Support type classes or implicits #243
Comments
dsyme
changed the title
|
For those who haven't seen it, there's an experimental implementation of type classes for F# Hopefully if this is implemented as a language feature the verbose attribute syntax will be dropped in favor of proper keywords. It's hard to see how [<Trait>]
type Eq<'A> =
abstract equal: 'A -> 'A -> bool
[<Witness>] // a.k.a instance
type EqInt =
interface Eq<int> with
member equal a b = a = bpresents any advantage over a more terse syntax like trait Eq<'A> =
abstract equal: 'A -> 'A -> bool
witness EqInt of Eq<int> =
member equal a b = a = bwhich provides both brevity and clarity |
|
@cloudRoutine There are advantages of a kind.
Note that the prototype has some significant limitations, specifically that if explicit instantiation of the witnesses is allowed, then the "dictionary" being passed can't easily "close" over any values - as envisaged it must be passed by passing |
|
It seems I misunderstood how this feature would work. I'd thought that a If an interface with the When an attribute needs to be used with a construct in all cases, isn't it effectively the same (from the programmer's perspective) as a top level declaration with extra boilerplate? I suppose we'll have to rely on tooling to deal with the boilerplate 😞 |
Off the top of my head there's no reason. We should look at the prototype though (which I felt was in-depth enough to determine questions like this) |
I don't understand why this would be useful to allow - assuming you mean something like: [<Witness>] // a.k.a instance
type EqInt(i:int) =
interface Eq<int> with
member equal a b = a = b
member __.TheInt = iPerhaps I'm missing something but allowing that looks very confusing to me. |
|
I don't mind the attribute style at all. I'm all for keeping the number of keywords in F# as low as possible and building more and more one existing constructs in this manner, avoiding keyword salad. I do rather like the jargon Rust uses for its type classes (trait and impl) though as I think it's more accessible to normal programmers, witness only makes intuitive sense to people in theorem proving circles, but I'm not super pushing for that to change here just noting my opinion. One note: because these are written in terms of types it seems like they could never be extended to support statically resolved type parameters. Am I correct? |
|
How would you support stuff like functor? [<Trait>]
type Functor<'f> =
abstract fmap: ??? |
|
@Rickasaurus can you explain how the example I posted, or one similar to it, creates a "keyword salad"? I don't follow the point you're trying to make. The keyword is already reserved, so it's not like we can use it for ourselves.
I can't follow what you mean here either, can you give an example of what you'd like to do? @Alxandr it can't be supported because we don't have type constructors |
|
Re explicit witnesses that close over values
e.g. dependency injection (i.e. parameterization of a witness over some dependency): [<Witness>] // a.k.a instance
type SomeWitness(someDependency: int->int) =
interface SomeTypeConstraint<int> with
member SomeOperation a b = someDependency a + someDependency b
... SomeConstrainedOperation(SomeWitness(myDependency),...) ...or The utility of this depends on the degree to which you use witnesses to propagate sets of functions which have a non-trivial dependency base. My understanding is that Scala implicits allow this technique. For example, witnesses propagated by implicits may capture a meta programming universe, which is a value. |
|
I still think that not figuring out how to deal with type constructors will severely limit the usefulness of this proposal. Type classes without type constructors would allow for doing abstractions over numerical types, sure, but the lack of ability to do a generic |
|
@dsyme I see, thanks for the explanation. I don't have extensive experience with Scala implicits. While allowing bringing witness values into scope is more powerful, you also lose ability to reason about code. Here is the argument in more detail in case anyone is interested: https://www.youtube.com/watch?v=hIZxTQP1ifo Disallow explicit instantiation of witnesses also means we can make them stateless structs and defaultof always works. It does mean we would have some shenanigans like having to add wrapper types if one type can be a witness of a trait in more than one way (e.g. 'Monoid' and 'int', via either '+' or '*') but it looks to me like that is the vast minority of cases. We then also have to think about coherence and orphan instances, e.g if there is more than one possible witness is in scope, warn or error, and have some explicit mechanism to disambiguate. even though this somewhat goes against the current F# practice of resolving names to the first thing in scope. Perhaps it would be enough to disallow orphans (i.e. defining a witness in a compilation unit without also declaring either the trait or the type), which would also cover pretty much all use cases I expect. @Alxandr What you're asking for are higher-kinded types. The feature request for that is here. I don't think the two should be conflated. |
|
@kurtschelfthout The video is good, thanks
I generally prefer arguments in utilitarian terms (bug reduction, safety under refactoring, stability of coding patterns under changing requirements, does a mechanism promote team-cooperation etc.). He makes some of these, though "reasoning about code" is not an end in itself, but can definitely give good utilitarian outcomes. But how many bugs (= lost developer time) are really caused by a lack of coherence, e.g. conflicting instances? I talked about this when last with Odersky and we figured it was very few. But how much time is spent fiddling around with type classes trying to get them to do the right thing, only later hitting some massive limitation like the inability to have local instances, or the inability to abstract instances with respect to a dependency? A good example where lack of abstraction can hit is string culture. Let's say you create a huge amount of code that uses a swathe of string-related type classes that assume invariant culture (e.g. the ubiquitous From what I see people in favor of type classes choose examples that are relatively context-free (no local instances required, or only in contrived examples), while people in favour of implicits choose examples where local instances are commonly needed (e.g. Scala meta programming examples, parameterizing by the meta-programming universe implied by a set of libraries). Both sets of examples also put on heavy pressure w.r.t. dependencies (e.g. types-with-nested-types as parameters - the scala meta-programming examples re replete with these) and higher-kinds. Swift is another important comparison area since it is setting expectations in a new generation of devs. |
|
@dsyme Scala implicits are plagued with problems, the least of which being that Scala's notion of parameterized modules is a runtime phenomenon, leading to issues like being able to have two instances of a @kurtschelfthout Given that it's already possible to encode higher-kinded types to some degree via SRTP, this is probably the best time to have that discussion if we're already having the long-awaited one on type classes for F#, so I don't think @Alxandr is wrong to be bringing it up in this thread. It's difficult to imagine a type class mechanism incapable of It's worth mentioning that the modular implicits1 proposal for OCaml solves many (all?) of the concerns related to both voiced so far in this thread. There is a shorter and more recent presentation3 from some of the designers as well for those curious. 1: arXiv:1512.01895 [cs.PL] |
Yeah, I know.
Yes, I know. However TBH I don't think the case is proven this causes bugs in practice. When talking to Odersky about this recently I said that I doubted that a any production bugs had been caused by this problem, or at least vanishingly few. And the workarounds (such as using a unique key type for sets/maps, which is basically what we already do in F# if you need a custom comparer) are not particularly problematic. Certainly the situation is no worse than the existing .NET collections. Anyway I'd need to see much stronger utilitarian evidence that this truly is as critical as claimed - it seems like a well-intentioned article of mathematical faith (and one which I would wholeheartedly subscribe to from a purely academic perspective) more than one grounded in the reality of software practice. To contrast, the problems of "not being able to parameterize or localize my instances" are very much grounded in modern software reality and standard F# practice. In F#, being able to parameterize a thing by values is very fundamental, even if you have to plumb a parameter to each function of a module or member of a class explicitly. Active patterns, for example, can be parameterized, for good reasons.
From the F# perspective the whole thing is really syntactic sugar just to pass some parameters implicitly :) I would like to see an analysis of the extra powers of Scala implicits by someone who favors the mechanism and uses it well, or at least can speak to its use cases. Some of the use cases I've seen in the area of meta-programming look quite reasonable. The mechanism has problems though. I'll look at the modular implicits work again, it's been a while. Last time I looked it would need extensive modification to be suitable for F#, and it didn't strike me that F# needed to aim for the same goals, but I'll look more carefully. It's a very tricky area to be honest, so many subtleties. |
|
I rather like that Scala will give you an error with an ambiguous instance. Ideally it wouldn't matter, but F# is neither pure nor lazy and so it seems much safer to me to be sure about which instance you're using. Along these lines I think tooling for this feature might be extremely important. It will certainly be necessary to have an easy way to figure out which instance is being used and where it lives. @drvink Modular implicits are pretty neat. I like that they are very explicit with their syntax and so it's more clear to beginners what is going on. One of the weakness (but also paradoxically a great strength) of Haskell is that there is so much magic going on that it ends up taking a lot of mental gymnastics to understand what complex code is doing because so much is inferred. Although, that magic also leads to very terse code. |
You put the many possibilities we already have to propagate values implicitly (statics, thread locals, whatever the thing is called that propagates across async calls) in a negative light, perhaps rightly so. What is the advantage of adding another implicit value propagation mechanism - how is it that much better than the existing ways? Concretely, in the string culture example. Without implicit value passing, we can change all the witnesses to take CultureInfo.CurrentCulture into account instead of the invariant culture, or refer to some other global. Then we have to make sure that the right value for that is set at the right places in the program. Where this sort of thing needs to be scoped statically I've usually resorted to With implicit value passing, we very similarly have to change all the witnesses to take an extra constructor argument - the culture - and use it in the implementation. And then we have to make sure that the right implicit value is brought in scope at the right places in the program. Perhaps I am missing something but it feels very similar. On the positive side, my main reasons for supporting this proposal is to:
Don't know if it's me but I keep running into this limitation, and there are no clean workarounds (I know, because I've worked around them many times in different ways). One example is FsCheck allows you to check any type that it can convert to a I realize you can do all of that with implicit values too, because they're strictly more powerful, but I just feel I already have plenty of choice to access values implicitly - perhaps even too many :) |
|
I've used implicits in scala (that being said I've used scala for all of about 2 weeks, so I'm no expert). And what it was used for was passing an execution context around to async code. Basically, it served the purpose of Also, I agree with @drvink that while allowing people to implement |
|
@Alxandr As a practicing "enterprise" software engineer, I can assure you, that traits are much needed today, while most of engineers in my immediate environment, which I consider typical, cannot and need not grasp the concept of type constructors in order to be more productive and output better architected programs due to traits. It's just that day-to-day programming does not involve writing (library) code which abstracts over type constructors. But also conceptually, it is not the case that traits as discussed here are "severely limited", because traits/type classes are about polymorphism/overloading, while type constructors are about which terms are considered legal types in a programming language. The two notions are quite orthogonal and we should not mix them here. |
|
Fwiw, my 2c: I've been playing around recently with a very simple dictionary-passing approach to typeclasses (encode the instances as record values holding the operations as functions), see e.g. https://github.com/yawaramin/fsharp-typed-json/blob/ae4c808d3619e3703451211ba2bf079cb6c61bc0/test/TypedJson.Core.Test/to_json_test.fs The core operation is a function module To_json =
type 'a t = { apply : 'a -> string }
...
module Ops = let to_json t = t.applyNow, if I could instead mark parameters as implicit, say e.g. with And correspondingly declare the instances as: The compiler would have to convert calls like |
Indeed. I think the closest Swift comes to something like this is through protocols. Compared to interfaces, besides methods and properties they can impose static methods and constructors on the implementing entity. Also some requirements can be specified as optional (you then need to use optional chaining, like the Swift then allows implementing these on types much in the same way as interfaces/abstract classes, but it also allows "protocol extensions". Again comparing to .NET these are like extension methods, but for entire protocols. In this sense, protocol extensions are close to what was proposed in #182. It's interesting also that like extension methods, protocol extension can impose additional requirements on the extended type at the point of extension using type argument constraints. The example they give is, translated to fictional F# syntax: //ICollection<'TElement> an existing type
//this extends all ICollections to be also TextRepresentable (another interface/protocol)
//_if_ their elements are also TextRepresentable.
type ICollection<'TElement when 'TElement:TextRepresentable> with
interface TextRepresentable with
member self.TextualDescription =
let itemsAsText = self |> Seq.map (fun elem -> elem.TextualDescription)
"[" + string.Join(", ", itemsAsText) + "]"This is very close to how protocols in Clojure work - except they are not typed. It seems to me that this is qualitatively different from type classes or implicits. In particular, type classes are a static overloading mechanism. Implicits are syntactic sugar to have the compiler pass implicit arguments to functions. UPDATE In terms of votes this wide range of possibilities for this one suggestion seems problematic, but then of course we have a BDFL @dsyme so the votes are just to appease us unwashed masses anyway ;) Perhaps it makes more sense to have a goal-directed discussion, instead of focusing on mechanisms. What can't you express right now (or is awkward to express) that you think this suggestion should address? (I gave my 2c on that in an earlier comment) |
@kurtschelfthout I'd like to see someone trawl through the various uses of protocols in Swift and pick out 3-4 examples (which couldn't be achieved by OO interfaces, and which feel different in application to type classes) |
|
@dsyme There are a number of use cases of protocols and protocol extensions in the video and slides here: https://developer.apple.com/videos/play/wwdc2015/408/ (note also my update in the comment above - protocol extensions are static constructs, closer to typeclasses than I originally thought, but with more of an OO "feel".). |
I think if you were to visit a variety of language communities, you would find it a prevalent view that that language has somehow hit upon an excellent set of features that has the perfect balance of cutting edge techniques and pragmatism. In reality, that's probably not the case, it is considerably more likely that those communities have learnt about some of the available cutting edge techniques, are familiar with how much easier those techniques make their life and are willing to advocate for them but are not willing to do the same for other techniques which are, to them, unproven and unfamiliar. To give an example where people here probably have no skin in the game, Haskell has terrible out of the box support for Haskell also lacks support for String interpolation. There are those in the Haskell community who would argue that the status quo is totally fine, you can pull in the To people inside the Haskell community, it's easy to workaround this issue and barely registers because doing so is so commonplace. To people outside the Haskell community, it's an immediate example of Haskell's perceived impracticality for production use-cases. In short, it is easy and natural to overvalue the status quo, we should be aware that this is our likely bias and try to consider how decisions might look to those outside a community who might wish to join as well as those on the inside.
This is a classic slippery slope argument and it's worth considering a number of related questions:
Obfuscation is something done intentionally. As for subtlety, I wonder if you can provide an example of what you're thinking of? It seems that there are plenty of things in F# that can already produce subtle code in much more egregious ways, the combination of side effects and lazy evaluation springs to mind. Are the subtleties introduced by type classes notably worse than the subtleties introduced by any other language feature that has been successfully integrated with the language?
On the plus side, more type-level programming has massive run-time performance upsides. This has been notable in the Scala world as the transition from reflection-based JSON codecs to typeclass-based JSON codecs resulted in dramatically faster code as well as fewer weird runtime errors.
The Haskell and Scala compilers have managed to avoid this but Scala IDE tooling is able to provide information on implicit resolution.
Programming communities tend to spend their time talking about lots of surprising things. Surely it's positive that you have an active community that is talking about things rather than a disengaged, uninterested community? Why do you think that these conversations are less useful or productive than any of the other topics that come up in language communities like the millionth iteration of the best form of dependency injection?
I'm not quite sure what the "fragile type classification" problem is because it doesn't appear to have ever been mentioned outside of this thread. I'm going to guess that you're talking about hierarchical relationships between type classes such as the Functor, Applicative, Monad relationship. Such things change quite rarely in practice but wouldn't necessarily need to be provided by FSharp.Core in any case. In Scala, these are provided by an external library, Cats.
Adding any new feature results in those who know how to use that feature having an advantage over those who don't yet know how to use it. This is great because it's an opportunity for people to learn things. If we didn't like learning things, we'd never have learnt F#. Indeed, very easily, at the beginning of F#'s days, you could have said something like: most programmers don't know how to use Since F# was created, far more engineers worldwide have become comfortable with functions like The question is, do you want F# to be setting the agenda like it used to or do you want to be Go, currently adding the cutting edge language features of the early 2000s, like Generics that no modern language should've been built without?
Fortunately, type-classes, unlike interfaces, don't require access to the underlying type to implement. Cats in the Scala-world is able to implement lots of Haskell-inspired abstractions without requiring the language to commit to them. Scala engineers are thus empowered with choice as to whether to write code just with the standard library tools or whether to adopt Haskell-like abstractions and use Cats/Cats Effect and various other libraries that facilitate purely functional Scala.
Person A: "No programmer obsesses over type-level programming." Or alternatively: Person A: "No Scotsman puts sugar on his porridge." On what basis do you conclude that type-level programming is somehow distinct from actual programming with any more validity than Person B concluding that Uncle Angus is not a true Scotsman? When you're about to make a comment that dismisses the fabulous work and contribution of thousands of engineers worldwide who write productive, useful code every day, you need to think very carefully about the implications of that message. There is plenty of both commercial investment and top tier academic research in purely functional, type-level programming and it would be a brave move indeed to write it all off in a single paragraph. I think you need to give more thought to what constitutes your view of "actual programming" and how the practitioners of it differ in their objectives from the practitioners of type-level programming. To create a facetious example, do "actual programmers" love their programs to crash with exceptions or run really slowly? Personally, I doubt it but whatever it is, once you've defined what constitutes that group, you might be able to figure out whether the two communities are capable of learning from one another. I hope that, if nothing else, with this post I have provided some pause for thought about the nature of the arguments voiced against including type-classes, higher kinds, etc in F#. There may be totally valid reasons for not doing so but I'm not 100% sure you've got to the heart of a rational, reasoned, case against here. It may be that you just need to reflect on how that case against has been framed or it may also be that you need to consider more seriously whether that case actually exists. |
|
@dsyme wrote
I've seen protocols misused, but it doesn't sound like that's what was going on here. It's hard to tell for sure without seeing the code you're referencing, but if the Python code was making use of open polymorphism, then the use of protocols almost surely wasn't superfluous. Of course, there are other ways to get open polymorphism in Swift: through closures or traditional OO-style classes (both of which confer reference semantics), so protocols are never "necessary." But it's absolutely natural that a Swift program includes protocol declarations that don't appear in a corresponding (dynamically typed) Python program, just like most OO languages would require the declaration of base classes that would be unneeded in Python. Ultimately this may come down to whether you believe static type systems are valuable. While your sense that the code was obfuscated by the use of protocols is certainly an important data point, the maintainability and performance benefits of static typing should not be overlooked either. I'd like to make a small edit to your conclusion 😄:
Swift protocols don't imply “type-level programming” any more than abstract base classes do. For what it's worth, protocol extensions can make even Generic Programming (per Stepanov) as simple as writing an ordinary method. I certainly have my complaints with details of how Swift defines protocols, but in my experience they have been excellent for Swift's standard library and approachable/understandable for its users. |
|
@TheInnerLight wrote:
I took dsyme's comments to mean. Writing code in terms of types-level abstractions + knowing how those will solve my problem VS writing code to solve my problem. Not to discount the beauty of encoding the solution in math. But it adds a layer. |
|
First, I politely request that the use of the personal pronoun "you" (referring to a specific person) be minimised in replies to this discussion thread. That is not the mode we generally use in the F# community in technical discussions (obviously, there are exceptions). I've carefully gone over my reply to ensure this is followed. If the personalized mode of argument is used I or other admins may delete replies, and request reposting without making arguments in this way. I will clarify this in the repo README. That said, I appreciate the points being made and offer some comments below.
This should indeed be noted in the section at the end of my note, which describes positive uses of this kind of machinery for interop. That said, source code generators for C# and Type Providers for F# can play similar roles (for F#, especially with #450, approved in principle). So if that's the outcome we're after then type-classes are not the mechanism we'd use to get it - indeed it argues for investing elsewhere. Note that F# does embrace very extensive type-level programming via type providers. These suffer some of the above problems (e.g. they run at compile-time, and can cause performance and debugging problems). So some of what I've written can also be seen as a criticism of this existing part of F#, or of compile-time programming in general. However type providers do at least allow debug/performance-profiling/unit-testing paths for the type-level programming logic (since they are compiler plugins, where F# code runs in the toolchain), and avoid the type-level abstractions of category theory or abstract algebra.
We love that people ask for stuff and we have 400+ open issues here for that. That said, "the classic slippery slope" is indeed a basis on which to exclude otherwise useful language features in this repository. In the F# and C# design traditions we have long avoided features with have an obvious slippery slope - we aim to do them complete, as a whole, and resolve the design point to a sufficient coherent closure (though we often fail in that).
It depends entirely what they're talking about. For those looking for community forums or teams discussing category theory or abstract algebraic concepts, then there are other communities for that.
I'm glad this establishes the kind of base line we're talking about when it comes to useless community discussions :) We're not keen on dependency injection in the F# world and are blissfully free of those discussions. Great example.
See the mention of dependency injection. More usefully, there are have been plenty of highly productive community conversations over the years (e.g. the creation of Fable, or SAFE Stack or Bolero or FsCheck or WebSharper or ...). By the high standards of the F# community, arguing over hierarchical type classifications would be solidly in the unproductive camp.
F# is not specifically a vehicle for educating people in new things, nor providing them with an opportuntity to learn new things. We could easily give our error messages in Latin, for example, or include links to category theory primers, or ask people to get to level 10 in Minecraft before their program compiles, or give links to half-relevant Knuth articles - all would be learning new things. Instead, F# is a vehicle for productive programming. This might need a small amount of (re-)education, but (re-)eduction is absolutely not a goal in itself. If a team can productively use F# without any (re-)education that's fabulous!
Yes, they are in the "pretty bad" category, and get worse the more type-level programming machinery is implicitly invoked. F# has some subtle features, it's true, but it goes without saying that we're not in the business of making that worse.
To clarify: F# is not a research vehicle (it hasn't been since ~2014). "Setting the agenda" is not intrinsically a goal, and to the extent we do we have choices over what agenda we set. Sometimes this means not doing things. A good summary of the dimensions of expression-oriented programming which F# has advanced is in my talk "The Early History of F#" at the ML Workshop 2021. We will continue to advance along those dimensions that particularly align with overall productivity improvements and net-utility for programming teams. Those looking for a language whose goal is specifically to be a research vehicle for cutting-edge ideas (e.g. advanced research in program verification, or touch-based-programming, or type-level programming, or functorial programming or similar) should look elsewhere.
Unfortunately that would give rise to bifurcated communities, some depending on this foundational library, some not, some depending on other foundational libraries.
Please replace "actual programming" by "value-based programming" or "expression-level programming" or "programming with data and information" or "programs that run at runtime". In the context of .NET and F#, these are sufficiently well-defined and highly distinct from type-level programming ("programs or logic that runs at compile-time and compute with types") that I'm comfortable with the distinction. As a joking aside - I suspect the strange porridge analogy doesn't apply, perhaps it's something to do with the milk being used? Or maybe he's really eating a burrito? :)
To be clear, this discussion thread is about the utility of adding type classes and its variations on type-level programming specifically in F#. Although it was put on twitter as an "attack" on type-level programming in general, and includes a speculative rant about psychology, it isn't intended as such, and the last paragraph clarifies that there are genuine dimensions of utility in this area - the rest is intended to give balance and rationale. As an example, Anders Hejlsberg has plenty of concern about how the type-level programming machinery of TypeScript is being over-deployed - but the foundational reason for why he's adding the features is genuine utility. He wouldn't be adding them if they weren't highly useful for interoperating with JavaScript. Other programming communities can do as they like. Academic research should absolutely be doing type-level programming and whatever other things thay want. They should be teaching it too - including teaching its critiques. It is fascinating and examinable and trains in abstract thought. If anyone wants me to write an article or a supporting letter for a research grant on why type-level programming makes for a great part of the curriculum, I'd be more than happy to do that. It is easily prevalent enough in C++, Scala, Haskell, TypeScript and even typed-Python to make it valuable to know about - both its pros and cons. |
Right, but the natural "best" F# program corresponding to a Python program generally doesn't use (explicit) protocols, nor does it necessarily use base classes. Instead, for the parts where these occur, it tends to use generic code with perhaps 1-3 function parameters to give adhoc meaning to the type parameters. These techniques are routine in F# and thoroughly well supported through HM type inference. SRTP may also be used for some things that Swift protocols may cover.
Thanks, yes, that's a useful clarification. Swift protocols are certainly at the very, very light end of type-level classification and involve little computation. |
I think you're well beyond my knowledge of F# at this point, but it sounds like this means the “best” F# program is prone to diagnostic backtraces like those that result from unbounded type inference in Haskell, or for that matter, template instantiation in C++. When everything in the program type-checks, the type annotations might seem like noise, but during development and maintenance they are essential, produce a better experience for library users, and ultimately reduce the amount of documentation required. Maybe this just proves F# isn't a language for me, but IMO programming is barely tenable when a type error in a caller can produce a diagnostic in its callee. |
I think Don means exactly the opposite of that: the "best" F# program doesn't make heavy use of SRTP and therefore such diagnostic noise is kept to a minimum (because typechecking errors are precise under these circumstances; otherwise you indeed have a similar problem to the one with C++-style simulation of typeclasses via templates). |
I understand, but I believe this is ok in a typical F# IDE, doing whole-file type inference and error reporting on-the-fly, with appropriate type information displayed in tooltips and codelens for partially complete programs. At least, I think that would be enough to be comfortable with the kind of type reconstruction and generalization that F# does. |
|
Just to clarify: I'm still against adding type classes without HKs. Category theory (focused in computer science) definitely matters as it provides a "framework reference" to make ad-hoc overloading less ad-hoc. This has currently an impact in (mostly library) F# programming and I've seen many times people hitting a wall after adding overloads in a disorganized way, which doesn't follow any common abstraction, and getting logically type inference confused in client code, which forces at some point a breaking change in their design, but it really takes long to realize that, sometimes it never happens. Moreover, those compiler errors are actually way more cryptic than say Haskell-like errors like "an instance for Monad is missing in type X". But even if/when those technical problems are sorted out, the resulting API is so complicated to use that we have to resort to docs or source code to know which kind of black magic the overload resolution is providing. On the same track, I'm convinced type inference would speed up if validating against type classes as opposed to a big chunk of unrelated overloads. I am not talking about advanced CT abstractions but simple day-to-day use, specially in library design, like Monads, Applicatives , Functors and Traversables, same ones that are explained nicely without requiring any degree in Maths in the acclaimed F# for fun and profit website, with real world examples. I think no one can deny those abstractions are present today in F# code and probably key language features like CEs weren't a reality today if those abstractions were always completely "banned" to use in programming languages. Now, using say Monads and not having a way to express them in the type system, complicates understanding them, and kind of relegate it use to language designers or library designers, a situation which actually promotes this:
I mean, nowadays developers who know other languages where HKs are a reality, are in a better position than the rest to create (or port to F#) libraries (I think I don't need to give examples of such popular F# libraries). So this is effectively causing such division. Reflecting the real type it represents in the type system would help consumers to understand the abstractions, i.e. it would "socialize" the abstraction, instead of keeping it as a secret sauce for the library (or language) designer. It's important to clarify that even in that hypothetical case, knowing CT won't be a requirement to use such library. Now let's forget about CT, higher kinds are still useful, they would allow for instance to better express functions dealing with unit of measures, also it would allow better expressing constraints to our domain model, when used in a proper way. Could it be abused? of course, like any other existing feature. My personal experience while learning generics was from time to time getting surprised when finding the HK limitation as I was not aware of Kinds at all, so it was more complicated to deal with that limitation and the compiler error was cryptic to me. I was wondering "if this allows generic types, why can they only be defined in certain cases". So, I think Higher Kinds are a natural evolution and the same way generics were criticsized 20 years ago, it will be seen as a natural way of expressing types for new generations of developers. And by the way, this whole discussion reminds me in general the way FP was regarded 25 years ago, like an academic thing only for people interested in the math behind computer programs. |
Likewise, I appreciate the time taken to reply.
Isn't the type of type level programming that is discussed above very far beyond, in terms of complexity, the use of type-classes as per the topic of this thread? F# type providers are a very nice piece of technology and probably one of the stand-out features of the language but they fall more into the same category of features as meta-programming features as Template Haskell and Scala macros. It's hard to imagine even the most ambitious Scala or Haskell type astronaut recommending reaching for them too quickly. They are a fine solution if one wishes to automatically generate types from a database or huge schema file but come with dramatically more scope to shoot oneself in the foot than a lawful typeclass representing something like a Semigroup. These are obviously at opposite ends of the spectrum but is it desirable to push people seeking type-level features to the most complicated extreme?
It's good that those features are available if needed but debugging is not at all desirable. If one has reached the point of firing up a debugger in order to understand their program flow, it's probably crossed a serious complexity barrier and the feedback loop is going to be slow and less productive. Unit tests are better, the feedback loop is faster and, especially with property tests, it's possible to cover things pretty exhaustively if you're extremely careful. Neither of them are remotely as fast and ergonomic as type errors directly fed back by the compiler.
I am pleased that F# has transcended such discussions, everyone in the community is totally clear on how to architect programs of arbitrary size, and the F# community is "blissfully free" of those discussions.
It is, in fact, so "blissfully free" of them, that a quick google yields an inexhaustive list of some massive ones, some of which amount to practically a small book about the topic. Even if we don't actually talk about dependency injection explictly and instead talk about free monads, effect systems, tagless final and ReaderT patterns, we're still fundamentally talking about how to architect programs achieving the best compromise of abstraction, readability, correctness, maintainability and testability. The answer of how best to do that certainly hasn't been answered. Maybe it will never be answered as the techniques we use just continue to slowly evolve. Even if we've had these conversations a million times and, mostly, little progress is made, the only thing worse than people talking about them is people not talking about them, because that means they've given up on solving the problem.
Meanwhile, were the Scala community so absorbed by arguing over hierarchical type classifications that they were unable to produce Http4s, Cats, Circe, Doobie, FS2 and Scalajs? Or did it prevent Haskellers producing Yesod, Hasql, Reflex, Pandoc, XMonad or IHP? It's not clear if this comment is supposed to imply otherwise but I think it's pretty safe to conclude that the existence of hierarchical type classifications is orthogonal to community achievement. I trust that is not in dispute.
My whole quote has been reintroduced here because the original point has actually not been considered. Let me reiterate that every new language feature is something that most people don't currently know how to use. The fact that some people do not currently know how to use new language features has not, seemingly, been a reason to freeze all language development on F# so far. That being the case, I'm struggling to understand how it is problematic to introduce well studied, proven, concepts from both other languages and academia simply because people who have spent years of their lives devoted to studying those things know more about them than people who have not. Surely it's reassuring to know that we can introduce proven concepts into the language, rather than something half baked on the back of an envelope? I'm also struggling to see how introducing well proven concepts from computer science to a programming language bears any relation to including Minecraft or Latin in the compiler (although maybe Microsoft have done weirder things in the past 😆).
That's great. I am pleased to hear that F# is for productive programming. Typeclasses are a language feature that can be highly productive. Sounds like F# and typeclasses could be a match made in heaven :)
What if someone is looking for a strongly typed, functional programming language that embraces type-level programming for industry productivity? It would be concerning, in my view, if F# concluded it was not for such people. I have tremendous respect for all the academics who contribute to the world of functional programming research and there's lots of exciting things going on that field that are set to contribute massively to the field of software engineering in the future. Ultimately though, my interest is primarily in building really scalable, maintainable, reliable web services. There are well proven, well understood, type level programming techniques that help to do that and if people interested in that sort of thing are encouraged to go elsewhere, they will, and they will take their opportunities for commercial functional programming with them.
Sounds like a community with a diverse set of opinions and ways of doing things within, with different parts that could learn a lot from each other, can imagine that would be awful :)
So, it is indeed about programs that annoyingly fail at runtime then? I work a lot with Json and Avro formats, it's useful to be able to serialise data into these formats, it's a simple, value-based, data-based programming problem. I can construct a program in F# and in Haskell or Scala to do that. In fact, the only difference between the F# and Scala/Haskell versions of this code is that the F# has extra failure modes because I can't guarantee that I don't try to encode something absurd. It's actually worse than the F# version just having an additional failure mode though. Since the serialisation is based on reflection, I have a massive security vulnerability since a small typo can lead to me readily exposing the wrong data via my web API. In both Haskell and Scala I have to explicitly mark a type as a encodable to JSON for it to be capable of being exposed to the outside world, in F#, reflection will magically figure out how to do that, no matter how horrible the consequences of that.
I'd like to conclude by adding that should a contributor here not wish their comments to be taken as attacks on the type-level programming community, they should avoid making statements that dismiss a very large and diverse community to some special category of not "actual" programming regardless of their intent when doing so and should instead focus on critique of the specific type-level programming techniques in question. |
|
can you take a look at spiral lang and see if it does fit the needs of F# + type stuff? also how the serialization work https://github.com/mrakgr/The-Spiral-Language |
This is the core of the matter, and the answer is that F# is not for such people. For that, there are some options (Haskell, PureScript, perhaps Scala depending on what you're into). Just to clarify this point:
But if you're looking for a home as a "I wish to embrace type-level programming in my language" kind of programmer, you should look elsewhere. You can be plenty productive in F# without hierarchies of this nature, but if that's not what you want, then you'll be a lot more satisfied with a different language. |
|
Some purely mundane and non-philosophical points on this matter: Type providers are tricky to understand and implement, but once you have one, client code can be written against it without having to understand any new type-system features. You just call normal methods and properties on what is for all practical purposes another normal .NET type, and the design decisions in TPs and the TP SDK don't ripple throughout the whole ecosystem. A new F# feature might be able to replace something you previously could have implemented with clever type provider magic, but TPs are mostly orthogonal to the design of the F# type system itself. Haskell's status as a research language affects how easily even "practical" new features can be incorporated into it. Lots of features that are de-facto standard and stable at this point still require opt-in language pragmas (often a combination of pragmas). In .NET, you basically just have Also, any new F# type-level features that can't be encoded at the level of the IL itself raise a whole host of interop issues. If I write a HKT/typeclass-based library in F#, can I use it to create a C# application? In an F# application, can I use a framework written in C# and instantiate it with things I've defined using the latest F# feature? (Functions with SRTPs, which form the basis of most demonstrations of advanced type-level programming in F# floating around the web, can't be called from C#). So arguments about new type-level programming features are more likely to derail the development of F# than Haskell. Even though there's been lots of cross-pollination of ideas between F# and the broader .NET ecosystem, to steer the ship of F# in some directions, you would need to steer the whole ocean. |
|
Responding to a few parts:
@cartermp - I believe type providers should be included in your list of available variations on type-level programming techniques
In practice no, not in terms of actual experienced complexity, nor in terms of the skillset required. When authoring type providers, the type-level programs are expressed using term-level programming with all the utility of the expression language of F# itself (that is, type providers are macro-like compiler addins authored with the F# expression language, manipulating and producing reflection objects - System.Type and quotations - at compile-time, and programming with these reflection objects is already known to most F# programmers). This means relatively good programming, debugging, profiling, logging, diagnostic reporting etc. is available for type-provider authors (though it could be improved, and there are many gotchas). In contrast, if the logic being implemented by typical type providers (e.g. validating SQL, or parsing HTML/JSON/CSV/XML) were instead encoded in type-level programming with, say, type classes, implicits, constraints, type-level conditionals, type-level loops, type-level web-requests etc. then it would be utterly intractable, entirely baroque and impossible to debug. At least it's beyond my capabilities to make F# into a language where this is simple enough. One can write JSON parsers using TypeScript type-level constraint logic but I'm not ever going down that path with F#, I'm really not. Especially given that most practical industrial uses are covered (or potentially covered) by type providers (F#) and source code generators (C#).
As mentioned by @DalekBaldwin authoring new type providers may require debugging and testing. However the usage model of a completed, tested type provider is based around the provided nominal types and their methods/properties, and is generally very simple, and does not need debugging or testing pathways for the type-level logic. F# Myriad is another approach worth mentioning. Source generators for C# are also an important comparison point, as is T4 template meta-programming. Again the result in each case is nominal types, and the delivered usage model of each mechanism simple. C# and F# analyzers are also relevant, again implemented using compiler-addins operating on reflection-like views of programs (from Roslyn and FSharp.Compiler.Service), including partially completed programs.
If that is the specific aim, I would point them to the specific kind of reflective type-level programming done using F# type providers, or C# code generators, or F#/C# analyzers. all of which have many proven practical industry uses, sometimes overlapping. I would also apologise that #450 has not been landed as yet (it is certainly useful for type providers to take types defined in the same assembly as inputs).
If any critique of type-level programming ("diminishing its achievements") is an attack, then type-level programming has become holy - a thing that may not diminished. The problem is, that itself could be added to the list of negatives.... |
I would tend to agree that this is a good use case for something like type providers, indeed doobie in Scala which I previously mentioned uses macros to do this. So far we're on firm footing in terms of using equivalent features to do like with like. The subtlety is that doobie introduces various type classes to permit values of particular types to be successfully rolled into or extracted as results from SQL queries. Doobie provides instances in the base library for common types and then various extensions (some of which could even be authored by other contributors) provide additional, more specific instances. An example being support for postgis types. This is a pretty good story for everyone involved because the macro/type provider author wouldn't need to provide a mapping for every use case, they provide the structural mapping to SQL and typeclasses provide the specific ability to work with individual types (and if we make mistakes and use incorrect types, we still get nice compiler errors) so a nice example of the two things working together.
This is trickier, I agree. If we get into using Haskell Generics, Template Haskell or Shapeless in Scala to generate JSON instances for example, we've potentially re-introduced some of the same problems as I previously called out with Reflection. It saves a lot of time but we're at the mercy of the algorithm used to structurally generate that type mapping. It's still better than reflection, it still avoids compiler errors, is fast at runtime, forces the author to mark types they want to be explicitly decodable or encodable but it's harder to customise specific behaviour we might be looking for on a per field basis. I think both of these approaches are absolutely valid but, particularly for production systems, I'd be much more comfortable with hand written
I think this is a good start. As I pointed out above, this is somewhat orthogonal to the question of type classes because one can use both of these approaches together to produce better results than with one of the approaches alone.
I'd like to apologise for the lack of clarity there and the somewhat clumsy wording of that particular clause of my final comment which has caused some confusion and I will modify it to clarify the intent. I trust that we're all capable of drawing a distinction between specific critique of elements of type-level programming which, I agree, are absolutely called for in the context of this thread and broad community bashing which is absolutely not. |
|
I agree with @TheInnerLight in that both features seems to be orthogonal. I think if we had both normal HKs and type providers (with the pending features above mentioned) there won't be any need to go down the road of complicated type level programming, since any advanced type level technique beyond normal HKs would be easier to achieve with type providers. That would effectively stop that slippery slope. Nowadays we don't have either: we have generics but limited to first order kinds, type providers that can't generate types from other types, oh and SRTPs that ignore extensions. That's probably one of the reasons this suggestion have so many votes, since as library developers we can't go too far and I'm under the impression that many devs have the (wrong) feeling that typeclasses alone (without HKs and other stuff) would solve lot of these limitations. Why don't we see so much progress? This is probably due to lack of resources / time to implement the above mentioned features, but HKs is a bit special as it was stated before that it won't happen until the CLR team implement them, but now I wonder if this statement means that even when the CLR team implement them and while C# embraces Higher Kinds, F# will forbid them. That would be really unfortunate IMHO. Note that if we have all above features (specially HKs) the lack of typeclasses won't be a big issue as we'll be able to use other similar techniques like SRTPs (but with real HK type constraints) or possibly the proposed static interfaces. |
|
I'm the weird one but I'm actually most hyped about improving type providers to shim features in. Maybe I'll set aside some time to mess with. |
I seem to have linked the wrong issue, and I cant find that one anymore, that I meant. It was based of a years old discussion, and maybe we get an update on that one. What is the official stance on HKT's and no type classes on top of it? |
|
One thing I would like to see is concrete example code (pseudo-code) of what people are trying to write that cannot be done today; it would help people see the pros and cons of the features. I have a model compiler that has HKTs and type-class-like thing with interfaces; intended for learning the features and how they are implemented. However, I don't have super clear examples of where they are useful. |
|
I have a type ResultExpression () =
member this.Bind (x, onOk) = Result.bind onOk x
member this.Return x = Ok x
member this.ReturnFrom x = xWith F# 5, I thought it would be cool if I expanded it to be an applicative that would append errors together instead of just reporting the first error: let resultExpr =
result {
let! x = Error [ "x failed" ]
and! y = Ok 5
and! z = Error [ "z failed" ]
return x+y+z
}
test <@ resultExpr = Error ["x failed"; "z failed"] @>I think I need typeclasses for this. Not HKT, just vanilla typeclasses. (It only needs Semigroup)
type ResultExpression () =
...
member inline this.MergeSources(t1, t2) =
match t1, t2 with
| Ok r1, Ok r2 -> Ok (r1, r2)
| Error e1, Error e2 -> Error (e1 ++ e2) // this is using ++ from F#+
| Error e, _ | _, Error e -> Error e
member this.BindReturn(x, f) = Result.map f xI hope this example doesn't add fuel to the fire though. I generally agree with @cartermp that this is straying from F#'s target audience, and if you want to do a lot of this kind of abstract programming, there are languages that are more suited to that task. You can get plenty far with F#+ and SRTP and keep it pragmatic. (Though wrestling with compiler errors when using F#+ is generally harder than equivalent errors in Haskell or whatever). |
|
Just curious, what is the status of this suggestion so far? Has any more thought been given to it by the devs? |
and others
NOTE: Current response by @dsyme is here: #243 (comment)
Submitted by exercitus vir on 4/12/2014 12:00:00 AM
392 votes on UserVoice prior to migration
(Updated the suggestion to "type classes or implicits", and edited it)
Please add support for type classes or implicits. Currently, it's possible to hack type classes into F# using statically resolved type parameters and operators, but it is really ugly and not easily extensible. I'd like to see something similar to an interface declaration:
Existing types could then be made instances of a type classes by writing them as type extensions:
I know that the 'class' keyword could be confusing for OO-folks but I could not come up with a better keyword for a type class but since 'class' is not used in F# anyway, this is probably less of a problem.
Original UserVoice Submission
Archived Uservoice Comments
The text was updated successfully, but these errors were encountered: