Type annotations for fn/lambda

[Foxboron]

Beta stage.

Reff #640

optional and kwargs does not work as syntax isn't really decided upon, and we can't annotate return values as it needs macro changes.
Current approach checks for a HyList as given var and destructs this. Passing the annotations through self.compile pulling out _expr from the Result() object. This way the types are rather "dynamic" when passed along.

=> (defn a [[y int]] (print "lol"))
=> a.--annotations--
{'y': <class 'int'>}

=> (defn a [[y "HEYHO LETS GO!"]] (print "lol"))
=> a.--annotations--
{'y': 'HEYHO LETS GO!'}

=> (defn a [[y (lambda [[x "Just wanna get hy"]] (print "Heyho!"))]] (print "lol"))
=> a.--annotations--
{'y': <function <lambda> at 0x7f4939167598>}

The current(?) syntax i imagen we could use is (adding with comment below, ninja edit)

(defn foo [[x int] &optional [[a str] "Hey"]] -> int
  (+ 3 4))

CC: @hylang/core
Comments, code review, rate and hate!

[Foxboron]

Parsing return types can be hard. I got no idea how to approach this.

There is no way to figure out when something is a return value or a body without adding new syntax.

(defn foo [a] (max 3 7))
(defn foo [a] int)
(defn foo [a] int (max 3 7)) 

I suggest maybe doing something like

(defn foo [a] -> int (max 3 7))

Opinions!

[Foxboron]

Hy seems to not be bothered by a clean arrow. So i figured borrowing from Python would be neat.
Pushed support for return annotations.

=> (defn a [[y int]] -> (max 3 7) 7)
=> a.--annotations--
{'y': <class 'int'>, 'return': 7}

[coveralls]

Coverage decreased (-2.7%) when pulling 5f54dbe on Foxboron:defn-type-annotations into 28c046c on hylang:master.

[coveralls]

Coverage decreased (-0.87%) when pulling 2f400e4 on Foxboron:defn-type-annotations into 28c046c on hylang:master.

[coveralls]

Coverage decreased (-0.14%) when pulling 69ae2cf on Foxboron:defn-type-annotations into 28c046c on hylang:master.

[rcarmo]

Doesn't feel very LISPy to me... Then again, I'm quite used to Clojure hinting, which has spoiled me somewhat.

[paultag]

Care to share syntax?
On Sep 10, 2014 2:08 AM, "Rui Carmo" notifications@github.com wrote:

Doesn't feel very LISPy to me... Then again, I'm quite used to Clojure
hinting, which has spoiled me somewhat.

—
Reply to this email directly or view it on GitHub
#656 (comment).

[Foxboron]

I did some testing for fun
https://github.com/ceronman/typeannotations

=> (import [annotation.typed [typechecked]])
=> (with-decorator typechecked (defn test [[a int]] -> int (print "yes")))
=> (test 1)
yes
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/fox/.virtualenvs/hy/lib/python3.4/site-packages/annotation/typed.py", line 450, in wrapper
    return _check_return_type(signature, target(*args, **kwargs))
  File "/home/fox/.virtualenvs/hy/lib/python3.4/site-packages/annotation/typed.py", line 427, in _check_return_type
    raise TypeError('Incorrect return type')
TypeError: Incorrect return type
=> (test "lol")
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/fox/.virtualenvs/hy/lib/python3.4/site-packages/annotation/typed.py", line 449, in wrapper
    _check_argument_types(signature, *args, **kwargs)
  File "/home/fox/.virtualenvs/hy/lib/python3.4/site-packages/annotation/typed.py", line 418, in _check_argument_types
    raise TypeError('Incorrect type for "{0}"'.format(name))
TypeError: Incorrect type for "a"
=> 

[Foxboron]

;; Something like?
(defn a [y ^str] (print "lol"))
(defn a [^str y] (print "lol"))

;; But then
(defn a [y ^"HEYHO LETS GO!"] (print "lol"))
(defn a [^"HEYHO LETS GO!" y] (print "lol"))

(defn a [^(lambda [x] (print x)) y] (print "lol"))
(defn a [y ^(lambda [x] (print x)) ] (print "lol"))

[rcarmo]

I prefer the (defn foo [^Prefix param] ...) syntax, same as Clojure's. I think it would save people a lot of time trying to figure things out, and that there's a lot to be said for recognizable patterns shared among LISPs.

(i.e., reinventing the wheel needs very good argumentation, and even clearer documentation for people coming across the idiom)

On 10/09/2014, at 13:48, Morten Linderud notifications@github.com wrote:

;; Something like?
(defn a y ^str)
(defn a ^str y)

;; But then
(defn a y ^"HEYHO LETS GO!")
(defn a ^"HEYHO LETS GO!" y)

(defn a ^(lambda [x](print x)) y)
(defn a y ^(lambda [x](print x)) )
—
Reply to this email directly or view it on GitHub.

[Foxboron]

@rcarmo The problem i see here is that its the same syntax for different problems. Type hints in clojure is used for optimization, nothing more. Type annotations is much more flexible and goes beyond the scope of hints.

If the question is about using the same syntax, we should much rather go with the syntax from typed-clojure (is this even possible?) or Racket.

[rcarmo]

+1 for typed-clojure. Larger "customer base" than Racket's.

[Foxboron]

@rcarmo i got a feeling that will be damn hard to implement correctly. But i think it would look better.

[agentultra]

Maybe we could consider the meaning of annotation in an old-school Pythonic lense of using a decorator to apply its arguments to the function's annotations dict?

(: [[a int] [b str] [-> str]]
  (defn foo [a b]
    (* b a)))

Wouldn't cost much to implement.

[Foxboron]

@agentultra I did have a implementation doing that. But i don't know the practical difference between adding it into the AST or as the decorator. I can write the given decorator above tho and test with some type checking libs.

[agentultra]

@Foxboron Just a suggestion, admittedly not the best. Annotations appear in the arg grammar definition... so kind of makes sense to mirror the Python syntax as closely as possible. Unless a macro can modify the arg objects of a FunctionDef at expansion time.

[Foxboron]

@agentultra it seems like there is a big difference between adding annotations on the AST level and just doing it in .__annotations__. Tried using the typeannotations lib to see what would happen, and it dosnt actually detect the types given by the decorator. It removes them.

Another option is adding the suggested "decorator" syntax into the compiler. Then just add everything there, as we got control on everything.

@paultag @olasd sanity checks! Common D:

[Foxboron]

Code i wrote btw:

(defn AddAnnotations [ann ret]
  (defn _ [fn]
    (def ret-ann (dict ann))
    (assoc ret-ann "return" ret)
    (setv fn.__annotations__ ret-ann)
    fn)
  _)

(defmacro ann [types ret fn]
  (def ret-type `(second '~ret))
  `(with-decorator (AddAnnotations '~types ~ret-type) ~fn))


(ann [[a int] [b int]] [-> str]
  (defn foo [a b]
    (print "lol")))

(print foo.--annotations--)

(import [annotation.typed [typechecked]])

(with-decorator typechecked
(ann [[a int] [b int]] [-> str]
  (defn foo [a b]
    (print "lol"))))

(print foo.--annotations--)

;; Output:
;; {'b': 'int', 'return': 'str', 'a': 'int'}
;; {}

[Foxboron]

quick update. @agentultra's suggsted syntax might actually work. I'm just having a problem trying to quote the variables without quoting the types aswell. I was just being a tad blunt and quoting everything, making several problems down the line.

[Foxboron]

Did some more work on type annotations.

(import inspect)

(defn AddAnnotations [ann ret-ann]
  (defn _ [fn]
    (assoc ann "return" (first ret-ann))
    (setv fn.--annotations-- ann)
    fn)
  _)

(defmacro ann [args fn]
  (def ret-ann `(if (in :-> ~args) (slice ~args (+ 1 (.index ~args :->)))))
  (def fn-args (get fn 2))
  (def ann-args `(dict (zip '~fn-args (slice ~args 0 (len '~fn-args)))))

  `(with-decorator (AddAnnotations ~ann-args ~ret-ann) ~fn))

(ann [int int :-> str]
  (defn foo [a b]
    (print "lol")))

(print foo.--annotations--)
(print (inspect.signature foo))

This should be compatible with any libs using annotations, as inspect returns:

# Map of annotations
{'return': <class 'str'>, 'a': <class 'int'>, 'b': <class 'int'>}

# Inspect returns
(a:int, b:int) -> str

For further discussion i suggest a keyword/optional annotation syntax along the lines of:

(ann [str &optional {:a int :b str} :-> str]
  (defn foo [bar &optional [a 12] [b "hey"]]
    "Somethingsomething"))

Atm borrowing from core.typed to see if this solves our problem. Not sure how we would be able to parse the latter examples, but might be the correct direction?

[Foxboron]

I'll just keep this open even tho i have kinda been trying to do this as a separate library and see how far i can take it.

https://github.com/Foxboron/Hype

[Tritlo]

+1, I really like the type annotation syntax, with the "ann" decorator.

[Foxboron]

@Tritlo The lib is somewhat functional for use :P

[paultag]

I'm going to close this until @Foxboron is done prototyping this stuff :)

👍