Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a line continuation digraph \# #35336

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions src/julia-parser.scm
Original file line number Diff line number Diff line change
Expand Up @@ -522,6 +522,21 @@

(define (scolno port) (string " near column " (input-port-column port)))

(define (line-continuation port s)
(if (eqv? (skip-to-eol port) #\newline)
(read-char port)) ;; Consume newline
(if (eof-object? (peek-char port))
(error "incomplete: expression ends in line continuation"))
(let ((tok (next-token port s)))
(aset! s 2 #t)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so a line continuation counts as nonzero whitespace? That makes sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. The newline should definitely count as whitespace. This prevents things like

@info "hello" \#
x

being parsed as an invalid string juxtaposition @info "hello"x.

Interestingly, this brings to light an existing behavior of the Julia parser. Whitespace before a comment gets ignored by the next token, so x #==#y is parsed as x*y even though there is whitespace after x.

tok))

(define (line-continuation-comment port s)
(read-char port) ;; Consume #
(if (eqv? (peek-char port) #\= )
(error "Cannot start block comment after \\#"))
(line-continuation port s))

(define (next-token port s)
(aset! s 2 (eq? (skip-ws port whitespace-newline) #t))
(let ((c (peek-char port)))
Expand All @@ -535,6 +550,12 @@

((eqv? c #\#) (skip-comment port) (next-token port s))

((eqv? c #\\)
(let ((c (read-char port)))
(if (eqv? (peek-char port) #\#)
(line-continuation-comment port s)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small nit: Might it be clearer to structure line-continuation-comment more similarly to skip-comment above where next-token is done separately from skipping the whitespace?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm completely fine either way. It probably makes sense to first see what the consensus will be about specific choice of characters, etc. and then perfect the implementation.

(read-operator port c))))

;; . is difficult to handle; it could start a number or operator
((and (eqv? c #\.)
(let ((c (read-char port))
Expand Down
6 changes: 6 additions & 0 deletions test/syntax.jl
Original file line number Diff line number Diff line change
Expand Up @@ -2221,3 +2221,9 @@ end
h35201(x; k=1) = (x, k)
f35201(c) = h35201((;c...), k=true)
@test f35201(Dict(:a=>1,:b=>3)) === ((a=1,b=3), true)

# Line contination: issues #18612 and #27533
@test Meta.parse("[1 \\#\n 2]") == Expr(:hcat, 1, 2)
@test Meta.parse("@f \\#\n a") == Expr(:macrocall, Symbol("@f"), LineNumberNode(1, :none), :a)
@test Meta.isexpr(Meta.parse("@f \\#"), :incomplete)
@test_throws ParseError("Cannot start block comment after \\#") Meta.parse("x \\#=")