Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JuliaParser dying in amend_coverage_from_source #77

Closed
kshyatt opened this issue Jul 27, 2015 · 5 comments
Closed

JuliaParser dying in amend_coverage_from_source #77

kshyatt opened this issue Jul 27, 2015 · 5 comments

Comments

@kshyatt
Copy link
Collaborator

kshyatt commented Jul 27, 2015

I've tried running CoverageBase on julia since it's failing on the buildbots. When I run Coverage.process_folder("base"), it runs happily for a while, but then I get this error:

Coverage.process_file: Detecting coverage for base/docs/Docs.jl
ERROR: ParseError("invalid \"import\" statement")
 in parse_import at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1150
 in anonymous at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1087
 in parse_comma_sep at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1158
 in parse_imports at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1087
 in parse_resword at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1013
 in parse_call at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:608
 in parse_decl at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:488
 in parse_factorh at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:510
 in parse_unary at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:544
 in parse_rational at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:393
 in parse_with_chains at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:191
 in parse_term at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:390
 in parse_shift at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:387
 in parse_with_chains at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:191
 in parse_range at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:437
 in parse_pipes at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:378
 in parse_comparison at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:396
 in parse_ineq at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:375
 in parse_arrow at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:372
 in parse_and at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:369
 in parse_cond at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:254
 in _parse_atom at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1680
 in _parse_atom at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1651
 in parse_atom at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1832
 in parse_unary_prefix at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:600
 in parse_call at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:606
 in parse_decl at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:488
 in parse_factorh at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:510
 in parse_unary at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:544
 in parse_rational at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:393
 in parse_with_chains at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:191
 in parse_term at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:390
 in parse_shift at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:387
 in parse_with_chains at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:191
 in parse_range at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:437
 in parse_pipes at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:378
 in parse_comparison at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:396
 in parse_ineq at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:375
 in parse_arrow at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:372
 in parse_and at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:369
 in parse_cond at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:254
 in _parse_arglist at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1249
 in parse_arglist at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1272
 in parse_call_chain at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:633
 in parse_call at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:608
 in parse_decl at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:488
 in parse_factorh at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:510
 in parse_unary at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:544
 in parse_rational at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:393
 in parse_with_chains at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:191
 in parse_term at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:390
 in parse_shift at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:387
 in parse_with_chains at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:191
 in parse_range at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:437
 in parse_pipes at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:378
 in parse_comparison at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:396
 in parse_ineq at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:375
 in parse_arrow at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:372
 in parse_and at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:369
 in parse_cond at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:254
 in parse_Nary at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:284
 in parse_comma at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:363
 in parse_eq at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:353
 in parse_Nary at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:315
 in parse_resword at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:987
 in parse_call at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:608
 in parse_decl at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:488
 in parse_factorh at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:510
 in parse_unary at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:544
 in parse_rational at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:393
 in parse_with_chains at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:191
 in parse_term at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:390
 in parse_shift at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:387
 in parse_with_chains at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:191
 in parse_range at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:437
 in parse_pipes at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:378
 in parse_comparison at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:396
 in parse_ineq at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:375
 in parse_arrow at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:372
 in parse_and at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:369
 in parse_cond at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:254
 in parse_Nary at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:284
 in parse_comma at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:363
 in parse_eq at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:353
 in parse_Nary at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:284
 in parse_stmts at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:341
 in parse at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1877
 in parse at /Users/kshyatt/.julia/v0.4/JuliaParser/src/parser.jl:1880
 in anonymous at /Users/kshyatt/.julia/v0.4/Coverage/src/Coverage.jl:127
 in open at iostream.jl:114
 in amend_coverage_from_src! at /Users/kshyatt/.julia/v0.4/Coverage/src/Coverage.jl:123
 in process_file at /Users/kshyatt/.julia/v0.4/Coverage/src/Coverage.jl:151
 in process_folder at /Users/kshyatt/.julia/v0.4/Coverage/src/Coverage.jl:171
 in process_folder at /Users/kshyatt/.julia/v0.4/Coverage/src/Coverage.jl:177
 in process_options at ./client.jl:284
 in _start at ./client.jl:411

It's dying here. The fact that this only happens for base/docs/Docs.jl whether we ran the inlined or non-inlined tests is really weird. Anyone have any ideas?
@IainNZ
Copy link
Contributor

IainNZ commented Jul 27, 2015

Only thing I changed in those parts is that the function used to return a coverage vector, but I changed it to nothing. Does JuliaParser actually parse the file separate from Coverage?

@kshyatt
Copy link
Collaborator Author

kshyatt commented Jul 27, 2015 

open(srcname) do io
            while !eof(io)
                pos = position(io)
                linestart = minimum(searchsorted(linepos, pos))
                ast = Parser.parse(io)

Certainly appears to be trying to parse each line. It could be a problem with the file in base, rather than anything you did! The line it's unhappy with is this.

@yuyichao yuyichao mentioned this issue Jul 28, 2015
Closed
@timholy
Copy link
Member

timholy commented Jul 28, 2015

As background, the parser is used to distinguish "real code" from useless crap (comments, lines with just end on them, etc). Using the parser was basically the breakthrough that allowed us to count uncovered lines against a package, rather than just counting lines prefixed with 0 against a package. So yes, it parses every line of the file and assigns 0 to the coverage of any untested line.

Of course, this has the side effect of making Coverage sensitive to any parsing error.

@IainNZ
Copy link
Contributor

IainNZ commented Jul 28, 2015

@tkelman pointed out another failure here: JuliaLang/METADATA.jl#2960 when I tagged the new version, also parsing, but different location I think?

@tkelman
Copy link
Contributor

tkelman commented Jul 29, 2015

Same failure I think.

@felipenoris felipenoris mentioned this issue Aug 6, 2015
Merged
14 tasks
@IainNZ IainNZ closed this as completed Aug 19, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@IainNZ @kshyatt @timholy @tkelman