New lexer#559

Merged

HalidOdat merged 321 commits intomasterfrom new_lexer

Aug 27, 2020

+4,896-2,572

This pull request is big! We're only showing the most recent 250 commits

Commits on Jun 22, 2020

Continuing to adapt parser to new cursor returns
Paul Lancaster
committed
Fix multiple mutable cursor reference
Paul Lancaster
committed
Updated benchmarks lexer/parser usage
Paul Lancaster
committed
Updated cli/wasm usage of parser
Paul Lancaster
committed
Back to a compiling state
Paul Lancaster
committed
Seperated cursor peek and peek(skip > 0)
Paul Lancaster
committed

Commits on Jun 24, 2020

VecDeque used to allow peeking 1 ahead
Paul Lancaster
committed
Removing back/prev from parser cursor
Paul Lancaster
committed

Commits on Jun 27, 2020

Fixed peek_more parser cursor which caused infinite loop
Paul Lancaster
committed

Commits on Jun 28, 2020

Modified parser cursor.expect behaviour to fit usage
Paul Lancaster
committed
Traced down / fixed bug related to semicolon insertion
Paul Lancaster
committed
Added push-back() which seems to be required for parsing the different between (expr) and an arrow function
Paul Lancaster
committed
Fixed bug where line terminators were not ignored during object literal parsing
Paul Lancaster
committed
Fixed bug where line terminators were rejected within object literal definition
Paul Lancaster
committed
fixed bug where anonymous functions as function arguments could not be parsed
Paul Lancaster
committed
Fixed bug with line terminators being supported in WhileLoop
Paul Lancaster
committed
Removed need for special peek_semicolon with do_while flag
Paul Lancaster
committed
Added dowhile semicolon insertion
Paul Lancaster
committed
Fixed bug where line terminators were not allowed between parts of dowhile
Paul Lancaster
committed
Disabled 2 new lexer tests which didn't pass anyway
Paul Lancaster
committed
Cargo fmt
Paul Lancaster
committed
Cleanup
Paul Lancaster
committed
Fmt
Paul Lancaster
committed
Tidying
Paul Lancaster
committed
Merge pull request #19 from Lan2u/bug_fix
Paul Lancaster
authored
Starting to add goal symbol set statements
Paul Lancaster
committed

Commits on Jun 29, 2020

Added goal symbol set to expression parser macro
Paul Lancaster
committed
Trying to workout where to put goal symbols
Paul Lancaster
committed
Merge pull request #20 from Lan2u/new_parser_lexer
Paul Lancaster
authored

Commits on Jul 3, 2020

Added a 'fallback' mechanism to try and regex if a div is found unexpectadly - all regex tests currently pass
Paul Lancaster
committed
All tests pass
Paul Lancaster
committed
Removed println statements
Paul Lancaster
committed
Added tests showing handling of ambigious regex/div case
Paul Lancaster
committed
Fmt
Paul Lancaster
committed
Disabled the expression lexer benchmark
Paul Lancaster
committed
Downgraded some cursor pub fn to fn
Paul Lancaster
committed
Tidy up spread literal lex()
Paul Lancaster
committed
fixed a couple small warnings
Paul Lancaster
committed
Changed lexer peek() to return values rather than references
Paul Lancaster
committed
Handled peek error returns
Paul Lancaster
committed
Making clippy happier
Paul Lancaster
committed
Making clippy happier
Paul Lancaster
committed
Removed macro for chosing branch in lexer
Paul Lancaster
committed
Cleaning up, making clippy happier
Paul Lancaster
committed
The insatiable clippy continues
Paul Lancaster
committed
Fmt
Paul Lancaster
committed
Fmt, clippy
Paul Lancaster
committed
Making clippy happy... continues...
Paul Lancaster
committed

Commits on Jul 4, 2020

Tidying up
Paul Lancaster
committed
Clippy is happier, needed to force allow certain closures which were misreported as redundant
Paul Lancaster
committed
Tidying cursor, modify peek_more to peek_skip
Paul Lancaster
committed
Merged master into new lexer
Paul Lancaster
committed
Merge branch 'boa-dev-master' into new_lexer
Paul Lancaster
committed
fixed bug where old expect behaviour used on switch
Paul Lancaster
committed
Fmt, clippy happy, all tests pass
Paul Lancaster
committed
Attempting to fix issues with benchmarks
Paul Lancaster
committed
Fixed bug in array decl not allowing line terminators
Paul Lancaster
committed
fmt
Paul Lancaster
committed
Fixed bug with line terminators being rejected in object arguments
Paul Lancaster
committed

Commits on Jul 5, 2020

Update boa/src/syntax/lexer/number.rs

Paul Lancaster
and
HalidOdat
authored
Update boa/src/builtins/array/tests.rs

Paul Lancaster
and
HalidOdat
authored
Update boa/src/syntax/lexer/cursor.rs

Paul Lancaster
and
HalidOdat
authored
Update boa/src/syntax/parser/cursor.rs

Paul Lancaster
and
HalidOdat
authored
Update boa/src/syntax/parser/expression/assignment/mod.rs

Paul Lancaster
and
HalidOdat
authored
Update boa/src/syntax/lexer/cursor.rs

Paul Lancaster
and
HalidOdat
authored
Fixes as per PR review
Paul Lancaster
committed
Cleaning up number lexing
Paul Lancaster
committed
Tidying, removed .unwrap()'s
Paul Lancaster
committed
Tidy up based on PR reviews
Paul Lancaster
committed
Tidying
Paul Lancaster
committed
Used transpose as per PR comment
Paul Lancaster
committed
Removed error returns in number in favour of immediate panic
Paul Lancaster
committed
Started refractor for cursor to return Result<Option<>>
Paul Lancaster
committed
Number lexing change
Paul Lancaster
committed
Fixing unused result warnings
Paul Lancaster
committed
Finished refractor
Paul Lancaster
committed

Commits on Jul 7, 2020

Merge branch 'master' into new_lexer
Paul Lancaster
authored
Update boa/src/syntax/parser/expression/update.rs

Paul Lancaster
and
HalidOdat
authored
Fixed error preventing compilation
Paul Lancaster
committed
Added missing doc comments
Paul Lancaster
committed
Missing doc
Paul Lancaster
committed
Switched multiple operator spec links for a single parent link
Paul Lancaster
committed
Refractor of benchmark
Paul Lancaster
committed

Commits on Jul 8, 2020

Removed expression_lexer benchmark
Paul Lancaster
committed
Fixed comments consuming line terminator (with test)
Paul Lancaster
committed
Updated test to reflect change to single-line comments
Paul Lancaster
committed
Small changes as per PR review
Paul Lancaster
committed
Modified multiline comment behaviour to return line terminator if encountered
Paul Lancaster
committed
Update boa/src/syntax/lexer/comment.rs

Paul Lancaster
and
Razican
authored

Commits on Jul 9, 2020

Merge pull request #24 from boa-dev/master
Paul Lancaster
authored
Fixed goal symbol benchmark
Paul Lancaster
committed
Inline set/get goal
Paul Lancaster
committed

Commits on Jul 11, 2020

Merge branch 'master' into new_lexer
Paul Lancaster
authored
fmt
Paul Lancaster
committed

Commits on Jul 12, 2020

Merge branch 'master' into new_lexer
Paul Lancaster
authored

Commits on Jul 13, 2020

Commits on Jul 15, 2020

Peek_skip impl
Paul Lancaster
committed
Initial non-vec dequeue impl
Paul Lancaster
committed
Merge branch 'new_lexer' into new_lexer_no_heap_peek
Paul Lancaster
authored
Removed peeked_back field
Paul Lancaster
committed
Merge branch 'new_lexer_no_heap_peek' of https://github.com/boa-dev/boa into new_lexer_no_heap_peek
Paul Lancaster
committed
Merge pull request #566 from boa-dev/new_lexer_no_heap_peek
Paul Lancaster
authored
Fixed error where number > usize size was used
Paul Lancaster
committed

Commits on Jul 17, 2020

Converted peek to implicitly skip line terminators
Paul Lancaster
committed
Moving the work of skipping line terminators into the lexer
Paul Lancaster
committed
Replaced different methods in parser with a skip terminators option
Paul Lancaster
committed
Added argument to peek/next to skip line terminators
Paul Lancaster
committed
Made clippy happier
Paul Lancaster
committed
Peek now discards a line terminator (if flag set) even if one was already previously peeked
Paul Lancaster
committed
Removed some usages of peek/next with skip line terminator allowing more tests to pass
Paul Lancaster
committed
Bringing across HalidoDat's stackoverflow fixes
Paul Lancaster
committed
Bringing across HalidoDat's stackoverflow fixes (2/2)
Paul Lancaster
committed
Re-added rmeoved bit of number tests
Paul Lancaster
committed
Basically back to where I started
Paul Lancaster
committed

Commits on Jul 21, 2020

Fixed bug with parse generalised
Paul Lancaster
committed
All tests pass
Paul Lancaster
committed
Revert changes to test running
Paul Lancaster
committed
Removed skip_line_terminators
Paul Lancaster
committed
Comment
Paul Lancaster
committed
Comment
Paul Lancaster
committed
Removed an instance of try_parse
Paul Lancaster
committed
Removed another try_parse usage
Paul Lancaster
committed
Removed further try_parse usage
Paul Lancaster
committed
Removed binding init try_parse!
Paul Lancaster
committed
Fmt, clippy
Paul Lancaster
committed

Commits on Jul 22, 2020

Removed arrow function try_parse
Paul Lancaster
committed
try_parse removed
Paul Lancaster
committed
Removed debug statements
Paul Lancaster
committed
Merge pull request #584 from boa-dev/new_lexer_peek_explicit
Paul Lancaster
authored
Merge branch 'master' into new_lexer
Paul Lancaster
authored

Commits on Jul 24, 2020

Added test covering desired behaviour
Paul Lancaster
committed
Addressed line terminators within assignment expression
Paul Lancaster
committed
Syntax error now thrown on assignment to constant
Paul Lancaster
committed
Added array decl assignment tests + impl
Paul Lancaster
committed
Added assign to object test
Paul Lancaster
committed

Commits on Jul 25, 2020

Moved tests
Paul Lancaster
committed

Commits on Jul 26, 2020

Refractored tests to check for presence of 'Syntax Error:' rather than a specific Error message
Paul Lancaster
committed

Commits on Jul 29, 2020

Merge branch 'master' into new_lexer
Paul Lancaster
authored
fixed merge
Paul Lancaster
committed
Merge branch 'master' into new_lexer
Paul Lancaster
authored
fixed compliation error
Paul Lancaster
committed
Added test to refine the current problem in the parser
Paul Lancaster
committed
Added further test to demonstrate issue
Paul Lancaster
committed

Commits on Aug 1, 2020

Added test to differentiate between (b) and (a,b)
Paul Lancaster
committed
Expanded check to differentiate between (expr) and (a) => {}
Paul Lancaster
committed
Identified need to peek ahead upto 4 tokens
Paul Lancaster
committed
fmt, clippy
Paul Lancaster
committed

Commits on Aug 2, 2020

Switched to new mechanism allowing arbitary peek buffer size
Paul Lancaster
committed
Moved parser cursor to add cursor specific tests
Paul Lancaster
committed
Added some cursor specific tests
Paul Lancaster
committed
Added more cursor tests
Paul Lancaster
committed
Implementing skip already peeked line terminators + added comments
Paul Lancaster
committed
Further improved documentation of desired behaviour of peek_skip
Paul Lancaster
committed
Skip peeked terminators doc test passes
Paul Lancaster
committed
Peek_skip now works upto skip_n == 3
Paul Lancaster
committed
Removed peek_skip, replaced with a singular peek() function
Paul Lancaster
committed
Cleanup
Paul Lancaster
committed
Updated usages of peek_skip with peek in documentation
Paul Lancaster
committed
Replaced push_back in assignment with peek(2)
Paul Lancaster
committed
Fixed arrow function detection with empty args
Paul Lancaster
committed
All tests including new (expr) tests pass
Paul Lancaster
committed
Cleanup, removed debug statements
Paul Lancaster
committed
Removed push_back
Paul Lancaster
committed
Additional docs
Paul Lancaster
committed
Merge branch 'new_lexer' into issue_588
Paul Lancaster
authored
Refractor to seperate lexer buffering from rest of parse cursor
Paul Lancaster
committed
Make clippy happy
Paul Lancaster
committed
Merge pull request #593 from boa-dev/issue_588
Paul Lancaster
authored

Commits on Aug 3, 2020

Remove duplicated "Parsing Error: Syntax Error:" messages
Razican
committed

Commits on Aug 14, 2020

Update boa/src/syntax/parser/cursor/buffered_lexer/mod.rs
Razican
authored

Commits on Aug 16, 2020

Fixes for the new lexer (#632 )
Razican
authored
Merge branch 'master' into new_lexer
Paul Lancaster
authored
Fixed merge issue with .to_length()
Paul Lancaster
committed
Removed track_caller usage for now
Paul Lancaster
committed

Commits on Aug 17, 2020

Small size optimization
Razican
committed
Merge branch 'master' into new_lexer
Paul Lancaster
authored
Fixed merge issue
Paul Lancaster
committed

Commits on Aug 20, 2020

Commits on Aug 21, 2020

Commits on Aug 22, 2020

Fixed if statement unwrap()
Paul Lancaster
committed
Renamed take_while_pred
Paul Lancaster
committed
Removed unneeded comment
Paul Lancaster
committed
Cleaned up member, removed unneeded clone
Paul Lancaster
committed
Removed ability to dump tokens
Paul Lancaster
committed
Removed old comment
Paul Lancaster
committed
Removed old/no-longer true comments
Paul Lancaster
committed
Removed out-dated comment
Paul Lancaster
committed
Reenabled test which now passes
Paul Lancaster
committed
Added module level documentation to lexer
Paul Lancaster
committed
Removed unwrap in callexpression
Paul Lancaster
committed
Clarified function
Paul Lancaster
committed
Updated arrow_function to make it more clear that the (expr) case should have already been handled
Paul Lancaster
committed
Removed old comment
Paul Lancaster
committed
Catchup new_lexer (#657 )

authored
Fixed benchmarks
Paul Lancaster
committed
Merge branch 'master' into new_lexer
Paul Lancaster
authored
Fixed merge
Paul Lancaster
committed
Added position to lex syntax error
Paul Lancaster
committed

Commits on Aug 25, 2020

Added some position tests
Paul Lancaster
committed
Added syntax error position checks to tests
Paul Lancaster
committed

Commits on Aug 27, 2020

Update boa/src/syntax/lexer/cursor.rs

Paul Lancaster
and
Razican
authored
Update boa/src/syntax/lexer/tests.rs

Paul Lancaster
and
Razican
authored
fmt
Paul Lancaster
committed