Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return a marker token at the end of any buffer #842

Merged
merged 4 commits into from
Apr 20, 2021

Conversation

Rangi42
Copy link
Contributor

@Rangi42 Rangi42 commented Apr 19, 2021

Removes the lexer hack mentioned in #778

@Rangi42 Rangi42 added bug Unexpected behavior / crashes; to be fixed ASAP! rgbasm This affects RGBASM labels Apr 19, 2021
@Rangi42
Copy link
Contributor Author

Rangi42 commented Apr 19, 2021

An INCLUDE at the end of a file, without a newline after it, was previously relying on the newline-at-EOF hack. Without that, include has to be a line_directive, so the included file contents will not be tokenized until after seeing the T_EOB.

This fix will allow #716 to be mergable since it needs the EOF hack removed to allow [[ inline fragments ]] without newlines.

@Rangi42
Copy link
Contributor Author

Rangi42 commented Apr 19, 2021

This also now allows a label in front of break, since (unlike most line_directives, REPT, IF, etc) there's no reason to prevent it.

@Rangi42 Rangi42 requested a review from ISSOtm April 19, 2021 00:36
@Rangi42 Rangi42 added this to the v0.5.1 milestone Apr 19, 2021
@Rangi42
Copy link
Contributor Author

Rangi42 commented Apr 19, 2021

The unterminated flag for reading REPT/FOR/MACRO contexts is also necessary because previously, the lack of a hacked T_NEWLINE was what stopped unterminated contexts from being evaluated. Now that T_EOB appears anyway, they need an explicit check (which also causes a more sensible error message for the nested-macrodef test case).

@Rangi42 Rangi42 added refactoring This PR is intended to clean up code more than change functionality and removed bug Unexpected behavior / crashes; to be fixed ASAP! labels Apr 19, 2021
src/asm/lexer.c Outdated Show resolved Hide resolved
src/asm/parser.y Outdated Show resolved Hide resolved
src/asm/parser.y Show resolved Hide resolved
Rangi42 added 2 commits April 19, 2021 16:54
Removes the lexer hack mentioned in gbdev#778
Bison 3.1 introduces "typed midrule values", which would write
`<captureTerminated>{ ... }` and `$$` instead of `{ ... }` and
`$<captureTerminated>[1-9]`, but rgbds supports 3.0 or even lower.
src/asm/lexer.c Outdated Show resolved Hide resolved
src/asm/lexer.c Outdated Show resolved Hide resolved
src/asm/lexer.c Outdated Show resolved Hide resolved
Rangi42 added 2 commits April 20, 2021 09:25
This avoids the need for a separate `terminated` flag
`yylex` calls `yywrap` at the beginning of the next call, after it
has set `lexerState->lastToken` to `T_EOB`.
@Rangi42 Rangi42 requested a review from ISSOtm April 20, 2021 14:37
@ISSOtm ISSOtm merged commit 8230e81 into gbdev:master Apr 20, 2021
@Rangi42 Rangi42 deleted the end-of-buffer branch April 20, 2021 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactoring This PR is intended to clean up code more than change functionality rgbasm This affects RGBASM
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants