Fixing Issue 186, scanner should account for newline characters when … #210
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…processing multi-line text. Without this source annotations line/column number (for this and all subsequent tokens) is inconsistent with plain text editors. e.g. #186. This addresses the issue specifically for single and double quote text only.
1.) We would have preferred to avoid introducing an additional state in the loop (isNewLine), but this requires placing the column marker to an invalid position (0) and believe it is safer not to do so. This is done to avoid potential bugs in the future if the loop exits before the column marker can be brought back in-bounds.
2.) We debated internally whether the slice sorting in the tests should be done through a helper function or not. We didn't feel a convenience function improved code readability to merit consolidation of the logic. If you disagree, lmk. We add the sort to be token order agnostic in our tests.