Modify comment_ranges slice in BackwardsTokenizer

[charliermarsh]

Summary

I was kinda curious to understand this issue (#7426) and just ended up attempting to address it.

Test Plan

cargo test

[github-actions]

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

[MichaReiser]

I must have missed this in the previous PR but it seems that the lexer now always initializes with after_newline. The lexer used to have a special up_to method indicating that it is guaranteed not to be after a newline to avoid testing for comments.

I'm bringing this up here because the downside is that we now always perform a binary search to find the last comment, even if the caller can guarantee that the start position isn't after a newline. Bringing back this optimization is probably worth its own PR

[konstin]

i didn't implement this initially because i thought about doing the binary search esp for large files with many comments is expensive to do for every is-expression-parenthesized check, but it doesn't seem to show in our benchmarks. Removing this is also kinda ugly because we have to bring back after_newline

[codspeed-hq]

CodSpeed Performance Report

Merging #7432 will improve performances by 3%

_{Comparing charlie/slice (ce61771) with main (aae02cf)}

Summary

⚡ 1 improvements
✅ 24 untouched benchmarks

Benchmarks breakdown

	Benchmark	main	charlie/slice	Change
⚡	linter/all-rules[numpy/ctypeslib.py]	35.1 ms	34 ms	+3%

[konstin]

thanks!