PR: Select full floating point numbers by double-clicking them

[athompson673]

Improve double click to select functionality from default to allow selecting floating point numbers, etc by parsing the line with the tokenize stdlib module and selecting entire NUMBER tokens.

Description of Changes

• Wrote at least one-line docstrings (for any new functions)
• Added unit test(s) covering the changes (if testable)

• Included a screenshot or animation (if affecting the UI, see Licecap)

Issue(s) Resolved

Fixes #22207

Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: athompson673

[pep8speaks]

Hello @athompson673! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2024-11-01 17:18:51 UTC

[ccordoba12]

Thanks @athompson673 for opening your again PR against master!

I only left some minor style suggestions for you. In case you haven't used Github reviews before, you can submit all of them in a single commit by adding them to the batch, as described here.

[athompson673]

@ccordoba12 just ran into an issue with double clicking a docstring... I'll evaluate how to fix

I'm potentially trying to use tokenize in a way it is not intended (for a single line rather than an entire document)

[athompson673]

I figured after finding and solving that last issue, it would be a good idea to try to actually confirm there wouldn't be any more hidden problems. I wasn't sure how to add actual tests for automated testing, but I broke out some of the logic of the newly edited function, and ran a test against every possible double click position in all the .py files in a spyder 6.0.1 installation. No errors raised:

from pathlib import Path

from io import StringIO
from token import NUMBER
from tokenize import generate_tokens, TokenError

from time import perf_counter

def get_number_selection(line: str, pos: int) -> tuple[int, int]|tuple[None, None]:
    """
    Return start and end position of number literal at pos in line.
    If text at pos in line is not a number literal, return (None, None)
    """
    # Strip quotes to prevent tokenizer from trying to emit STRING tokens
    #   because we want to interpret numbers inside strings. This solves
    #   an EOF error trying to double click a line with opening or closing
    #   triple quotes as well.
    text = line.replace('"', ' ').replace("'", ' ')
    readline = StringIO(text).read
    
    try:
        for t_type, _, start, end, _ in generate_tokens(readline):
            if t_type == NUMBER and start[1] <= pos <= end[1]:
                return start[1], end[1]
            elif start[1] > pos:
                break

    except TokenError:
        # Ignore 'EOF in multi-line statement' from tokenize._tokenize
        # IndentationError should be impossible from tokenizing one line
        pass
    
    return None, None  # start and end of NUMBER not found

count = 0
start_time = perf_counter()
for p in Path(r'D:\python\py311\Lib\site-packages\spyder').glob("**/*.py"):
    with p.open(encoding="utf-8") as f:
        print(p)
        for line in f:
            for i in range(len(line)):
                start, end = get_number_selection(line, i)
                if start is None:
                    assert end is None
                elif end is None:
                    raise ValueError("1")
                else:
                    assert type(start) is int
                    assert type(end) is int
                    assert start < end
                    assert end <= len(line)
                count += 1
print("\n{count}, {perf_counter() - start_time}")

>>> %runfile D:/scripts/untitled0.py --wdir --post-mortem
5320059, 81.67973330011591
    

It also shows the function shouldn't slow down double click handling appreciably; about 15μs per call for me.

[ccordoba12]

I wasn't sure how to add actual tests for automated testing, but I broke out some of the logic of the newly edited function, and ran a test against every possible double click position in all the .py files in a spyder 6.0.1 installation. No errors raised

That's great! Thanks for testing that @athompson673.

I'll give you a hand to implement a simple test for the new behavior you added, but just for the IPython console because it should be the same for the editor.

[ccordoba12]

Thanks for your work on this @athompson673!! It's a really cool contribution!

[ccordoba12]

Note: This will be included in 6.1 (to be released in six months or so) because it's a new feature.