Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update logical line rules for the new f-string tokens #7690

Merged
merged 2 commits into from
Sep 28, 2023

Conversation

dhruvmanila
Copy link
Member

@dhruvmanila dhruvmanila commented Sep 28, 2023

Summary

This PR updates the logical line rules for the new f-string tokens.

Specifically, it tries to ignore the rules for which it might be hard to differentiate between different usages inside f-strings. For some tokens, the whitespace addition or removal could change the semantics so they're ignored for every scenario inside f-string.

  • Ignore E225 for the = operator. For example, f"{x=}" we don't want to add whitespace as the output will then be different than what the user indended.
  • Ignore E231 for the : operator. For example, f"{x:.3f}" we don't want to add whitespace as the output will be different:
In [1]: a = 100

In [2]: f'{a: .3f}'
Out[2]: ' 100.000'

In [3]: f'{a:.3f}'
Out[3]: '100.000'
  • Ignore E201, E202 for { and } tokens. The double braces are used as an escape mechanism to include the raw character (f"{{foo}}") but if a dictionary is being used then the whitespace is required (f"{ {'a': 1} }").

Test Plan

Add new test cases for f-strings along with nested f-strings and update the snapshots.

1 similar comment
@dhruvmanila dhruvmanila linked an issue Sep 28, 2023 that may be closed by this pull request
3 tasks
@dhruvmanila dhruvmanila marked this pull request as ready for review September 28, 2023 04:06
@dhruvmanila dhruvmanila added rule Implementing or modifying a lint rule python312 Related to Python 3.12 labels Sep 28, 2023
continue;
}
_ => {}
}
if let Some(symbol) = BracketOrPunctuation::from_kind(kind) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just skip all of this when we're inside an f-string?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, or actually, we do want to catch some errors within f-strings -- is that right?

Copy link
Member Author

@dhruvmanila dhruvmanila Sep 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing whitespaces before the punctuations should be fine. It's only when they're removed after when the semantics changes.

So, f"{foo :.3f}" and f"{foo:.3f}" are equivalent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, or actually, we do want to catch some errors within f-strings -- is that right?

Yes

@codspeed-hq
Copy link

codspeed-hq bot commented Sep 28, 2023

CodSpeed Performance Report

Merging #7690 will degrade performances by 2.28%

Comparing dhruv/fstring-logical-line-rules (11e27da) with dhruv/pep-701 (d60f30e)

Summary

❌ 1 regressions
✅ 24 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark dhruv/pep-701 dhruv/fstring-logical-line-rules Change
linter/default-rules[large/dataset.py] 93.4 ms 95.6 ms -2.28%

let needs_space = if kind == TokenKind::Equal && parens > 0 {
// Allow keyword args or defaults: foo(bar=None).
let needs_space = if kind == TokenKind::Equal && (parens > 0 || fstrings > 0) {
// Allow keyword args, defaults: foo(bar=None) and f-strings: f'{foo=}'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, should we just have something like:

if fstrings > 0 {
    continue;
}

Or is it important that we run this body logic?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so we're trying to catch some of these errors within f-strings, but not all -- is that right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's only when the equal sign is present which in f-string context will be used as debug expression. So, f"{x=} and f"{x = }" gives different output.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 28, 2023

PR Check Results

Ecosystem

ℹ️ ecosystem check detected changes. (+0, -105, 0 error(s))

rotki (+0, -37)

- rotkehlchen/accounting/structures/base.py:148:37: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:149:35: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:150:30: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:151:29: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:152:31: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:153:34: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:154:26: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:155:28: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:156:35: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:157:26: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/base.py:158:31: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/eth2.py:139:58: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/eth2.py:139:77: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/eth2.py:265:139: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/eth2.py:265:53: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/eth2.py:265:72: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/eth2.py:394:55: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/eth2.py:394:74: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/eth2.py:394:91: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/evm_event.py:261:28: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/evm_event.py:262:33: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/evm_event.py:263:28: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/evm_event.py:264:28: E225 Missing whitespace around operator
- rotkehlchen/accounting/structures/evm_event.py:265:31: E225 Missing whitespace around operator
- rotkehlchen/chain/ethereum/modules/aave/v2/decoder.py:175:126: E231 [*] Missing whitespace after ':'
- rotkehlchen/fval.py:159:31: E231 [*] Missing whitespace after ':'
- rotkehlchen/fval.py:49:27: E231 [*] Missing whitespace after ':'
- rotkehlchen/tests/conftest.py:140:47: E231 [*] Missing whitespace after ':'
- rotkehlchen/tests/unit/test_defi_oracles.py:44:102: E225 Missing whitespace around operator
- rotkehlchen/tests/unit/test_defi_oracles.py:44:84: E225 Missing whitespace around operator
- tools/profiling/cpu.py:12:35: E231 [*] Missing whitespace after ':'
- tools/profiling/cpu.py:13:33: E231 [*] Missing whitespace after ':'
- tools/profiling/cpu.py:30:36: E231 [*] Missing whitespace after ':'
- tools/profiling/graph.py:219:33: E231 [*] Missing whitespace after ':'
- tools/profiling/sampler.py:64:30: E231 [*] Missing whitespace after ':'
- tools/profiling/sampler.py:64:43: E231 [*] Missing whitespace after ':'
- tools/profiling/trace.py:26:28: E231 [*] Missing whitespace after ':'

sphinx (+0, -68)

- 
-      = help: Added missing whitespace after ':'
-      |
-      |                                                                                   ^ E231
-     = help: Added missing whitespace after ':'
-     |
-     |                                                                    ^ E231
-     |                                                      ^ E231
-     |                                                   ^ E231
-     |                                   ^ E231
-     |                                  ^ E231
-     |                                 ^ E231
-    = help: Added missing whitespace after ':'
-    |
-    |                                                                         ^ E231
-    |                                                                 ^ E231
-    |                                                         ^ E231
-    |                                                 ^ E231
-    |                                 ^ E231
-    |                         ^ E231
- 1202 |     """Return an RFC 3339 formatted string representing the given timestamp."""
- 1203 |     seconds, fraction = divmod(timestamp, 1)
- 1204 |     return time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(seconds)) + f'.{fraction:.3f}'
- 130 |     if total <= 0:
- 131 |         return _('no translated elements!')
- 132 |     return f'{translated / total:.2%}'
- 144 |     if not content:
- 145 |         return ''
- 146 |     return f'{zlib.crc32(content):08x}'
- 22 |     weekday_name = _WEEKDAY_NAME[wd]
- 23 |     month = _MONTH_NAME[mn]
- 24 |     return f'{weekday_name}, {dd:02} {month} {yr:04} {hh:02}:{mm:02}:{ss:02} GMT'
- 282 |             res = r"%.5f\linewidth" % (amount_float / 100.0)
- 283 |         else:
- 284 |             res = f"{amount_float:.5f}{unit}"
- 285 |     return res
- 293 |         table.append([
- 294 |             'TOTAL',
- 295 |             f'{100 * len(all_documented_objects) / len(all_objects):.2f}%',
- 296 |             f'{len(all_objects) - len(all_documented_objects)}',
- 297 |         ])
- 341 |         c = f'')
- 345 |     raise nodes.SkipNode
- 725 |             for entry, (_proj, _ver, url_path, display_name) in inv_entries:
- 726 |                 display_name = display_name * (display_name != '-')
- 727 |                 print(f'    {entry:<40} {display_name:<40}: {url_path}')
- 728 |     except ValueError as exc:
- 729 |         print(exc.args[0] % exc.args[1:], file=sys.stderr)
- 76 |     logger.info(__('====================== slowest reading durations ======================='))
- 77 |     for docname, d in islice(durations, 5):
- 78 |         logger.info(f'{d:.3f} {docname}')  # NoQA: G004
- sphinx/builders/html/__init__.py:1204:83: E231 Missing whitespace after ':'
- sphinx/builders/html/_assets.py:146:34: E231 Missing whitespace after ':'
- sphinx/ext/coverage.py:295:68: E231 Missing whitespace after ':'
- sphinx/ext/duration.py:78:25: E231 Missing whitespace after ':'
- sphinx/ext/imgmath.py:343:51: E231 Missing whitespace after ':'
- sphinx/ext/intersphinx.py:727:35: E231 Missing whitespace after ':'
- sphinx/ext/intersphinx.py:727:54: E231 Missing whitespace after ':'
- sphinx/transforms/__init__.py:132:33: E231 Missing whitespace after ':'
- sphinx/util/http_date.py:24:33: E231 Missing whitespace after ':'
- sphinx/util/http_date.py:24:49: E231 Missing whitespace after ':'
- sphinx/util/http_date.py:24:57: E231 Missing whitespace after ':'
- sphinx/util/http_date.py:24:65: E231 Missing whitespace after ':'
- sphinx/util/http_date.py:24:73: E231 Missing whitespace after ':'
- sphinx/writers/latex.py:284:34: E231 Missing whitespace after ':'

Rules changed: 3
Rule Changes Additions Removals
E225 26 0 26
E231 25 0 25
G004 1 0 1

Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me -- lets just make sure we're setting prev_token in any of those continue blocks.

@dhruvmanila
Copy link
Member Author

This looks good to me -- lets just make sure we're setting prev_token in any of those continue blocks.

Thanks! I'll look at that in a bit

@charliermarsh
Copy link
Member

How did you identify this specific subset of logical line rules? Did they show up in the ecosystem CI on the PEP 701 PR?

@dhruvmanila
Copy link
Member Author

dhruvmanila commented Sep 28, 2023

How did you identify this specific subset of logical line rules? Did they show up in the ecosystem CI on the PEP 701 PR?

Yes, then I thought of checking out all the relevant operators and trivia tokens specific to f-string usages.

@dhruvmanila dhruvmanila merged commit f9aa07e into dhruv/pep-701 Sep 28, 2023
15 of 16 checks passed
@dhruvmanila dhruvmanila deleted the dhruv/fstring-logical-line-rules branch September 28, 2023 05:22
@dhruvmanila dhruvmanila restored the dhruv/fstring-logical-line-rules branch September 29, 2023 02:39
@dhruvmanila dhruvmanila deleted the dhruv/fstring-logical-line-rules branch September 29, 2023 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python312 Related to Python 3.12 rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update logical line rules for the new f-string tokens
2 participants