Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automatically remove duplicate dictionary keys #1710

Merged
merged 2 commits into from
Jan 7, 2023
Merged

Automatically remove duplicate dictionary keys #1710

merged 2 commits into from
Jan 7, 2023
+1,094 −77

Conversation

charliermarsh
Copy link
Member

@charliermarsh charliermarsh commented Jan 7, 2023 •
edited
Loading

For now, to be safe, we're only removing keys with duplicate values.

See: #1647.

@andersk
Copy link
Contributor

andersk commented Jan 7, 2023

This is incorrect:

-d = {"b": 2, "a": 1, "b": 2}
+d = {"b": 2, "a": 2}

charliermarsh
charliermarsh commented Jan 7, 2023
View reviewed changes
src/pyflakes/plugins/repeated_keys.rs Outdated
values[j - 1].end_location.unwrap(),
values[j].end_location.unwrap(),
)
});
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can maybe be reduced to a single case? But it's 12:30 so I'm gonna revisit in the morning.

andersk
andersk reviewed Jan 7, 2023
View reviewed changes
src/pyflakes/plugins/repeated_keys.rs Outdated
Comment on lines 31 to 33
for i in 0..num_keys {
let k1: Option<DictionaryKey> = (&keys[i]).try_into().ok();
for j in i + 1..num_keys {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This quadratic loop could be slow on a large dictionary. A HashMap would avoid that problem.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literally a leetcode problem and I failed it!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andersk - Do you think it's important to avoid removing duplicate keys with non-identical values?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do, it's hard to avoid quadratic time in pathological cases because values aren't hashable. As an alternative, I could stringify them.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did start on that, but f64 itself isn't hashable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can hash via f64::to_bits.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm ok, cool. I'll try that out. (Above, you're suggesting that ComparableExpr compares by ignoring the Location fields, right?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will come in a separate PR.

@charliermarsh charliermarsh force-pushed the charlie/duplicate-keys branch 2 times, most recently from c485f76 to eac452d Compare January 7, 2023 17:14
@charliermarsh charliermarsh merged commit 4de6c26 into main Jan 7, 2023
@charliermarsh charliermarsh deleted the charlie/duplicate-keys branch January 7, 2023 21:16
renovate bot added a commit to ixm-one/pytest-cmake-presets that referenced this pull request Jan 8, 2023
@renovate
⬆️ Update dependency ruff to ^0.0.215 (#26)
631cabe 
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://github.com/charliermarsh/ruff) | `^0.0.214` ->
`^0.0.215` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.215/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.215/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.215/compatibility-slim/0.0.214)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.215/confidence-slim/0.0.214)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.215`](https://github.com/charliermarsh/ruff/releases/tag/v0.0.215)

[Compare
Source](https://github.com/charliermarsh/ruff/compare/v0.0.214...v0.0.215)

#### What's Changed

- Automatically remove duplicate dictionary keys by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[astral-sh/ruff#1710
- Add `ComparableExpr` hierarchy for comparing expressions by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[astral-sh/ruff#1721
- Respect isort:skip action comment by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[astral-sh/ruff#1722
- Treat failures to fix TypedDict conversions as debug logs by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[astral-sh/ruff#1723
- Implement `--isolated` CLI flag by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[astral-sh/ruff#1727
- Add more unittest assert methods to PT009 by
[@&#8203;harupy](https://github.com/harupy) in
[astral-sh/ruff#1730
- Add `RUFF_FORMAT` environment variable support by
[@&#8203;messense](https://github.com/messense) in
[astral-sh/ruff#1731
- Move RUFF_CACHE_DIR to Clap's env support by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[astral-sh/ruff#1733
- buf-fix: flake8\_simplify SIM212 by
[@&#8203;chammika-become](https://github.com/chammika-become) in
[astral-sh/ruff#1732
- Remove `assertNotContains` by
[@&#8203;harupy](https://github.com/harupy) in
[astral-sh/ruff#1729

**Full Changelog**:
astral-sh/ruff@v0.0.214...v0.0.215

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/ixm-one/pytest-cmake-presets).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC44NC4yIiwidXBkYXRlZEluVmVyIjoiMzQuODQuMiJ9-->

Signed-off-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andersk andersk andersk left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@charliermarsh @andersk