Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns. #108657

Merged
merged 7 commits into from
Aug 30, 2023

Conversation

CorvinM
Copy link
Contributor

@CorvinM CorvinM commented Aug 29, 2023

Fixes an exception where sqlite3.iterdump would fail to decode if someone stuffed invalid unicode into a database column marked as TEXT (or alias of, like VARCHAR).
See #108590 for more info

@CorvinM CorvinM marked this pull request as ready for review August 29, 2023 22:49
Lib/sqlite3/dump.py Outdated Show resolved Hide resolved
Lib/sqlite3/dump.py Outdated Show resolved Hide resolved
CorvinM and others added 3 commits August 30, 2023 03:41
…k0pOl.rst

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Lib/sqlite3/dump.py Outdated Show resolved Hide resolved
Copy link
Contributor

@erlend-aasland erlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this, Corvin; it is a nice solution to this corner case.

I'll make some final adjustments before landing.

@erlend-aasland erlend-aasland enabled auto-merge (squash) August 30, 2023 08:28
@erlend-aasland erlend-aasland added needs backport to 3.11 only security fixes needs backport to 3.12 bug and security fixes labels Aug 30, 2023
@erlend-aasland erlend-aasland linked an issue Aug 30, 2023 that may be closed by this pull request
2 tasks
@erlend-aasland
Copy link
Contributor

@CorvinM, you may also be interested in looking at #108364

@CorvinM
Copy link
Contributor Author

CorvinM commented Aug 30, 2023

Thank you for the speedy reviews and helpful feedback 😄 Let me know if you want me to pick up anything else with this (e.g., backports)

@erlend-aasland erlend-aasland merged commit 400a1ce into python:main Aug 30, 2023
19 checks passed
@miss-islington

This comment was marked as outdated.

@miss-islington
Copy link
Contributor

Sorry, @CorvinM and @erlend-aasland, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 400a1cebc743515e40157ed7af86e48d654290ce 3.12

@miss-islington
Copy link
Contributor

Sorry, @CorvinM and @erlend-aasland, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 400a1cebc743515e40157ed7af86e48d654290ce 3.11

@erlend-aasland
Copy link
Contributor

@CorvinM, can you handle the backports? You can use the cherry-picker. See the devguide.

@CorvinM
Copy link
Contributor Author

CorvinM commented Aug 30, 2023

@CorvinM, can you handle the backports? You can use the cherry-picker. See the devguide.

Sure

CorvinM added a commit to CorvinM/cpython that referenced this pull request Aug 30, 2023
…EXT columns (pythonGH-108657)

Co-authored-by: Erlend E. Aasland <erlend@python.org>.
(cherry picked from commit 400a1ce)

Co-authored-by: Corvin <corvin@corvin.dev>
@bedevere-bot
Copy link

GH-108673 is a backport of this pull request to the 3.12 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 bug and security fixes label Aug 30, 2023
CorvinM added a commit to CorvinM/cpython that referenced this pull request Aug 30, 2023
…EXT columns (pythonGH-108657)

Co-authored-by: Erlend E. Aasland <erlend@python.org>.
(cherry picked from commit 400a1ce)

Co-authored-by: Corvin <corvin@corvin.dev>
@bedevere-bot
Copy link

GH-108674 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Aug 30, 2023
erlend-aasland pushed a commit that referenced this pull request Aug 30, 2023
erlend-aasland added a commit to erlend-aasland/cpython that referenced this pull request Aug 30, 2023
erlend-aasland added a commit that referenced this pull request Aug 30, 2023
Reverted per Serhiy's request.
carljm added a commit to carljm/cpython that referenced this pull request Aug 30, 2023
* main:
  pythongh-108520: Fix bad fork detection in nested multiprocessing use case (python#108568)
  pythongh-108590: Revert pythongh-108657 (commit 400a1ce) (python#108686)
  pythongh-108494: Argument Clinic: Document how to generate code that uses the limited C API (python#108584)
  Document Python build requirements (python#108646)
  pythongh-101100: Fix Sphinx warnings in the Logging Cookbook (python#108678)
  Fix typo in multiprocessing docs (python#108666)
  pythongh-108669: unittest: Fix documentation for TestResult.collectedDurations (python#108670)
  pythongh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (python#108657)
  Revert "pythongh-103224: Use the realpath of the Python executable in `test_venv` (pythonGH-103243)" (pythonGH-108667)
  pythongh-106320: Remove private _Py_ForgetReference() (python#108664)
  Mention Ellipsis pickling in the docs (python#103660)
  Revert "Use non alternate name for Kyiv (pythonGH-108533)" (pythonGH-108649)
  pythongh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (python#108632)
  pythongh-108455: peg_generator: install two stubs packages before running mypy (python#108637)
  pythongh-107801: Improve the accuracy of io.IOBase.seek docs (python#108268)
erlend-aasland added a commit to erlend-aasland/cpython that referenced this pull request Aug 30, 2023
…on#108686)

(cherry picked from commit 2a3926f)

Reverted per Serhiy's request.
erlend-aasland added a commit that referenced this pull request Aug 30, 2023
(cherry picked from commit 2a3926f)

Reverted per Serhiy's request.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sqlite3.iterdump() incompatible with binary data
5 participants