Twine progress bar permanently clobbers the tty's cursor character

[CAM-Gerlach]

Your Environment

1. Your operating system:

Windows 8.1 x64

2. Version of python you are running:

$ python --version
Python 3.9.12

3. How did you install twine? Did you use your operating system's package manager or pip or something else?

Tried both Conda-Forge and pip. Running in Git Bash (MSYS2) or legacy cmd.exe. All same result.

4. Version of twine you have installed (include complete output of):

$ twine --version
twine version 0.0.0 (importlib-metadata: 4.11.4, keyring: 23.5.1, pkginfo: 1.8.2, requests: 2.27.1, requests-
toolbelt: 0.9.1, urllib3: 1.26.9)

I'm using Twine 4.0.0 as confirmed via cconda list; Twine's version isn't getting written correctly to the standard core metadata, probably due to the use of a SCM-based version tracking system combined with the conda recipe not handling it correctly. In any case, appears to be orthogonal to the problem at hand.

5. Which package repository are you targeting?

PyPI

If you're having issues uploading a specific package, you must include a copy of the following:

• The package's PKG-INFO file
• A redacted version of your .pypirc file (REMOVE ALL USERNAMES & PASSWORDS BEFORE UPLOADING)

N/A, happens on any package I try to upload

The Issue

Please describe the issue that you are experiencing.

Running twine upload dist/* without the --disable-progress-bar option clobbers the tty's current cursor and sets it to a cyan-colored block instead. This occurs both in Git Bash (MSYS2) and in legacy cmd.exe (with ConEmu as the terminal emulator) and regardless of whether the upload succeeds or fails. After trying a bunch of things with no resent and even a full reset, the only way to get it back to normal is to fully close and re-open (or restart) the shell,

This means means the new progress bar functionality is useless for me and I have to always remember to pass --disable-progress-bar, or I loose my entire session and history (typically with extensive scrollback and previous commands) in the current tab as I have to restart it to get my cursor back. This is highly disruptive, given it not only affects Twine but the entire shell session. Fortunately, there's a workaround with relatively modest cost, so long as I remember to use it, but its still rather annoying and I lost a huge amount of scrollback and history the first time I ran into it, as well as a ton of time attempting to reverse it to no avail.

Steps to Reproduce

If the issue is predictable and consistently reproducible, please list the steps here.

1. Run `twine upload dist/*
2. Issue occurs

[bhrutledge]

Can you try twine --no-color upload dist/*?

We haven't received any other reports of progress bar issues on Windows, so I suspect the underlying issue has to do with Rich (the library used for the progress bar) and your terminal environment.

Can you try installing Rich into a fresh virtual environment, and experimenting with standalone progress bars?

https://rich.readthedocs.io/en/stable/progress.html

[CAM-Gerlach]

Thanks for the quick response!

Can you try twine --no-color upload dist/*?

Same result—but color is still shown, so that option doesn't appear to make any difference.

We haven't received any other reports of progress bar issues on Windows, so I suspect the underlying issue has to do with Rich (the library used for the progress bar) and your terminal environment.

Indeed, I assumed it had something to do with Rich, but in my initial testing I hadn't been able to isolate a specific factor as to why this might be occurring in my environment but not others, given it was equally reproducible on Git Bash and legacy Windows cmd running under ConEmu (which are popular, mainstream environments), and also cmd.exe and Git bash running under the built-in legacy Windows Command Prompt, and pip also uses Rich and I've never seen the same issue until now.

Can you try installing Rich into a fresh virtual environment, and experimenting with standalone progress bars?

I had just created the environment and performed a fresh solve to update all packages to the latest just minutes before, though it did have some other packages installed. For completeness, I've tested a newly-created environment with just Twine installed via conda (CF), and the behavior was the same.

Conda install details

$ conda create -n twine-test-env python=3.9 twine

[snip]

$ conda activate twine-test-env

$ conda list
# packages in environment at C:\Miniconda3\envs\twine-test-env:
#
# Name                    Version                   Build  Channel
bleach                    5.0.0              pyhd8ed1ab_0    conda-forge
brotlipy                  0.7.0           py39hb82d6ee_1004    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2022.5.18.1          h5b45459_0    conda-forge
certifi                   2022.5.18.1      py39hcbf5309_0    conda-forge
cffi                      1.15.0           py39h0878f49_0    conda-forge
charset-normalizer        2.0.12             pyhd8ed1ab_0    conda-forge
cmarkgfm                  0.8.0            py39hb82d6ee_1    conda-forge
commonmark                0.9.1                      py_0    conda-forge
cryptography              37.0.2           py39h7bc7c5c_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
docutils                  0.18.1           py39hcbf5309_1    conda-forge
future                    0.18.2           py39hcbf5309_5    conda-forge
idna                      3.3                pyhd8ed1ab_0    conda-forge
importlib-metadata        4.11.4           py39hcbf5309_0    conda-forge
importlib_metadata        4.11.4               hd8ed1ab_0    conda-forge
keyring                   23.5.1           py39hcbf5309_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libzlib                   1.2.12               h8ffe710_0    conda-forge
openssl                   1.1.1o               h8ffe710_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pip                       22.1.2             pyhd8ed1ab_0    conda-forge
pkginfo                   1.8.2              pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pygments                  2.12.0             pyhd8ed1ab_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1            py39hcbf5309_5    conda-forge
python                    3.9.13          h9a09f29_0_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
pywin32-ctypes            0.2.0           py39hcbf5309_1005    conda-forge
readme_renderer           35.0               pyhd8ed1ab_0    conda-forge
requests                  2.27.1             pyhd8ed1ab_0    conda-forge
requests-toolbelt         0.9.1                      py_0    conda-forge
rfc3986                   2.0.0              pyhd8ed1ab_0    conda-forge
rich                      12.4.4             pyhd8ed1ab_0    conda-forge
setuptools                62.3.2           py39hcbf5309_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.38.5               h8ffe710_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
twine                     4.0.0              pyhd8ed1ab_0    conda-forge
typing_extensions         4.2.0              pyha770c72_1    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
urllib3                   1.26.9             pyhd8ed1ab_0    conda-forge
vc                        14.2                 hb210afc_6    conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0            py39hcbf5309_4    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
zipp                      3.8.0              pyhd8ed1ab_0    conda-forge

$ twine --version
twine version 0.0.0 (importlib-metadata: 4.11.4, keyring: 23.5.1, pkginfo: 1.8.2, requests: 2.27.1, requests-
toolbelt: 0.9.1, urllib3: 1.26.9)

$ twine upload dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Enter your username: __token__
Enter your password:
Uploading serviceinstaller-0.1.4-py3-none-any.whl
100% ---------------------------------------- 9.9/9.9 kB • 00:00 • ?
WARNING  Error during upload. Retry with the --verbose option for more details.
ERROR    HTTPError: 403 Forbidden from https://upload.pypi.org/legacy/
         Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for more
         information.
$ # Block cursor now

HOWEVER, while I've never seen it before (perhaps simply not using pip enough on my Windows machine with uncached non-local packages), when I installed Twine with the latest pip from scratch, I actually saw the issue with pip too, which leads me to believe it is presumably with Rich and nothing Twine-specific.

Pip install details

$ conda create -n twine-test-env python=3.9
Collecting package metadata (current_repodata.json): done
Solving environment: done


[snip]

C. A. M. Gerlach@QUASIMODO MINGW64 ~/Documents/Bitzer/dev/serviceinstaller (fix-packaging)
$ conda activate twine-test-env
(twine-test-env)
C. A. M. Gerlach@QUASIMODO MINGW64 ~/Documents/Bitzer/dev/serviceinstaller (fix-packaging)
$ pip install twine
Collecting twine
  Downloading twine-4.0.1-py3-none-any.whl (36 kB)
Collecting rfc3986>=1.4.0
  Downloading rfc3986-2.0.0-py2.py3-none-any.whl (31 kB)
Collecting requests-toolbelt!=0.9.0,>=0.8.0
  Downloading requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
     ---------------------------------------- 54.3/54.3 kB 2.8 MB/s eta 0:00:00
Collecting urllib3>=1.26.0
  Downloading urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
     ---------------------------------------- 139.0/139.0 kB 8.6 MB/s eta 0:00:00
Collecting importlib-metadata>=3.6
  Downloading importlib_metadata-4.11.4-py3-none-any.whl (18 kB)
Collecting pkginfo>=1.8.1
  Downloading pkginfo-1.8.2-py2.py3-none-any.whl (26 kB)
Collecting rich>=12.0.0
  Downloading rich-12.4.4-py3-none-any.whl (232 kB)
     ---------------------------------------- 232.0/232.0 kB 13.9 MB/s eta 0:00:00
Collecting readme-renderer>=35.0
  Downloading readme_renderer-35.0-py3-none-any.whl (14 kB)
Collecting requests>=2.20
  Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)
     ---------------------------------------- 63.1/63.1 kB ? eta 0:00:00
Collecting keyring>=15.1
  Downloading keyring-23.5.1-py3-none-any.whl (33 kB)
Collecting zipp>=0.5
  Downloading zipp-3.8.0-py3-none-any.whl (5.4 kB)
Collecting pywin32-ctypes!=0.1.0,!=0.1.1
  Downloading pywin32_ctypes-0.2.0-py2.py3-none-any.whl (28 kB)
Collecting Pygments>=2.5.1
  Downloading Pygments-2.12.0-py3-none-any.whl (1.1 MB)
     ---------------------------------------- 1.1/1.1 MB 17.2 MB/s eta 0:00:00
Collecting bleach>=2.1.0
  Downloading bleach-5.0.0-py3-none-any.whl (160 kB)
     ---------------------------------------- 160.3/160.3 kB 9.4 MB/s eta 0:00:00
Collecting docutils>=0.13.1
  Downloading docutils-0.18.1-py2.py3-none-any.whl (570 kB)
     ---------------------------------------- 570.0/570.0 kB 17.5 MB/s eta 0:00:00
Collecting certifi>=2017.4.17
  Downloading certifi-2022.5.18.1-py3-none-any.whl (155 kB)
     ---------------------------------------- 155.2/155.2 kB 9.7 MB/s eta 0:00:00
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.3-py3-none-any.whl (61 kB)
     ---------------------------------------- 61.2/61.2 kB 3.2 MB/s eta 0:00:00
Collecting commonmark<0.10.0,>=0.9.0
  Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
     ---------------------------------------- 51.1/51.1 kB 2.6 MB/s eta 0:00:00
Collecting webencodings
  Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting six>=1.9.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: webencodings, pywin32-ctypes, pkginfo, commonmark, zipp, urllib3, six, rfc3986, Pygments, idna, docutils, charset-normalizer, certifi, rich, requests, importlib-metadata, bleach, requests-toolbelt, readme-renderer, keyring, twine
Successfully installed Pygments-2.12.0 bleach-5.0.0 certifi-2022.5.18.1 charset-normalizer-2.0.12 commonmark-0.9.1 docutils-0.18.1 idna-3.3 importlib-metadata-4.11.4 keyring-23.5.1 pkginfo-1.8.2 pywin32-ctypes-0.2.0 readme-renderer-35.0 requests-2.27.1 requests-toolbelt-0.9.1 rfc3986-2.0.0 rich-12.4.4 six-1.16.0 twine-4.0.1 urllib3-1.26.9 webencodings-0.5.1 zipp-3.8.0

So, I assume I should report this to Rich, then?

Some other notes, for completeness,

• When I SSH into Linux machines in the very same ConEmu tab and shell session (and both running bash with minimal custom config) that I can reproduce the bad behavior locally, running the latest Twine 4.0.0 installed into a fresh venv, the Rich progress bars display for both pip and Twine and the cursor issue does not occur (with the identical twine upload dist/* command running on an identical build of an identical checkout of the same project, for completeness).
• Up uninstall now, I've never seen this issue with pip. However, installing unchached packages with the latest 22.1.2 does cause the issue
• To confirm, I can actually reproduce this running pip and twine through cmd.exe in the native Command Prompt application as well—the cursor changes from the underbar to the block, though its hard to tell if the colors are any different. It just isn't as obvious because the default cursor there looks more similar to the block cursor than the default bar in ConEmu.

[bhrutledge]

Thanks for taking the time to troubleshoot this further. Your findings do strongly suggest that this is an issue with Rich in your environment.

I'd start by searching the Rich repo for open or closed issues that might be similar to yours. If you don't find anything, I suggest creating a minimal reproducible example using only Rich in a standalone virtual environment, e.g.:

python3 -m venv venv

.\venv\Scripts\activate

python3 -m install rich

Then write a tiny script that uses the Rich progress bar and reproduces the issue that you're seeing. Then, open a new issue on Rich that includes or links to that script.

[CAM-Gerlach]

Just FYI, I fixed the version issue in the conda-forge feedstock in conda-forge/twine-feedstock#52 , and opened a Rich issue, Textualize/rich#2333 , with further details and a minimal reproducer (python -c "import rich.console; rich.console.Console().show_cursor()"), and it looks like they've isolated the problem.