Show progressbar on generation hashes in verbose mode #743

atugushev · 2019-02-25T23:34:16Z

Shows a progressbar on a slow hashes generation. Uses a bar provided by click package.

Example:

$ echo "numpy" > requirements.in
$ pip-compile -v --generate-hashes

See stdout

Using indexes:
  https://pypi.org/simple

                          ROUND 1
Current constraints:
  numpy

Finding the best candidates:
  found candidate numpy==1.16.2 (constraint was <any>)

Finding secondary dependencies:
  numpy==1.16.2 not in cache, need to check index
  numpy==1.16.2             requires -
------------------------------------------------------------
Result of round 1: stable, done

Generating hashes:
  numpy
    Hashing https://files.pythonhosted.org/packages/61/be/b4d697563d4a211596a350414a87612204a8bb987c4c1b34598cd4904f55/numpy-1.16.2-cp37-cp37m-win32.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/7c/61/e0affb3a94043d493cbd3abaeb1ed75d9b2a2426ecdfd3bc985f75df1803/numpy-1.16.2-cp27-cp27m-win32.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/a6/6f/cb20ccd8f0f8581e0e090775c0e3c3e335b037818416e6fa945d924397d2/numpy-1.16.2-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/91/e7/6c780e612d245cca62bc3ba8e263038f7c144a96a54f877f3714a0e8427e/numpy-1.16.2-cp37-cp37m-manylinux1_x86_64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/bc/90/3e71b5392bd81d8559917ee38857bb2e4b92c88e87211a68e339127b86f5/numpy-1.16.2-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/ed/29/d97b6252591da5f8add0d25eecda296ea72729a0aad7998edba1981b47c8/numpy-1.16.2-cp36-cp36m-win_amd64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/66/8f/2f32f7283aae2a351feb5b39f0df53d62ee2845479ce5d2a3a5da6717d60/numpy-1.16.2-cp37-cp37m-manylinux1_i686.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/cf/8d/6345b4f32b37945fedc1e027e83970005fc9c699068d2f566b82826515f2/numpy-1.16.2.zip
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/6e/b9/3bab7c9a5fc02b6c8b659502c3a4c1779f0faf65b1c59b34ef2ae5fa94c6/numpy-1.16.2-cp27-cp27m-manylinux1_i686.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/5b/0f/a93ea6864511e121a31fb15ac6fcd85fcaef64ce1f995661cc29ea5f1814/numpy-1.16.2-cp27-cp27mu-manylinux1_i686.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/0b/be/7933dd42e95044624ed8ea200a392a965b6bf9e89ea36944e59ddbd579c2/numpy-1.16.2-cp35-cp35m-win_amd64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/45/04/7a738e489a25a9638520a43a0cbfcc4be3ed056266e3110a330a905b36b5/numpy-1.16.2-cp35-cp35m-manylinux1_i686.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/e3/18/4f013c3c3051f4e0ffbaa4bf247050d6d5e527fe9cb1907f5975b172f23f/numpy-1.16.2-cp35-cp35m-manylinux1_x86_64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/7f/65/a5a0ca3695bb3358faef4bd2131c8174aef78c4b2182d8cae404312bcc26/numpy-1.16.2-cp27-cp27m-win_amd64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/35/d5/4f8410ac303e690144f0a0603c4b8fd3b986feb2749c435f7cdbb288f17e/numpy-1.16.2-cp36-cp36m-manylinux1_x86_64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/c4/33/8ec8dcdb4ede5d453047bbdbd01916dbaccdb63e98bba60989718f5f0876/numpy-1.16.2-cp27-cp27mu-manylinux1_x86_64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/3f/a2/e8762e3c31366c53bf122afbc23edc150881f8d87c6ca23dc2e2b21e4cbe/numpy-1.16.2-cp35-cp35m-win32.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/3a/3c/515afabfe4f29bfc0a67037efaf518c33d0076b32d22ba865241cee295c4/numpy-1.16.2-cp37-cp37m-win_amd64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/5e/aa/2b8df68ba219718ce016f624610b08179a3f9ed2566b2c2b61224c58db5d/numpy-1.16.2-cp36-cp36m-manylinux1_i686.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/65/9e/f7fe595f62b8f7fcc154afb20df19b58ddf2723ca2e21dbbf749cd1b8e0c/numpy-1.16.2-cp27-cp27m-manylinux1_x86_64.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/b3/84/41c7af95bab850819bddbc13e3e10317dacd3e28e2a0a5f14d8dbdc1c725/numpy-1.16.2-cp36-cp36m-win32.whl
    [####################################]  100%
    Hashing https://files.pythonhosted.org/packages/93/0e/30aaa357c3065957344b240482818eef31d4080f73dfa5f1ef7dcd8744d2/numpy-1.16.2-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
    Hashing https://files.pythonhosted.org/packages/0e/65/a27186c1692901f7b451572857f6d8d0031b6928500fa479c30a489afeed/numpy-1.16.2-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
    [####################################]  100%

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile -v --generate-hashes
#
numpy==1.16.2 \
    --hash=sha256:1980f8d84548d74921685f68096911585fee393975f53797614b34d4f409b6da \
    --hash=sha256:22752cd809272671b273bb86df0f505f505a12368a3a5fc0aa811c7ece4dfd5c \
    --hash=sha256:23cc40313036cffd5d1873ef3ce2e949bdee0646c5d6f375bf7ee4f368db2511 \
    --hash=sha256:2b0b118ff547fecabc247a2668f48f48b3b1f7d63676ebc5be7352a5fd9e85a5 \
    --hash=sha256:3a0bd1edf64f6a911427b608a894111f9fcdb25284f724016f34a84c9a3a6ea9 \
    --hash=sha256:3f25f6c7b0d000017e5ac55977a3999b0b1a74491eacb3c1aa716f0e01f6dcd1 \
    --hash=sha256:4061c79ac2230594a7419151028e808239450e676c39e58302ad296232e3c2e8 \
    --hash=sha256:560ceaa24f971ab37dede7ba030fc5d8fa173305d94365f814d9523ffd5d5916 \
    --hash=sha256:62be044cd58da2a947b7e7b2252a10b42920df9520fc3d39f5c4c70d5460b8ba \
    --hash=sha256:6c692e3879dde0b67a9dc78f9bfb6f61c666b4562fd8619632d7043fb5b691b0 \
    --hash=sha256:6f65e37b5a331df950ef6ff03bd4136b3c0bbcf44d4b8e99135d68a537711b5a \
    --hash=sha256:7a78cc4ddb253a55971115f8320a7ce28fd23a065fc33166d601f51760eecfa9 \
    --hash=sha256:80a41edf64a3626e729a62df7dd278474fc1726836552b67a8c6396fd7e86760 \
    --hash=sha256:893f4d75255f25a7b8516feb5766c6b63c54780323b9bd4bc51cdd7efc943c73 \
    --hash=sha256:972ea92f9c1b54cc1c1a3d8508e326c0114aaf0f34996772a30f3f52b73b942f \
    --hash=sha256:9f1d4865436f794accdabadc57a8395bd3faa755449b4f65b88b7df65ae05f89 \
    --hash=sha256:9f4cd7832b35e736b739be03b55875706c8c3e5fe334a06210f1a61e5c2c8ca5 \
    --hash=sha256:adab43bf657488300d3aeeb8030d7f024fcc86e3a9b8848741ea2ea903e56610 \
    --hash=sha256:bd2834d496ba9b1bdda3a6cf3de4dc0d4a0e7be306335940402ec95132ad063d \
    --hash=sha256:d20c0360940f30003a23c0adae2fe50a0a04f3e48dc05c298493b51fd6280197 \
    --hash=sha256:d3b3ed87061d2314ff3659bb73896e622252da52558f2380f12c421fbdee3d89 \
    --hash=sha256:dc235bf29a406dfda5790d01b998a1c01d7d37f449128c0b1b7d1c89a84fae8b \
    --hash=sha256:fb3c83554f39f48f3fa3123b9c24aecf681b1c289f9334f8215c1d3c8e2f6e5b

Progressbar is shown for the packages which hashes generate operation took a time more that 0.5 secs (default click's behavior).

Context manager nullcontext added for conditional with statements.

Refs #521.

Changelog-friendly one-liner: Show progressbar on generation hashes in verbose mode

Contributor checklist

Provided the tests for the changes.
Requested a review from another contributor.
Gave a clear one-line description in the PR (that the maintainers can add to CHANGELOG.md on release).
Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

codecov · 2019-02-26T08:26:51Z

Codecov Report

Merging #743 into master will increase coverage by 0.35%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #743      +/-   ##
==========================================
+ Coverage   98.06%   98.41%   +0.35%     
==========================================
  Files          34       34              
  Lines        1962     2019      +57     
  Branches      256      258       +2     
==========================================
+ Hits         1924     1987      +63     
+ Misses         26       21       -5     
+ Partials       12       11       -1

Impacted Files	Coverage Δ
piptools/resolver.py	`100% <100%> (ø)`	⬆️
tests/test_repository_pypi.py	`100% <100%> (ø)`	⬆️
tests/test_cli_compile.py	`100% <100%> (ø)`	⬆️
piptools/repositories/pypi.py	`93.47% <100%> (+4.32%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d0a91a...0d7cbfd. Read the comment docs.

piptools/repositories/pypi.py

kb-1000 · 2019-03-02T10:05:20Z

Does this also show the downloading progress of the files to hash or only hashing itself?

atugushev · 2019-03-04T12:19:33Z

Does this also show the downloading progress of the files to hash or only hashing itself?

@kaeptmblaubaer1000 including the downloading progress.

blueyed · 2019-03-25T10:42:37Z

@kaeptmblaubaer1000
Can you try this PR and report feedback / review it, please?

atugushev · 2019-03-25T10:46:04Z

@graingert @proinsias
Also i would appreciate if you had a chance to review this PR.

kb-1000 · 2019-03-25T14:15:02Z

I ran it with --verbose --verbose, but I didn't get a progress bar.

atugushev · 2019-03-25T14:37:09Z

@kaeptmblaubaer1000
Could you try this one:

$ echo numpy > requirements.in
$ pip-compile -v --generate-hashes

and paste here the output?

kb-1000 · 2019-03-25T16:33:25Z

u0_a104 in pip-tools on  bb92715 via piptools-progressbar took 9s
➜ cd ../..

u0_a104 in ~/piptools-progressbar via piptools-progressbar
➜ echo numpy > req.in

u0_a104 in ~/piptools-progressbar via piptools-progressbar
➜ pip-compile --generate-hashes -v req.in
Using indexes:
  https://pypi.org/simple

                          ROUND 1
Current constraints:
  numpy

Finding the best candidates:
  found candidate numpy==1.16.2 (constraint was <any>)

Finding secondary dependencies:
  numpy==1.16.2             requires -
------------------------------------------------------------
Result of round 1: stable, done

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --generate-hashes -v req.in
#
numpy==1.16.2 \
    --hash=sha256:1980f8d84548d74921685f68096911585fee393975f53797614b34d4f409b6da \
    --hash=sha256:22752cd809272671b273bb86df0f505f505a12368a3a5fc0aa811c7ece4dfd5c \
    --hash=sha256:23cc40313036cffd5d1873ef3ce2e949bdee0646c5d6f375bf7ee4f368db2511 \
    --hash=sha256:2b0b118ff547fecabc247a2668f48f48b3b1f7d63676ebc5be7352a5fd9e85a5 \
    --hash=sha256:3a0bd1edf64f6a911427b608a894111f9fcdb25284f724016f34a84c9a3a6ea9 \
    --hash=sha256:3f25f6c7b0d000017e5ac55977a3999b0b1a74491eacb3c1aa716f0e01f6dcd1 \
    --hash=sha256:4061c79ac2230594a7419151028e808239450e676c39e58302ad296232e3c2e8 \
    --hash=sha256:560ceaa24f971ab37dede7ba030fc5d8fa173305d94365f814d9523ffd5d5916 \
    --hash=sha256:62be044cd58da2a947b7e7b2252a10b42920df9520fc3d39f5c4c70d5460b8ba \
    --hash=sha256:6c692e3879dde0b67a9dc78f9bfb6f61c666b4562fd8619632d7043fb5b691b0 \
    --hash=sha256:6f65e37b5a331df950ef6ff03bd4136b3c0bbcf44d4b8e99135d68a537711b5a \
    --hash=sha256:7a78cc4ddb253a55971115f8320a7ce28fd23a065fc33166d601f51760eecfa9 \
    --hash=sha256:80a41edf64a3626e729a62df7dd278474fc1726836552b67a8c6396fd7e86760 \
    --hash=sha256:893f4d75255f25a7b8516feb5766c6b63c54780323b9bd4bc51cdd7efc943c73 \
    --hash=sha256:972ea92f9c1b54cc1c1a3d8508e326c0114aaf0f34996772a30f3f52b73b942f \
    --hash=sha256:9f1d4865436f794accdabadc57a8395bd3faa755449b4f65b88b7df65ae05f89 \
    --hash=sha256:9f4cd7832b35e736b739be03b55875706c8c3e5fe334a06210f1a61e5c2c8ca5 \
    --hash=sha256:adab43bf657488300d3aeeb8030d7f024fcc86e3a9b8848741ea2ea903e56610 \
    --hash=sha256:bd2834d496ba9b1bdda3a6cf3de4dc0d4a0e7be306335940402ec95132ad063d \
    --hash=sha256:d20c0360940f30003a23c0adae2fe50a0a04f3e48dc05c298493b51fd6280197 \
    --hash=sha256:d3b3ed87061d2314ff3659bb73896e622252da52558f2380f12c421fbdee3d89 \
    --hash=sha256:dc235bf29a406dfda5790d01b998a1c01d7d37f449128c0b1b7d1c89a84fae8b \
    --hash=sha256:fb3c83554f39f48f3fa3123b9c24aecf681b1c289f9334f8215c1d3c8e2f6e5b

u0_a104 in ~/piptools-progressbar via piptools-progressbar took 19s
➜

atugushev · 2019-03-25T18:22:17Z

@kaeptmblaubaer1000 it seems you didn't install a forked pip-tools. Try this one:

$ pip install -U git+https://github.com/atugushev/pip-tools.git@progressbar-generate-hashes#egg=pip-tools
$ pip-compile --version
$ echo numpy > requirements.in
$ pip-compile -v --generate-hashes

and paste here the output.

kb-1000 · 2019-03-26T14:20:09Z

u0_a104 in ~/piptools-progressbar via piptools-progressbar took 8s
➜ pip install -U -e git+https://github.com/atugushev/pip-tools.git@progressbar-generate-hashes#egg=pip-tools
Obtaining pip-tools from git+https://github.com/atugushev/pip-tools.git@progressbar-generate-hashes#egg=pip-tools
  Updating ./src/pip-tools clone (to revision progressbar-generate-hashes)
Requirement already satisfied, skipping upgrade: click>=6 in ./lib/python3.7/site-packages (from pip-tools) (7.0)
Requirement already satisfied, skipping upgrade: six in ./lib/python3.7/site-packages (from pip-tools) (1.12.0)
Installing collected packages: pip-tools
  Found existing installation: pip-tools 3.2.1.dev58+ge23e617
    Uninstalling pip-tools-3.2.1.dev58+ge23e617:
      Successfully uninstalled pip-tools-3.2.1.dev58+ge23e617
  Running setup.py develop for pip-tools
Successfully installed pip-tools

u0_a104 in ~/piptools-progressbar via piptools-progressbar took 8s
➜ XDG_CACHE_HOME=cache pip-compile --verbose --generate-hashes req.in -o req.txt
Using indexes:
  https://pypi.org/simple

                          ROUND 1
Current constraints:
  numpy

Finding the best candidates:
  found candidate numpy==1.16.2 (constraint was <any>)

Finding secondary dependencies:
  numpy==1.16.2 not in cache, need to check index
  numpy==1.16.2             requires -
------------------------------------------------------------
Result of round 1: stable, done

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --verbose --generate-hashes req.in -o req.txt
#
numpy==1.16.2 \
    --hash=sha256:1980f8d84548d74921685f68096911585fee393975f53797614b34d4f409b6da \
    --hash=sha256:22752cd809272671b273bb86df0f505f505a12368a3a5fc0aa811c7ece4dfd5c \
    --hash=sha256:23cc40313036cffd5d1873ef3ce2e949bdee0646c5d6f375bf7ee4f368db2511 \
    --hash=sha256:2b0b118ff547fecabc247a2668f48f48b3b1f7d63676ebc5be7352a5fd9e85a5 \
    --hash=sha256:3a0bd1edf64f6a911427b608a894111f9fcdb25284f724016f34a84c9a3a6ea9 \
    --hash=sha256:3f25f6c7b0d000017e5ac55977a3999b0b1a74491eacb3c1aa716f0e01f6dcd1 \
    --hash=sha256:4061c79ac2230594a7419151028e808239450e676c39e58302ad296232e3c2e8 \
    --hash=sha256:560ceaa24f971ab37dede7ba030fc5d8fa173305d94365f814d9523ffd5d5916 \
    --hash=sha256:62be044cd58da2a947b7e7b2252a10b42920df9520fc3d39f5c4c70d5460b8ba \
    --hash=sha256:6c692e3879dde0b67a9dc78f9bfb6f61c666b4562fd8619632d7043fb5b691b0 \
    --hash=sha256:6f65e37b5a331df950ef6ff03bd4136b3c0bbcf44d4b8e99135d68a537711b5a \
    --hash=sha256:7a78cc4ddb253a55971115f8320a7ce28fd23a065fc33166d601f51760eecfa9 \
    --hash=sha256:80a41edf64a3626e729a62df7dd278474fc1726836552b67a8c6396fd7e86760 \
    --hash=sha256:893f4d75255f25a7b8516feb5766c6b63c54780323b9bd4bc51cdd7efc943c73 \
    --hash=sha256:972ea92f9c1b54cc1c1a3d8508e326c0114aaf0f34996772a30f3f52b73b942f \
    --hash=sha256:9f1d4865436f794accdabadc57a8395bd3faa755449b4f65b88b7df65ae05f89 \
    --hash=sha256:9f4cd7832b35e736b739be03b55875706c8c3e5fe334a06210f1a61e5c2c8ca5 \
    --hash=sha256:adab43bf657488300d3aeeb8030d7f024fcc86e3a9b8848741ea2ea903e56610 \
    --hash=sha256:bd2834d496ba9b1bdda3a6cf3de4dc0d4a0e7be306335940402ec95132ad063d \
    --hash=sha256:d20c0360940f30003a23c0adae2fe50a0a04f3e48dc05c298493b51fd6280197 \
    --hash=sha256:d3b3ed87061d2314ff3659bb73896e622252da52558f2380f12c421fbdee3d89 \
    --hash=sha256:dc235bf29a406dfda5790d01b998a1c01d7d37f449128c0b1b7d1c89a84fae8b \
    --hash=sha256:fb3c83554f39f48f3fa3123b9c24aecf681b1c289f9334f8215c1d3c8e2f6e5b

u0_a104 in ~/piptools-progressbar via piptools-progressbar took 12s

I can't see any difference, probably because I already used pull/743/merge before.

atugushev · 2019-03-26T15:28:36Z

@kaeptmblaubaer1000 please provide a version by:
pip-compile --version

kb-1000 · 2019-03-26T15:30:35Z

pip-compile, version 3.5.0

atugushev · 2019-03-26T15:36:10Z

@kaeptmblaubaer1000
it's a wrong version, you are using last release. It should be version from the fork, like this:

$ pip-compile --version
pip-compile, version 3.3.3.dev34+ge23e617.d20190326

where e23e617 is a hash of the commit.

So, let's do it again in virtualenv:

virtualenv .venv
source .venv/bin/activate
pip install -U git+https://github.com/atugushev/pip-tools.git@progressbar-generate-hashes#egg=pip-tools
pip-compile --version
echo numpy > requirements.in
pip-compile -v --generate-hashes

kb-1000 · 2019-03-26T15:39:48Z

Nope. It was already in a virtualenv, and it was activated. I guess it didn't change the PATH correctly... FYI, I'm using zsh.

kb-1000 · 2019-03-26T15:41:13Z

Ok, now activated again:

u0_a104 in ~/piptools-progressbar via piptools-progressbar took 4s
➜ deactivate

u0_a104 in ~/piptools-progressbar
➜ . ./bin/activate

u0_a104 in ~/piptools-progressbar via piptools-progressbar
➜ rm -r cache

u0_a104 in ~/piptools-progressbar via piptools-progressbar
➜ pip-compile --generate-hashes -v req.in
Using indexes:
  https://pypi.org/simple

                          ROUND 1
Current constraints:
  numpy

Finding the best candidates:
  found candidate numpy==1.16.2 (constraint was <any>)

Finding secondary dependencies:
  numpy==1.16.2             requires -
------------------------------------------------------------
Result of round 1: stable, done

Generating hashes:

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --generate-hashes -v req.in
#
numpy==1.16.2 \
    --hash=sha256:1980f8d84548d74921685f68096911585fee393975f53797614b34d4f409b6da \
    --hash=sha256:22752cd809272671b273bb86df0f505f505a12368a3a5fc0aa811c7ece4dfd5c \
    --hash=sha256:23cc40313036cffd5d1873ef3ce2e949bdee0646c5d6f375bf7ee4f368db2511 \
    --hash=sha256:2b0b118ff547fecabc247a2668f48f48b3b1f7d63676ebc5be7352a5fd9e85a5 \
    --hash=sha256:3a0bd1edf64f6a911427b608a894111f9fcdb25284f724016f34a84c9a3a6ea9 \
    --hash=sha256:3f25f6c7b0d000017e5ac55977a3999b0b1a74491eacb3c1aa716f0e01f6dcd1 \
    --hash=sha256:4061c79ac2230594a7419151028e808239450e676c39e58302ad296232e3c2e8 \
    --hash=sha256:560ceaa24f971ab37dede7ba030fc5d8fa173305d94365f814d9523ffd5d5916 \
    --hash=sha256:62be044cd58da2a947b7e7b2252a10b42920df9520fc3d39f5c4c70d5460b8ba \
    --hash=sha256:6c692e3879dde0b67a9dc78f9bfb6f61c666b4562fd8619632d7043fb5b691b0 \
    --hash=sha256:6f65e37b5a331df950ef6ff03bd4136b3c0bbcf44d4b8e99135d68a537711b5a \
    --hash=sha256:7a78cc4ddb253a55971115f8320a7ce28fd23a065fc33166d601f51760eecfa9 \
    --hash=sha256:80a41edf64a3626e729a62df7dd278474fc1726836552b67a8c6396fd7e86760 \
    --hash=sha256:893f4d75255f25a7b8516feb5766c6b63c54780323b9bd4bc51cdd7efc943c73 \
    --hash=sha256:972ea92f9c1b54cc1c1a3d8508e326c0114aaf0f34996772a30f3f52b73b942f \
    --hash=sha256:9f1d4865436f794accdabadc57a8395bd3faa755449b4f65b88b7df65ae05f89 \
    --hash=sha256:9f4cd7832b35e736b739be03b55875706c8c3e5fe334a06210f1a61e5c2c8ca5 \
    --hash=sha256:adab43bf657488300d3aeeb8030d7f024fcc86e3a9b8848741ea2ea903e56610 \
    --hash=sha256:bd2834d496ba9b1bdda3a6cf3de4dc0d4a0e7be306335940402ec95132ad063d \
    --hash=sha256:d20c0360940f30003a23c0adae2fe50a0a04f3e48dc05c298493b51fd6280197 \
    --hash=sha256:d3b3ed87061d2314ff3659bb73896e622252da52558f2380f12c421fbdee3d89 \
    --hash=sha256:dc235bf29a406dfda5790d01b998a1c01d7d37f449128c0b1b7d1c89a84fae8b \
    --hash=sha256:fb3c83554f39f48f3fa3123b9c24aecf681b1c289f9334f8215c1d3c8e2f6e5b

u0_a104 in ~/piptools-progressbar via piptools-progressbar took 3s
➜

Looks like it does work.

atugushev · 2019-03-26T15:54:28Z

@kaeptmblaubaer1000 looks good now, but it has been generated so quick :) Could you try some heavy weight packages:

$ cat requirements.in
matplotlib
pandas
scikit-learn
scipy

kb-1000 · 2019-03-26T15:59:52Z

I already have my big set of requirements... something is wrong with it though, it now shows that it generates the hashes of numpy, and it now took more than 30 seconds even though it already has downloaded it. And it gave no progress bar for a minute and now prints it'd take 1:23:29...

atugushev · 2019-04-05T02:34:30Z

@kaeptmblaubaer1000

I already have my big set of requirements.

Could you share it? I'll try to run locally.

proinsias · 2019-04-05T13:25:58Z

@atugushev – this looks great! Thanks for doing this work.

atugushev · 2019-04-06T19:24:17Z

@blueyed @kaeptmblaubaer1000 @proinsias

Guys! I've added more details on hashes generation process and made a progress-bar(s) smooth. Could you please review it again? 🙏

atugushev · 2019-04-06T19:53:41Z

It looks like this:

piptools/repositories/pypi.py

[ci skip]

atugushev added this to the 3.5.0 milestone Feb 25, 2019

atugushev requested review from blueyed and vphilippon February 25, 2019 23:34

atugushev mentioned this pull request Feb 25, 2019

pip-compile with generate hashes is very slow #521

Closed

atugushev added the logging Related to log or console output label Feb 25, 2019

blueyed reviewed Mar 1, 2019

View reviewed changes

piptools/repositories/pypi.py Outdated Show resolved Hide resolved

atugushev force-pushed the progressbar-generate-hashes branch from 3001b7e to e23e617 Compare March 1, 2019 22:37

atugushev modified the milestones: 3.5.0, 3.6.0 Mar 11, 2019

blueyed added the enhancement Improvements to functionality label Mar 25, 2019

Show a progress of a hashes generation process in a verbose mode

ede863b

atugushev force-pushed the progressbar-generate-hashes branch from e23e617 to ede863b Compare April 6, 2019 19:15

atugushev requested a review from suutari April 6, 2019 20:11

atugushev commented Apr 6, 2019

View reviewed changes

piptools/repositories/pypi.py Outdated Show resolved Hide resolved

piptools/repositories/pypi.py Outdated Show resolved Hide resolved

piptools/repositories/pypi.py Show resolved Hide resolved

atugushev added 3 commits April 7, 2019 03:23

Rename File to FileStream

6417179

[ci skip]

Add tests for open_local_or_remote_file function

81bfd19

Exclude line from coverage

0d7cbfd

atugushev requested a review from blueyed April 14, 2019 10:36

auvipy added this to the 3.7.0 milestone May 2, 2019

auvipy approved these changes May 2, 2019

View reviewed changes

auvipy merged commit 3b1b99d into jazzband:master May 2, 2019

atugushev mentioned this pull request Sep 15, 2019

pip-compile with generate hashes is very slow for new requirements.txt generation #643

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Show progressbar on generation hashes in verbose mode #743

Show progressbar on generation hashes in verbose mode #743

atugushev commented Feb 25, 2019 •

edited by auvipy

Loading

codecov bot commented Feb 26, 2019 •

edited

Loading

kb-1000 commented Mar 2, 2019

atugushev commented Mar 4, 2019

blueyed commented Mar 25, 2019

atugushev commented Mar 25, 2019

kb-1000 commented Mar 25, 2019

atugushev commented Mar 25, 2019

kb-1000 commented Mar 25, 2019

atugushev commented Mar 25, 2019

kb-1000 commented Mar 26, 2019

atugushev commented Mar 26, 2019

kb-1000 commented Mar 26, 2019

atugushev commented Mar 26, 2019

kb-1000 commented Mar 26, 2019

kb-1000 commented Mar 26, 2019 •

edited

Loading

atugushev commented Mar 26, 2019

kb-1000 commented Mar 26, 2019 •

edited

Loading

atugushev commented Apr 5, 2019 •

edited

Loading

proinsias commented Apr 5, 2019

atugushev commented Apr 6, 2019 •

edited

Loading

atugushev commented Apr 6, 2019

Show progressbar on generation hashes in verbose mode #743

Show progressbar on generation hashes in verbose mode #743

Conversation

atugushev commented Feb 25, 2019 • edited by auvipy Loading

Contributor checklist

codecov bot commented Feb 26, 2019 • edited Loading

Codecov Report

kb-1000 commented Mar 2, 2019

atugushev commented Mar 4, 2019

blueyed commented Mar 25, 2019

atugushev commented Mar 25, 2019

kb-1000 commented Mar 25, 2019

atugushev commented Mar 25, 2019

kb-1000 commented Mar 25, 2019

atugushev commented Mar 25, 2019

kb-1000 commented Mar 26, 2019

atugushev commented Mar 26, 2019

kb-1000 commented Mar 26, 2019

atugushev commented Mar 26, 2019

kb-1000 commented Mar 26, 2019

kb-1000 commented Mar 26, 2019 • edited Loading

atugushev commented Mar 26, 2019

kb-1000 commented Mar 26, 2019 • edited Loading

atugushev commented Apr 5, 2019 • edited Loading

proinsias commented Apr 5, 2019

atugushev commented Apr 6, 2019 • edited Loading

atugushev commented Apr 6, 2019

atugushev commented Feb 25, 2019 •

edited by auvipy

Loading

codecov bot commented Feb 26, 2019 •

edited

Loading

kb-1000 commented Mar 26, 2019 •

edited

Loading

kb-1000 commented Mar 26, 2019 •

edited

Loading

atugushev commented Apr 5, 2019 •

edited

Loading

atugushev commented Apr 6, 2019 •

edited

Loading