diff --git a/.flake8 b/.flake8 index 08001ffac..1cc049a69 100644 --- a/.flake8 +++ b/.flake8 @@ -1,38 +1,26 @@ [flake8] + show-source = True -count= True +count = True statistics = True -# E265 = comment blocks like @{ section, which it can't handle + # E266 = too many leading '#' for block comment # E731 = do not assign a lambda expression, use a def -# W293 = Blank line contains whitespace -# W504 = Line break after operator -# E704 = multiple statements in one line - used for @override # TC002 = move third party import to TYPE_CHECKING -# ANN = flake8-annotations # TC, TC2 = flake8-type-checking -# D = flake8-docstrings # select = C,E,F,W ANN, TC, TC2 # to enable code. Disabled if not listed, including builtin codes enable-extensions = TC, TC2 # only needed for extensions not enabled by default -ignore = E265,E266,E731,E704, - W293, W504, - ANN0 ANN1 ANN2, - TC002, - TC0, TC1, TC2 - # B, - A, - D, - RST, RST3 +ignore = E266, E731 -exclude = .tox,.venv,build,dist,doc,git/ext/,test +exclude = .tox, .venv, build, dist, doc, git/ext/ rst-roles = # for flake8-RST-docstrings - attr,class,func,meth,mod,obj,ref,term,var # used by sphinx + attr, class, func, meth, mod, obj, ref, term, var # used by sphinx min-python-version = 3.7.0 # for `black` compatibility max-line-length = 120 -extend-ignore = E203,W503 +extend-ignore = E203, W503 diff --git a/.gitattributes b/.gitattributes index 6d2618f2f..3f3d2f050 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,2 +1,3 @@ test/fixtures/* eol=lf -init-tests-after-clone.sh +*.sh eol=lf +/Makefile eol=lf diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 203f3c889..8c139c7be 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -3,4 +3,4 @@ updates: - package-ecosystem: "github-actions" directory: "/" schedule: - interval: "weekly" + interval: "weekly" diff --git a/.github/workflows/cygwin-test.yml b/.github/workflows/cygwin-test.yml index 962791ae7..cd913385f 100644 --- a/.github/workflows/cygwin-test.yml +++ b/.github/workflows/cygwin-test.yml @@ -5,20 +5,24 @@ on: [push, pull_request, workflow_dispatch] jobs: build: runs-on: windows-latest + strategy: fail-fast: false + env: CHERE_INVOKING: 1 - SHELLOPTS: igncr TMP: "/tmp" TEMP: "/tmp" + defaults: run: - shell: bash.exe --noprofile --norc -exo pipefail -o igncr "{0}" + shell: C:\cygwin\bin\bash.exe --noprofile --norc -exo pipefail -o igncr "{0}" steps: - name: Force LF line endings - run: git config --global core.autocrlf input + run: | + git config --global core.autocrlf false # Affects the non-Cygwin git. + shell: bash - uses: actions/checkout@v4 with: @@ -29,36 +33,42 @@ jobs: with: packages: python39 python39-pip python39-virtualenv git - - name: Show python and git versions + - name: Special configuration for Cygwin's git run: | - /usr/bin/python --version - /usr/bin/git version - - - name: Tell git to trust this repo - run: | - /usr/bin/git config --global --add safe.directory "$(pwd)" + git config --global --add safe.directory "$(pwd)" + git config --global core.autocrlf false - name: Prepare this repo for tests run: | - TRAVIS=yes ./init-tests-after-clone.sh + ./init-tests-after-clone.sh - - name: Further prepare git configuration for tests + - name: Set git user identity and command aliases for the tests run: | - /usr/bin/git config --global user.email "travis@ci.com" - /usr/bin/git config --global user.name "Travis Runner" + git config --global user.email "travis@ci.com" + git config --global user.name "Travis Runner" # If we rewrite the user's config by accident, we will mess it up # and cause subsequent tests to fail cat test/fixtures/.gitconfig >> ~/.gitconfig - name: Update PyPA packages run: | - /usr/bin/python -m pip install --upgrade pip setuptools wheel + # Get the latest pip, wheel, and prior to Python 3.12, setuptools. + python -m pip install -U pip $(pip freeze --all | grep -oF setuptools) wheel - name: Install project and test dependencies run: | - /usr/bin/python -m pip install ".[test]" + python -m pip install ".[test]" + + - name: Show version and platform information + run: | + uname -a + command -v git python + git version + python --version + python -c 'import sys; print(sys.platform)' + python -c 'import os; print(os.name)' + python -c 'import git; print(git.compat.is_win)' - name: Test with pytest run: | - set +x - /usr/bin/python -m pytest + python -m pytest --color=yes -p no:sugar --instafail -vv diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 5e79664a8..91dd919e0 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -7,8 +7,14 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v4 - with: - python-version: "3.x" - - uses: pre-commit/action@v3.0.0 + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v4 + with: + python-version: "3.x" + + - uses: pre-commit/action@v3.0.0 + with: + extra_args: --all-files --hook-stage manual + env: + SKIP: black-format diff --git a/.github/workflows/pythonpackage.yml b/.github/workflows/pythonpackage.yml index a5467ef94..2a82e0e03 100644 --- a/.github/workflows/pythonpackage.yml +++ b/.github/workflows/pythonpackage.yml @@ -10,16 +10,15 @@ permissions: jobs: build: - runs-on: ubuntu-latest + strategy: fail-fast: false matrix: python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"] include: - - experimental: false - - python-version: "3.12" - experimental: true + - experimental: false + defaults: run: shell: /bin/bash --noprofile --norc -exo pipefail {0} @@ -36,16 +35,11 @@ jobs: python-version: ${{ matrix.python-version }} allow-prereleases: ${{ matrix.experimental }} - - name: Show python and git versions - run: | - python --version - git version - - name: Prepare this repo for tests run: | - TRAVIS=yes ./init-tests-after-clone.sh + ./init-tests-after-clone.sh - - name: Prepare git configuration for tests + - name: Set git user identity and command aliases for the tests run: | git config --global user.email "travis@ci.com" git config --global user.name "Travis Runner" @@ -55,17 +49,23 @@ jobs: - name: Update PyPA packages run: | - python -m pip install --upgrade pip - if pip freeze --all | grep --quiet '^setuptools=='; then - # Python prior to 3.12 ships setuptools. Upgrade it if present. - python -m pip install --upgrade setuptools - fi - python -m pip install --upgrade wheel + # Get the latest pip, wheel, and prior to Python 3.12, setuptools. + python -m pip install -U pip $(pip freeze --all | grep -oF setuptools) wheel - name: Install project and test dependencies run: | pip install ".[test]" + - name: Show version and platform information + run: | + uname -a + command -v git python + git version + python --version + python -c 'import sys; print(sys.platform)' + python -c 'import os; print(os.name)' + python -c 'import git; print(git.compat.is_win)' + - name: Check types with mypy run: | mypy -p git @@ -75,7 +75,7 @@ jobs: - name: Test with pytest run: | - pytest + pytest --color=yes -p no:sugar --instafail -vv continue-on-error: false - name: Documentation diff --git a/.gitignore b/.gitignore index 72da84eee..191e0e6c3 100644 --- a/.gitignore +++ b/.gitignore @@ -1,12 +1,15 @@ *.py[co] *.swp *~ +.env/ +env/ .venv/ venv/ /*.egg-info /lib/GitPython.egg-info cover/ .coverage +.coverage.* /build /dist /doc/_build @@ -22,4 +25,3 @@ nbproject .pytest_cache/ monkeytype.sqlite3 output.txt -tox.ini diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 581cb69b2..be97d5f9b 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,19 +1,39 @@ repos: - - repo: https://github.com/PyCQA/flake8 - rev: 6.0.0 - hooks: - - id: flake8 - additional_dependencies: - [ - flake8-bugbear==22.12.6, - flake8-comprehensions==3.10.1, - flake8-typing-imports==1.14.0, - ] - exclude: ^doc|^git/ext/|^test/ +- repo: https://github.com/psf/black-pre-commit-mirror + rev: 23.9.1 + hooks: + - id: black + alias: black-check + name: black (check) + args: [--check, --diff] + exclude: ^git/ext/ + stages: [manual] - - repo: https://github.com/pre-commit/pre-commit-hooks - rev: v4.4.0 - hooks: - - id: check-merge-conflict - - id: check-toml - - id: check-yaml + - id: black + alias: black-format + name: black (format) + exclude: ^git/ext/ + +- repo: https://github.com/PyCQA/flake8 + rev: 6.1.0 + hooks: + - id: flake8 + additional_dependencies: + - flake8-bugbear==23.9.16 + - flake8-comprehensions==3.14.0 + - flake8-typing-imports==1.14.0 + exclude: ^doc|^git/ext/ + +- repo: https://github.com/shellcheck-py/shellcheck-py + rev: v0.9.0.5 + hooks: + - id: shellcheck + args: [--color] + exclude: ^git/ext/ + +- repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.4.0 + hooks: + - id: check-toml + - id: check-yaml + - id: check-merge-conflict diff --git a/AUTHORS b/AUTHORS index ba5636db8..3e99ff785 100644 --- a/AUTHORS +++ b/AUTHORS @@ -52,4 +52,5 @@ Contributors are: -Joseph Hale -Santos Gallegos -Wenhan Zhu + Portions derived from other open source works and are clearly marked. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 56af0df2a..e108f1b80 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,7 +2,7 @@ The following is a short step-by-step rundown of what one typically would do to contribute. -- [fork this project](https://github.com/gitpython-developers/GitPython/fork) on GitHub. +- [Fork this project](https://github.com/gitpython-developers/GitPython/fork) on GitHub. - For setting up the environment to run the self tests, please run `init-tests-after-clone.sh`. - Please try to **write a test that fails unless the contribution is present.** - Try to avoid massive commits and prefer to take small steps, with one commit for each. diff --git a/LICENSE b/LICENSE index 5a9a6f8d3..ba8a219fe 100644 --- a/LICENSE +++ b/LICENSE @@ -1,30 +1,29 @@ Copyright (C) 2008, 2009 Michael Trier and contributors All rights reserved. -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: -* Redistributions of source code must retain the above copyright +* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. -* Redistributions in binary form must reproduce the above copyright -notice, this list of conditions and the following disclaimer in the +* Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. -* Neither the name of the GitPython project nor the names of -its contributors may be used to endorse or promote products derived +* Neither the name of the GitPython project nor the names of +its contributors may be used to endorse or promote products derived from this software without specific prior written permission. -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED -TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR -PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF -LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING -NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED +TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR +PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF +LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - diff --git a/Makefile b/Makefile index 389337d08..839dc9f78 100644 --- a/Makefile +++ b/Makefile @@ -1,22 +1,19 @@ -.PHONY: all clean release force_release +.PHONY: all lint clean release force_release all: - @grep -Ee '^[a-z].*:' Makefile | cut -d: -f1 | grep -vF all + @awk -F: '/^[[:alpha:]].*:/ && !/^all:/ {print $$1}' Makefile + +lint: + SKIP=black-format pre-commit run --all-files --hook-stage manual clean: rm -rf build/ dist/ .eggs/ .tox/ release: clean - # Check if latest tag is the current head we're releasing - echo "Latest tag = $$(git tag | sort -nr | head -n1)" - echo "HEAD SHA = $$(git rev-parse head)" - echo "Latest tag SHA = $$(git tag | sort -nr | head -n1 | xargs git rev-parse)" - @test "$$(git rev-parse head)" = "$$(git tag | sort -nr | head -n1 | xargs git rev-parse)" + ./check-version.sh make force_release force_release: clean - # IF we're in a virtual environment, add build tools - test -z "$$VIRTUAL_ENV" || pip install -U build twine - python3 -m build --sdist --wheel || echo "Use a virtual-env with 'python -m venv env && source env/bin/activate' instead" + ./build-release.sh twine upload dist/* git push --tags origin main diff --git a/README.md b/README.md index ca470a851..b9e61912a 100644 --- a/README.md +++ b/README.md @@ -76,17 +76,20 @@ To clone the [the GitHub repository](https://github.com/gitpython-developers/Git ```bash git clone https://github.com/gitpython-developers/GitPython cd GitPython -git fetch --tags ./init-tests-after-clone.sh ``` +On Windows, `./init-tests-after-clone.sh` can be run in a Git Bash shell. + If you are cloning [your own fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/about-forks), then replace the above `git clone` command with one that gives the URL of your fork. Or use this [`gh`](https://cli.github.com/) command (assuming you have `gh` and your fork is called `GitPython`): ```bash gh repo clone GitPython ``` -Having cloned the repo, create and activate your [virtual environment](https://docs.python.org/3/tutorial/venv.html). Then make an [editable install](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs): +Having cloned the repo, create and activate your [virtual environment](https://docs.python.org/3/tutorial/venv.html). + +Then make an [editable install](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs): ```bash pip install -e ".[test]" @@ -114,13 +117,9 @@ See [Issue #525](https://github.com/gitpython-developers/GitPython/issues/525). ### RUNNING TESTS -_Important_: Right after cloning this repository, please be sure to have -executed `git fetch --tags` followed by the `./init-tests-after-clone.sh` -script in the repository root. Otherwise you will encounter test failures. - -On _Windows_, make sure you have `git-daemon` in your PATH. For MINGW-git, the `git-daemon.exe` -exists in `Git\mingw64\libexec\git-core\`; CYGWIN has no daemon, but should get along fine -with MINGW's. +_Important_: Right after cloning this repository, please be sure to have executed +the `./init-tests-after-clone.sh` script in the repository root. Otherwise +you will encounter test failures. #### Install test dependencies @@ -144,30 +143,41 @@ To test, run: pytest ``` -To lint, run: +To lint, and apply automatic code formatting, run: ```bash pre-commit run --all-files ``` +- Linting without modifying code can be done with: `make lint` +- Auto-formatting without other lint checks can be done with: `black .` + To typecheck, run: ```bash mypy -p git ``` -For automatic code formatting, run: +#### CI (and tox) -```bash -black git -``` +The same linting, and running tests on all the different supported Python versions, will be performed: + +- Upon submitting a pull request. +- On each push, *if* you have a fork with GitHub Actions enabled. +- Locally, if you run [`tox`](https://tox.wiki/) (this skips any Python versions you don't have installed). + +#### Configuration files + +Specific tools: -Configuration for flake8 is in the `./.flake8` file. +- Configurations for `mypy`, `pytest`, `coverage.py`, and `black` are in `./pyproject.toml`. +- Configuration for `flake8` is in the `./.flake8` file. -Configurations for `mypy`, `pytest`, `coverage.py`, and `black` are in `./pyproject.toml`. +Orchestration tools: -The same linting and testing will also be performed against different supported python versions -upon submitting a pull request (or on each push if you have a fork with a "main" branch and actions enabled). +- Configuration for `pre-commit` is in the `./.pre-commit-config.yaml` file. +- Configuration for `tox` is in `./tox.ini`. +- Configuration for GitHub Actions (CI) is in files inside `./.github/workflows/`. ### Contributions @@ -177,7 +187,7 @@ Please have a look at the [contributions file][contributing]. - [User Documentation](http://gitpython.readthedocs.org) - [Questions and Answers](http://stackexchange.com/filters/167317/gitpython) -- Please post on stackoverflow and use the `gitpython` tag +- Please post on Stack Overflow and use the `gitpython` tag - [Issue Tracker](https://github.com/gitpython-developers/GitPython/issues) - Post reproducible bugs and feature requests as a new issue. Please be sure to provide the following information if posting bugs: @@ -267,6 +277,7 @@ gpg --edit-key 4C08421980C9 ### LICENSE -New BSD License. See the LICENSE file. +[New BSD License](https://opensource.org/license/bsd-3-clause/). See the [LICENSE file][license]. -[contributing]: https://github.com/gitpython-developers/GitPython/blob/master/CONTRIBUTING.md +[contributing]: https://github.com/gitpython-developers/GitPython/blob/main/CONTRIBUTING.md +[license]: https://github.com/gitpython-developers/GitPython/blob/main/LICENSE diff --git a/VERSION b/VERSION index b402c1a8b..1f1a39706 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -3.1.36 +3.1.37 diff --git a/build-release.sh b/build-release.sh new file mode 100755 index 000000000..49c13b93a --- /dev/null +++ b/build-release.sh @@ -0,0 +1,27 @@ +#!/bin/bash +# +# This script builds a release. If run in a venv, it auto-installs its tools. +# You may want to run "make release" instead of running this script directly. + +set -eEu + +function release_with() { + "$1" -m build --sdist --wheel +} + +function suggest_venv() { + local venv_cmd='python -m venv env && source env/bin/activate' + printf "HELP: To avoid this error, use a virtual-env with '%s' instead.\n" "$venv_cmd" +} + +if test -n "${VIRTUAL_ENV-}"; then + deps=(build twine) # Install twine along with build, as we need it later. + echo "Virtual environment detected. Adding packages: ${deps[*]}" + pip install --quiet --upgrade "${deps[@]}" + echo 'Starting the build.' + release_with python +else + trap suggest_venv ERR # This keeps the original exit (error) code. + echo 'Starting the build.' + release_with python3 # Outside a venv, use python3. +fi diff --git a/check-version.sh b/check-version.sh new file mode 100755 index 000000000..dac386e46 --- /dev/null +++ b/check-version.sh @@ -0,0 +1,56 @@ +#!/bin/bash +# +# This script checks if we are in a consistent state to build a new release. +# See the release instructions in README.md for the steps to make this pass. +# You may want to run "make release" instead of running this script directly. + +set -eEfuo pipefail +trap 'echo "$0: Check failed. Stopping." >&2' ERR + +readonly version_path='VERSION' +readonly changes_path='doc/source/changes.rst' + +function check_status() { + git status -s "$@" + test -z "$(git status -s "$@")" +} + +function get_latest_tag() { + local config_opts + printf -v config_opts ' -c versionsort.suffix=-%s' alpha beta pre rc RC + # shellcheck disable=SC2086 # Deliberately word-splitting the arguments. + git $config_opts tag -l '[0-9]*' --sort=-v:refname | head -n1 +} + +echo 'Checking current directory.' +test "$(cd -- "$(dirname -- "$0")" && pwd)" = "$(pwd)" # Ugly, but portable. + +echo "Checking that $version_path and $changes_path exist and have no uncommitted changes." +test -f "$version_path" +test -f "$changes_path" +check_status -- "$version_path" "$changes_path" + +# This section can be commented out, if absolutely necessary. +echo 'Checking that ALL changes are committed.' +check_status --ignore-submodules + +version_version="$(<"$version_path")" +changes_version="$(awk '/^[0-9]/ {print $0; exit}' "$changes_path")" +latest_tag="$(get_latest_tag)" +head_sha="$(git rev-parse HEAD)" +latest_tag_sha="$(git rev-parse "${latest_tag}^{commit}")" + +# Display a table of all the current version, tag, and HEAD commit information. +echo +echo 'The VERSION must be the same in all locations, and so must the HEAD and tag SHA' +printf '%-14s = %s\n' 'VERSION file' "$version_version" \ + 'changes.rst' "$changes_version" \ + 'Latest tag' "$latest_tag" \ + 'HEAD SHA' "$head_sha" \ + 'Latest tag SHA' "$latest_tag_sha" + +# Check that the latest tag and current version match the HEAD we're releasing. +test "$version_version" = "$changes_version" +test "$latest_tag" = "$version_version" +test "$head_sha" = "$latest_tag_sha" +echo 'OK, everything looks good.' diff --git a/doc/Makefile b/doc/Makefile index ef2d60e5f..ddeadbd7e 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -2,6 +2,7 @@ # # You can set these variables from the command line. +BUILDDIR = build SPHINXOPTS = -W SPHINXBUILD = sphinx-build PAPER = @@ -9,7 +10,7 @@ PAPER = # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter -ALLSPHINXOPTS = -d build/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source +ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source .PHONY: help clean html web pickle htmlhelp latex changes linkcheck @@ -24,52 +25,52 @@ help: @echo " linkcheck to check all external links for integrity" clean: - -rm -rf build/* + -rm -rf $(BUILDDIR)/* html: - mkdir -p build/html build/doctrees - $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) build/html + mkdir -p $(BUILDDIR)/html $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo - @echo "Build finished. The HTML pages are in build/html." + @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." pickle: - mkdir -p build/pickle build/doctrees - $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) build/pickle + mkdir -p $(BUILDDIR)/pickle $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." web: pickle json: - mkdir -p build/json build/doctrees - $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) build/json + mkdir -p $(BUILDDIR)/json $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: - mkdir -p build/htmlhelp build/doctrees - $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) build/htmlhelp + mkdir -p $(BUILDDIR)/htmlhelp $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ - ".hhp project file in build/htmlhelp." + ".hhp project file in $(BUILDDIR)/htmlhelp." latex: - mkdir -p build/latex build/doctrees - $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) build/latex + mkdir -p $(BUILDDIR)/latex $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo - @echo "Build finished; the LaTeX files are in build/latex." + @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \ "run these through (pdf)latex." changes: - mkdir -p build/changes build/doctrees - $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) build/changes + mkdir -p $(BUILDDIR)/changes $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo - @echo "The overview file is in build/changes." + @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: - mkdir -p build/linkcheck build/doctrees - $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) build/linkcheck + mkdir -p $(BUILDDIR)/linkcheck $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ - "or in build/linkcheck/output.txt." + "or in $(BUILDDIR)/linkcheck/output.txt." diff --git a/doc/source/changes.rst b/doc/source/changes.rst index 06ec4b72c..a789b068d 100644 --- a/doc/source/changes.rst +++ b/doc/source/changes.rst @@ -2,6 +2,15 @@ Changelog ========= +3.1.37 +====== + +This release contains another security fix that further improves validation of symbolic references +and thus properly fixes this CVE: https://github.com/advisories/GHSA-cwvm-v4w8-q58c . + +See the following for all changes. +https://github.com/gitpython-developers/gitpython/milestone/67?closed=1 + 3.1.36 ====== diff --git a/etc/sublime-text/git-python.sublime-project b/etc/sublime-text/git-python.sublime-project deleted file mode 100644 index 3dab9f656..000000000 --- a/etc/sublime-text/git-python.sublime-project +++ /dev/null @@ -1,62 +0,0 @@ -{ - "folders": - [ - // GIT-PYTHON - ///////////// - { - "follow_symlinks": true, - "path": "../..", - "file_exclude_patterns" : [ - "*.sublime-workspace", - ".git", - ".noseids", - ".coverage" - ], - "folder_exclude_patterns" : [ - ".git", - "cover", - "git/ext", - "dist", - ".tox", - "doc/build", - "*.egg-info" - ] - }, - // GITDB - //////// - { - "follow_symlinks": true, - "path": "../../git/ext/gitdb", - "file_exclude_patterns" : [ - "*.sublime-workspace", - ".git", - ".noseids", - ".coverage" - ], - "folder_exclude_patterns" : [ - ".git", - "cover", - "gitdb/ext", - "dist", - "doc/build", - ".tox", - ] - }, - // // SMMAP - // //////// - { - "follow_symlinks": true, - "path": "../../git/ext/gitdb/gitdb/ext/smmap", - "file_exclude_patterns" : [ - "*.sublime-workspace", - ".git", - ".noseids", - ".coverage" - ], - "folder_exclude_patterns" : [ - ".git", - "cover", - ] - }, - ] -} diff --git a/git/__init__.py b/git/__init__.py index ef7493079..be8338ddc 100644 --- a/git/__init__.py +++ b/git/__init__.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ # flake8: noqa # @PydevCodeAnalysisIgnore from git.exc import * # @NoMove @IgnorePep8 diff --git a/git/cmd.py b/git/cmd.py index d6f8f946a..7c448e3f2 100644 --- a/git/cmd.py +++ b/git/cmd.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from __future__ import annotations import re import contextlib @@ -66,10 +66,10 @@ "with_extended_output", "with_exceptions", "as_process", - "stdout_as_string", "output_stream", - "with_stdout", + "stdout_as_string", "kill_after_timeout", + "with_stdout", "universal_newlines", "shell", "env", @@ -105,7 +105,7 @@ def handle_process_output( ) -> None: """Registers for notifications to learn that process output is ready to read, and dispatches lines to the respective line handlers. - This function returns once the finalizer returns + This function returns once the finalizer returns. :return: result of finalizer :param process: subprocess.Popen instance @@ -294,9 +294,7 @@ def __setstate__(self, d: Dict[str, Any]) -> None: @classmethod def refresh(cls, path: Union[None, PathLike] = None) -> bool: - """This gets called by the refresh function (see the top level - __init__). - """ + """This gets called by the refresh function (see the top level __init__).""" # discern which path to refresh with if path is not None: new_git = os.path.expanduser(path) @@ -446,9 +444,9 @@ def polish_url(cls, url: str, is_cygwin: Union[None, bool] = None) -> PathLike: if is_cygwin: url = cygpath(url) else: - """Remove any backslahes from urls to be written in config files. + """Remove any backslashes from urls to be written in config files. - Windows might create config-files containing paths with backslashed, + Windows might create config files containing paths with backslashes, but git stops liking them as it will escape the backslashes. Hence we undo the escaping just to be sure. """ @@ -464,8 +462,8 @@ def check_unsafe_protocols(cls, url: str) -> None: Check for unsafe protocols. Apart from the usual protocols (http, git, ssh), - Git allows "remote helpers" that have the form `::
`, - one of these helpers (`ext::`) can be used to invoke any arbitrary command. + Git allows "remote helpers" that have the form ``::
``, + one of these helpers (``ext::``) can be used to invoke any arbitrary command. See: @@ -517,7 +515,7 @@ def __init__(self, proc: Union[None, subprocess.Popen], args: Any) -> None: self.status: Union[int, None] = None def _terminate(self) -> None: - """Terminate the underlying process""" + """Terminate the underlying process.""" if self.proc is None: return @@ -572,7 +570,7 @@ def wait(self, stderr: Union[None, str, bytes] = b"") -> int: """Wait for the process and return its status code. :param stderr: Previously read value of stderr, in case stderr is already closed. - :warn: may deadlock if output or error pipes are used and not handled separately. + :warn: May deadlock if output or error pipes are used and not handled separately. :raise GitCommandError: if the return status is not 0""" if stderr is None: stderr_b = b"" @@ -605,13 +603,12 @@ def read_all_from_possibly_closed_stream(stream: Union[IO[bytes], None]) -> byte # END auto interrupt class CatFileContentStream(object): - """Object representing a sized read-only stream returning the contents of an object. It behaves like a stream, but counts the data read and simulates an empty stream once our sized content region is empty. - If not all data is read to the end of the objects's lifetime, we read the - rest to assure the underlying stream continues to work""" + If not all data is read to the end of the object's lifetime, we read the + rest to assure the underlying stream continues to work.""" __slots__: Tuple[str, ...] = ("_stream", "_nbr", "_size") @@ -740,11 +737,11 @@ def __getattr__(self, name: str) -> Any: def set_persistent_git_options(self, **kwargs: Any) -> None: """Specify command line options to the git executable - for subsequent subcommand calls + for subsequent subcommand calls. :param kwargs: is a dict of keyword arguments. - these arguments are passed as in _call_process + These arguments are passed as in _call_process but will be passed to the git command rather than the subcommand. """ @@ -775,7 +772,7 @@ def version_info(self) -> Tuple[int, int, int, int]: """ :return: tuple(int, int, int, int) tuple with integers representing the major, minor and additional version numbers as parsed from git version. - This value is generated on demand and is cached""" + This value is generated on demand and is cached.""" return self._version_info @overload @@ -842,16 +839,16 @@ def execute( strip_newline_in_stdout: bool = True, **subprocess_kwargs: Any, ) -> Union[str, bytes, Tuple[int, Union[str, bytes], str], AutoInterrupt]: - """Handles executing the command on the shell and consumes and returns - the returned information (stdout) + """Handles executing the command and consumes and returns the returned + information (stdout). :param command: The command argument list to execute. - It should be a string, or a sequence of program arguments. The + It should be a sequence of program arguments, or a string. The program to execute is the first item in the args sequence or string. :param istream: - Standard input filehandle passed to subprocess.Popen. + Standard input filehandle passed to `subprocess.Popen`. :param with_extended_output: Whether to return a (status, stdout, stderr) tuple. @@ -862,8 +859,7 @@ def execute( :param as_process: Whether to return the created process instance directly from which streams can be read on demand. This will render with_extended_output and - with_exceptions ineffective - the caller will have - to deal with the details himself. + with_exceptions ineffective - the caller will have to deal with the details. It is important to note that the process will be placed into an AutoInterrupt wrapper that will interrupt the process once it goes out of scope. If you use the command in iterators, you should pass the whole process instance @@ -876,13 +872,34 @@ def execute( always be created with a pipe due to issues with subprocess. This merely is a workaround as data will be copied from the output pipe to the given output stream directly. - Judging from the implementation, you shouldn't use this flag ! + Judging from the implementation, you shouldn't use this parameter! :param stdout_as_string: - if False, the commands standard output will be bytes. Otherwise, it will be - decoded into a string using the default encoding (usually utf-8). + If False, the command's standard output will be bytes. Otherwise, it will be + decoded into a string using the default encoding (usually UTF-8). The latter can fail, if the output contains binary data. + :param kill_after_timeout: + Specifies a timeout in seconds for the git command, after which the process + should be killed. This will have no effect if as_process is set to True. It is + set to None by default and will let the process run until the timeout is + explicitly specified. This feature is not supported on Windows. It's also worth + noting that kill_after_timeout uses SIGKILL, which can have negative side + effects on a repository. For example, stale locks in case of ``git gc`` could + render the repository incapable of accepting changes until the lock is manually + removed. + + :param with_stdout: + If True, default True, we open stdout on the created process. + + :param universal_newlines: + if True, pipes will be opened as text, and lines are split at + all known line endings. + + :param shell: + Whether to invoke commands through a shell (see `Popen(..., shell=True)`). + It overrides :attr:`USE_SHELL` if it is not `None`. + :param env: A dictionary of environment variables to be passed to `subprocess.Popen`. @@ -891,38 +908,23 @@ def execute( one invocation of write() method. If the given number is not positive then the default value is used. + :param strip_newline_in_stdout: + Whether to strip the trailing ``\\n`` of the command stdout. + :param subprocess_kwargs: - Keyword arguments to be passed to subprocess.Popen. Please note that - some of the valid kwargs are already set by this method, the ones you + Keyword arguments to be passed to `subprocess.Popen`. Please note that + some of the valid kwargs are already set by this method; the ones you specify may not be the same ones. - :param with_stdout: If True, default True, we open stdout on the created process - :param universal_newlines: - if True, pipes will be opened as text, and lines are split at - all known line endings. - :param shell: - Whether to invoke commands through a shell (see `Popen(..., shell=True)`). - It overrides :attr:`USE_SHELL` if it is not `None`. - :param kill_after_timeout: - To specify a timeout in seconds for the git command, after which the process - should be killed. This will have no effect if as_process is set to True. It is - set to None by default and will let the process run until the timeout is - explicitly specified. This feature is not supported on Windows. It's also worth - noting that kill_after_timeout uses SIGKILL, which can have negative side - effects on a repository. For example, stale locks in case of git gc could - render the repository incapable of accepting changes until the lock is manually - removed. - :param strip_newline_in_stdout: - Whether to strip the trailing ``\\n`` of the command stdout. :return: * str(output) if extended_output = False (Default) * tuple(int(status), str(stdout), str(stderr)) if extended_output = True - if output_stream is True, the stdout value will be your output stream: + If output_stream is True, the stdout value will be your output stream: * output_stream if extended_output = False * tuple(int(status), output_stream, str(stderr)) if extended_output = True - Note git is executed with LC_MESSAGES="C" to ensure consistent + Note that git is executed with ``LC_MESSAGES="C"`` to ensure consistent output regardless of system language. :raise GitCommandError: @@ -971,16 +973,15 @@ def execute( # end handle stdout_sink = PIPE if with_stdout else getattr(subprocess, "DEVNULL", None) or open(os.devnull, "wb") - istream_ok = "None" - if istream: - istream_ok = "" + if shell is None: + shell = self.USE_SHELL log.debug( - "Popen(%s, cwd=%s, universal_newlines=%s, shell=%s, istream=%s)", + "Popen(%s, cwd=%s, stdin=%s, shell=%s, universal_newlines=%s)", redacted_command, cwd, - universal_newlines, + "" if istream else "None", shell, - istream_ok, + universal_newlines, ) try: with maybe_patch_caller_env: @@ -992,7 +993,7 @@ def execute( stdin=istream or DEVNULL, stderr=PIPE, stdout=stdout_sink, - shell=shell is not None and shell or self.USE_SHELL, + shell=shell, close_fds=is_posix, # unsupported on windows universal_newlines=universal_newlines, creationflags=PROC_CREATIONFLAGS, @@ -1008,8 +1009,8 @@ def execute( if as_process: return self.AutoInterrupt(proc, command) - def _kill_process(pid: int) -> None: - """Callback method to kill a process.""" + def kill_process(pid: int) -> None: + """Callback to kill a process.""" p = Popen( ["ps", "--ppid", str(pid)], stdout=PIPE, @@ -1042,7 +1043,7 @@ def _kill_process(pid: int) -> None: if kill_after_timeout is not None: kill_check = threading.Event() - watchdog = threading.Timer(kill_after_timeout, _kill_process, args=(proc.pid,)) + watchdog = threading.Timer(kill_after_timeout, kill_process, args=(proc.pid,)) # Wait for the process to return status = 0 @@ -1206,7 +1207,7 @@ def _unpack_args(cls, arg_list: Sequence[str]) -> List[str]: def __call__(self, **kwargs: Any) -> "Git": """Specify command line options to the git executable - for a subcommand call + for a subcommand call. :param kwargs: is a dict of keyword arguments. @@ -1244,7 +1245,7 @@ def _call_process( self, method: str, *args: Any, **kwargs: Any ) -> Union[str, bytes, Tuple[int, Union[str, bytes], str], "Git.AutoInterrupt"]: """Run the given git command with the specified arguments and return - the result as a String + the result as a string. :param method: is the command. Contained "_" characters will be converted to dashes, @@ -1253,7 +1254,7 @@ def _call_process( :param args: is the list of arguments. If None is included, it will be pruned. This allows your commands to call git more conveniently as None - is realized as non-existent + is realized as non-existent. :param kwargs: It contains key-values for the following: @@ -1383,7 +1384,7 @@ def get_object_header(self, ref: str) -> Tuple[str, str, int]: return self.__get_object_header(cmd, ref) def get_object_data(self, ref: str) -> Tuple[str, str, int, bytes]: - """As get_object_header, but returns object data as well + """As get_object_header, but returns object data as well. :return: (hexsha, type_string, size_as_int, data_string) :note: not threadsafe""" @@ -1393,10 +1394,10 @@ def get_object_data(self, ref: str) -> Tuple[str, str, int, bytes]: return (hexsha, typename, size, data) def stream_object_data(self, ref: str) -> Tuple[str, str, int, "Git.CatFileContentStream"]: - """As get_object_header, but returns the data as a stream + """As get_object_header, but returns the data as a stream. :return: (hexsha, type_string, size_as_int, stream) - :note: This method is not threadsafe, you need one independent Command instance per thread to be safe !""" + :note: This method is not threadsafe, you need one independent Command instance per thread to be safe!""" cmd = self._get_persistent_cmd("cat_file_all", "cat_file", batch=True) hexsha, typename, size = self.__get_object_header(cmd, ref) cmd_stdout = cmd.stdout if cmd.stdout is not None else io.BytesIO() diff --git a/git/compat.py b/git/compat.py index e7ef28c30..624f26116 100644 --- a/git/compat.py +++ b/git/compat.py @@ -3,7 +3,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ """utilities to help provide compatibility with python 3""" # flake8: noqa diff --git a/git/config.py b/git/config.py index 1973111eb..76b149179 100644 --- a/git/config.py +++ b/git/config.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ """Module containing module parser implementation able to properly read and write configuration files""" @@ -406,15 +406,14 @@ def release(self) -> None: return try: - try: - self.write() - except IOError: - log.error("Exception during destruction of GitConfigParser", exc_info=True) - except ReferenceError: - # This happens in PY3 ... and usually means that some state cannot be written - # as the sections dict cannot be iterated - # Usually when shutting down the interpreter, don'y know how to fix this - pass + self.write() + except IOError: + log.error("Exception during destruction of GitConfigParser", exc_info=True) + except ReferenceError: + # This happens in PY3 ... and usually means that some state cannot be + # written as the sections dict cannot be iterated + # Usually when shutting down the interpreter, don't know how to fix this + pass finally: if self._lock is not None: self._lock._release_lock() diff --git a/git/diff.py b/git/diff.py index 1424ff3ad..3e3de7bc1 100644 --- a/git/diff.py +++ b/git/diff.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import re from git.cmd import handle_process_output diff --git a/git/index/base.py b/git/index/base.py index 193baf3ad..0cdeb1ce5 100644 --- a/git/index/base.py +++ b/git/index/base.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from contextlib import ExitStack import datetime @@ -224,13 +224,11 @@ def write( lfd = LockedFD(file_path or self._file_path) stream = lfd.open(write=True, stream=True) - ok = False try: self._serialize(stream, ignore_extension_data) - ok = True - finally: - if not ok: - lfd.rollback() + except BaseException: + lfd.rollback() + raise lfd.commit() diff --git a/git/objects/base.py b/git/objects/base.py index eb9a8ac3d..1d07fd0f6 100644 --- a/git/objects/base.py +++ b/git/objects/base.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from git.exc import WorkTreeRepositoryUnsupported from git.util import LazyMixin, join_path_native, stream_copy, bin_to_hex diff --git a/git/objects/blob.py b/git/objects/blob.py index 1881f210c..96ce486f5 100644 --- a/git/objects/blob.py +++ b/git/objects/blob.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from mimetypes import guess_type from . import base diff --git a/git/objects/commit.py b/git/objects/commit.py index 6db3ea0f3..88c485d09 100644 --- a/git/objects/commit.py +++ b/git/objects/commit.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import datetime import re from subprocess import Popen, PIPE diff --git a/git/objects/submodule/base.py b/git/objects/submodule/base.py index 0d20305c6..13d897263 100644 --- a/git/objects/submodule/base.py +++ b/git/objects/submodule/base.py @@ -1,44 +1,37 @@ -# need a dict to set bloody .name field from io import BytesIO import logging import os +import os.path as osp import stat import uuid import git from git.cmd import Git -from git.compat import ( - defenc, - is_win, -) -from git.config import SectionConstraint, GitConfigParser, cp +from git.compat import defenc, is_win +from git.config import GitConfigParser, SectionConstraint, cp from git.exc import ( + BadName, InvalidGitRepositoryError, NoSuchPathError, RepositoryDirtyError, - BadName, ) from git.objects.base import IndexObject, Object from git.objects.util import TraversableIterableObj - from git.util import ( - join_path_native, - to_native_path_linux, + IterableList, RemoteProgress, + join_path_native, rmtree, + to_native_path_linux, unbare_repo, - IterableList, ) -from git.util import HIDE_WINDOWS_KNOWN_ERRORS - -import os.path as osp from .util import ( + SubmoduleConfigParser, + find_first_remote_branch, mkhead, sm_name, sm_section, - SubmoduleConfigParser, - find_first_remote_branch, ) @@ -1060,28 +1053,13 @@ def remove( import gc gc.collect() - try: - rmtree(str(wtd)) - except Exception as ex: - if HIDE_WINDOWS_KNOWN_ERRORS: - from unittest import SkipTest - - raise SkipTest("FIXME: fails with: PermissionError\n {}".format(ex)) from ex - raise + rmtree(str(wtd)) # END delete tree if possible # END handle force if not dry_run and osp.isdir(git_dir): self._clear_cache() - try: - rmtree(git_dir) - except Exception as ex: - if HIDE_WINDOWS_KNOWN_ERRORS: - from unittest import SkipTest - - raise SkipTest(f"FIXME: fails with: PermissionError\n {ex}") from ex - else: - raise + rmtree(git_dir) # end handle separate bare repository # END handle module deletion @@ -1403,7 +1381,7 @@ def iter_items( # END handle critical error # Make sure we are looking at a submodule object - if type(sm) != git.objects.submodule.base.Submodule: + if type(sm) is not git.objects.submodule.base.Submodule: continue # fill in remaining info - saves time as it doesn't have to be parsed again diff --git a/git/objects/tag.py b/git/objects/tag.py index 3956a89e7..56fd05d1a 100644 --- a/git/objects/tag.py +++ b/git/objects/tag.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ """ Module containing all object based types. """ from . import base from .util import get_object_type_by_name, parse_actor_and_date diff --git a/git/objects/tree.py b/git/objects/tree.py index a9b491e23..4f490af54 100644 --- a/git/objects/tree.py +++ b/git/objects/tree.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from git.util import IterableList, join_path import git.diff as git_diff diff --git a/git/objects/util.py b/git/objects/util.py index 56938507e..992a53d9c 100644 --- a/git/objects/util.py +++ b/git/objects/util.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ """Module for general utility functions""" # flake8: noqa F401 diff --git a/git/refs/log.py b/git/refs/log.py index 1f86356a4..ef3f86b8b 100644 --- a/git/refs/log.py +++ b/git/refs/log.py @@ -244,7 +244,7 @@ def entry_at(cls, filepath: PathLike, index: int) -> "RefLogEntry": for i in range(index + 1): line = fp.readline() if not line: - raise IndexError(f"Index file ended at line {i+1}, before given index was reached") + raise IndexError(f"Index file ended at line {i + 1}, before given index was reached") # END abort on eof # END handle runup @@ -262,8 +262,7 @@ def to_file(self, filepath: PathLike) -> None: try: self._serialize(fp) lfd.commit() - except Exception: - # on failure it rolls back automatically, but we make it clear + except BaseException: lfd.rollback() raise # END handle change diff --git a/git/refs/symbolic.py b/git/refs/symbolic.py index 5c293aa7b..549160444 100644 --- a/git/refs/symbolic.py +++ b/git/refs/symbolic.py @@ -161,6 +161,51 @@ def dereference_recursive(cls, repo: "Repo", ref_path: Union[PathLike, None]) -> return hexsha # END recursive dereferencing + @staticmethod + def _check_ref_name_valid(ref_path: PathLike) -> None: + # Based on the rules described in https://git-scm.com/docs/git-check-ref-format/#_description + previous: Union[str, None] = None + one_before_previous: Union[str, None] = None + for c in str(ref_path): + if c in " ~^:?*[\\": + raise ValueError( + f"Invalid reference '{ref_path}': references cannot contain spaces, tildes (~), carets (^)," + f" colons (:), question marks (?), asterisks (*), open brackets ([) or backslashes (\\)" + ) + elif c == ".": + if previous is None or previous == "/": + raise ValueError( + f"Invalid reference '{ref_path}': references cannot start with a period (.) or contain '/.'" + ) + elif previous == ".": + raise ValueError(f"Invalid reference '{ref_path}': references cannot contain '..'") + elif c == "/": + if previous == "/": + raise ValueError(f"Invalid reference '{ref_path}': references cannot contain '//'") + elif previous is None: + raise ValueError( + f"Invalid reference '{ref_path}': references cannot start with forward slashes '/'" + ) + elif c == "{" and previous == "@": + raise ValueError(f"Invalid reference '{ref_path}': references cannot contain '@{{'") + elif ord(c) < 32 or ord(c) == 127: + raise ValueError(f"Invalid reference '{ref_path}': references cannot contain ASCII control characters") + + one_before_previous = previous + previous = c + + if previous == ".": + raise ValueError(f"Invalid reference '{ref_path}': references cannot end with a period (.)") + elif previous == "/": + raise ValueError(f"Invalid reference '{ref_path}': references cannot end with a forward slash (/)") + elif previous == "@" and one_before_previous is None: + raise ValueError(f"Invalid reference '{ref_path}': references cannot be '@'") + elif any(component.endswith(".lock") for component in str(ref_path).split("/")): + raise ValueError( + f"Invalid reference '{ref_path}': references cannot have slash-separated components that end with" + f" '.lock'" + ) + @classmethod def _get_ref_info_helper( cls, repo: "Repo", ref_path: Union[PathLike, None] @@ -168,8 +213,9 @@ def _get_ref_info_helper( """Return: (str(sha), str(target_ref_path)) if available, the sha the file at rela_path points to, or None. target_ref_path is the reference we point to, or None""" - if ".." in str(ref_path): - raise ValueError(f"Invalid reference '{ref_path}'") + if ref_path: + cls._check_ref_name_valid(ref_path) + tokens: Union[None, List[str], Tuple[str, str]] = None repodir = _git_dir(repo, ref_path) try: @@ -370,14 +416,12 @@ def set_reference( lfd = LockedFD(fpath) fd = lfd.open(write=True, stream=True) - ok = True try: fd.write(write_value.encode("utf-8") + b"\n") lfd.commit() - ok = True - finally: - if not ok: - lfd.rollback() + except BaseException: + lfd.rollback() + raise # Adjust the reflog if logmsg is not None: self.log_append(oldbinsha, logmsg) diff --git a/git/remote.py b/git/remote.py index 95a2b8ac6..fc2b2ceba 100644 --- a/git/remote.py +++ b/git/remote.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ # Module implementing a remote object allowing easy access to git remotes import logging diff --git a/git/repo/base.py b/git/repo/base.py index 113fca459..bc1b8876d 100644 --- a/git/repo/base.py +++ b/git/repo/base.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from __future__ import annotations import logging import os @@ -206,7 +206,8 @@ def __init__( if expand_vars and re.search(self.re_envvars, epath): warnings.warn( "The use of environment variables in paths is deprecated" - + "\nfor security reasons and may be removed in the future!!" + + "\nfor security reasons and may be removed in the future!!", + stacklevel=1, ) epath = expand_path(epath, expand_vars) if epath is not None: diff --git a/git/types.py b/git/types.py index 9f8621721..21276b5f1 100644 --- a/git/types.py +++ b/git/types.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ # flake8: noqa import os diff --git a/git/util.py b/git/util.py index 636e79806..97f461a83 100644 --- a/git/util.py +++ b/git/util.py @@ -2,27 +2,27 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from abc import abstractmethod -import os.path as osp -from .compat import is_win import contextlib from functools import wraps import getpass import logging import os +import os.path as osp +import pathlib import platform -import subprocess import re import shutil import stat -from sys import maxsize +import subprocess +import sys import time from urllib.parse import urlsplit, urlunsplit import warnings -# from git.objects.util import Traversable +from .compat import is_win # typing --------------------------------------------------------- @@ -42,22 +42,17 @@ Tuple, TypeVar, Union, - cast, TYPE_CHECKING, + cast, overload, ) -import pathlib - if TYPE_CHECKING: from git.remote import Remote from git.repo.base import Repo from git.config import GitConfigParser, SectionConstraint from git import Git - # from git.objects.base import IndexObject - - from .types import ( Literal, SupportsIndex, @@ -75,7 +70,6 @@ # --------------------------------------------------------------------- - from gitdb.util import ( # NOQA @IgnorePep8 make_sha, LockedFD, # @UnusedImport @@ -88,7 +82,6 @@ hex_to_bin, # @UnusedImport ) - # NOTE: Some of the unused imports might be used/imported by others. # Handle once test-cases are back up and running. # Most of these are unused here, but are for use by git-python modules so these @@ -116,14 +109,33 @@ log = logging.getLogger(__name__) -# types############################################################ + +def _read_env_flag(name: str, default: bool) -> bool: + try: + value = os.environ[name] + except KeyError: + return default + + log.warning( + "The %s environment variable is deprecated. Its effect has never been documented and changes without warning.", + name, + ) + + adjusted_value = value.strip().lower() + + if adjusted_value in {"", "0", "false", "no"}: + return False + if adjusted_value in {"1", "true", "yes"}: + return True + log.warning("%s has unrecognized value %r, treating as %r.", name, value, default) + return default #: We need an easy way to see if Appveyor TCs start failing, #: so the errors marked with this var are considered "acknowledged" ones, awaiting remedy, #: till then, we wish to hide them. -HIDE_WINDOWS_KNOWN_ERRORS = is_win and os.environ.get("HIDE_WINDOWS_KNOWN_ERRORS", True) -HIDE_WINDOWS_FREEZE_ERRORS = is_win and os.environ.get("HIDE_WINDOWS_FREEZE_ERRORS", True) +HIDE_WINDOWS_KNOWN_ERRORS = is_win and _read_env_flag("HIDE_WINDOWS_KNOWN_ERRORS", True) +HIDE_WINDOWS_FREEZE_ERRORS = is_win and _read_env_flag("HIDE_WINDOWS_FREEZE_ERRORS", True) # { Utility Methods @@ -177,25 +189,29 @@ def patch_env(name: str, value: str) -> Generator[None, None, None]: def rmtree(path: PathLike) -> None: - """Remove the given recursively. + """Remove the given directory tree recursively. - :note: we use shutil rmtree but adjust its behaviour to see whether files that - couldn't be deleted are read-only. Windows will not remove them in that case""" + :note: We use :func:`shutil.rmtree` but adjust its behaviour to see whether files that + couldn't be deleted are read-only. Windows will not remove them in that case.""" - def onerror(func: Callable, path: PathLike, exc_info: str) -> None: - # Is the error an access error ? + def handler(function: Callable, path: PathLike, _excinfo: Any) -> None: + """Callback for :func:`shutil.rmtree`. Works either as ``onexc`` or ``onerror``.""" + # Is the error an access error? os.chmod(path, stat.S_IWUSR) try: - func(path) # Will scream if still not possible to delete. - except Exception as ex: + function(path) + except PermissionError as ex: if HIDE_WINDOWS_KNOWN_ERRORS: from unittest import SkipTest - raise SkipTest("FIXME: fails with: PermissionError\n {}".format(ex)) from ex + raise SkipTest(f"FIXME: fails with: PermissionError\n {ex}") from ex raise - return shutil.rmtree(path, False, onerror) + if sys.version_info >= (3, 12): + shutil.rmtree(path, onexc=handler) + else: + shutil.rmtree(path, onerror=handler) def rmfile(path: PathLike) -> None: @@ -995,7 +1011,7 @@ def __init__( self, file_path: PathLike, check_interval_s: float = 0.3, - max_block_time_s: int = maxsize, + max_block_time_s: int = sys.maxsize, ) -> None: """Configure the instance @@ -1136,7 +1152,7 @@ class IterableClassWatcher(type): def __init__(cls, name: str, bases: Tuple, clsdict: Dict) -> None: for base in bases: - if type(base) == IterableClassWatcher: + if type(base) is IterableClassWatcher: warnings.warn( f"GitPython Iterable subclassed by {name}. " "Iterable is deprecated due to naming clash since v3.1.18" diff --git a/init-tests-after-clone.sh b/init-tests-after-clone.sh index 95ced98b7..21d1f86d8 100755 --- a/init-tests-after-clone.sh +++ b/init-tests-after-clone.sh @@ -1,18 +1,73 @@ -#!/usr/bin/env bash +#!/bin/sh -set -e +set -eu -if [[ -z "$TRAVIS" ]]; then - read -rp "This operation will destroy locally modified files. Continue ? [N/y]: " answer - if [[ ! $answer =~ [yY] ]]; then - exit 2 - fi +fallback_repo_for_tags='https://github.com/gitpython-developers/GitPython.git' + +ci() { + # For now, check just these, as a false positive could lead to data loss. + test -n "${TRAVIS-}" || test -n "${GITHUB_ACTIONS-}" +} + +no_version_tags() { + test -z "$(git tag -l '[0-9]*' 'v[0-9]*')" +} + +warn() { + if test -n "${GITHUB_ACTIONS-}"; then + printf '::warning ::%s\n' "$*" >&2 # Annotate workflow. + else + printf '%s\n' "$@" >&2 + fi +} + +if ! ci; then + printf 'This operation will destroy locally modified files. Continue ? [N/y]: ' >&2 + read -r answer + case "$answer" in + [yY]) + ;; + *) + exit 2 ;; + esac fi +# Stop if we have run this. (You can delete __testing_point__ to let it rerun.) +# This also keeps track of where we are, so we can get back here. git tag __testing_point__ -git checkout master || git checkout -b master + +# The tests need a branch called master. +git checkout master -- || git checkout -b master + +# The tests need a reflog history on the master branch. git reset --hard HEAD~1 git reset --hard HEAD~1 git reset --hard HEAD~1 + +# Point the master branch where we started, so we test the correct code. git reset --hard __testing_point__ -git submodule update --init --recursive + +# The tests need submodules. (On CI, they would already have been checked out.) +if ! ci; then + git submodule update --init --recursive +fi + +# The tests need some version tags. Try to get them even in forks. This fetches +# other objects too. So, locally, we always do it, for a consistent experience. +if ! ci || no_version_tags; then + git fetch --all --tags +fi + +# If we still have no version tags, try to get them from the original repo. +if no_version_tags; then + warn 'No local or remote version tags found. Trying fallback remote:' \ + "$fallback_repo_for_tags" + + # git fetch supports * but not [], and --no-tags means no *other* tags, so... + printf 'refs/tags/%d*:refs/tags/%d*\n' 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 | + xargs git fetch --no-tags "$fallback_repo_for_tags" + + if no_version_tags; then + warn 'No version tags found anywhere. Some tests will fail.' + fi +fi diff --git a/pyproject.toml b/pyproject.toml index 42bb31eda..f4fc33fec 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta" [tool.pytest.ini_options] python_files = 'test_*.py' testpaths = 'test' # space separated list of paths from root e.g test tests doc/testing -addopts = '--cov=git --cov-report=term --maxfail=10 --force-sugar --disable-warnings' +addopts = '--cov=git --cov-report=term --disable-warnings' filterwarnings = 'ignore::DeprecationWarning' # --cov coverage # --cov-report term # send report to terminal term-missing -> terminal with line numbers html xml @@ -45,4 +45,4 @@ omit = ["*/git/ext/*"] [tool.black] line-length = 120 target-version = ['py37'] -exclude = "git/ext/gitdb" +extend-exclude = "git/ext/gitdb" diff --git a/requirements-dev.txt b/requirements-dev.txt index f6705341c..e3030c597 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -7,6 +7,3 @@ flake8-type-checking;python_version>="3.8" # checks for TYPE_CHECKING only pytest-icdiff # pytest-profiling - - -tox diff --git a/setup.py b/setup.py index bc53bf6c8..90df8d7ea 100755 --- a/setup.py +++ b/setup.py @@ -4,7 +4,6 @@ from setuptools import setup, find_packages from setuptools.command.build_py import build_py as _build_py from setuptools.command.sdist import sdist as _sdist -import fnmatch import os import sys @@ -62,24 +61,6 @@ def _stamp_version(filename: str) -> None: print("WARNING: Couldn't find version line in file %s" % filename, file=sys.stderr) -def build_py_modules(basedir: str, excludes: Sequence = ()) -> Sequence: - # create list of py_modules from tree - res = set() - _prefix = os.path.basename(basedir) - for root, _, files in os.walk(basedir): - for f in files: - _f, _ext = os.path.splitext(f) - if _ext not in [".py"]: - continue - _f = os.path.join(root, _f) - _f = os.path.relpath(_f, basedir) - _f = "{}.{}".format(_prefix, _f.replace(os.sep, ".")) - if any(fnmatch.fnmatch(_f, x) for x in excludes): - continue - res.add(_f) - return list(res) - - setup( name="GitPython", cmdclass={"build_py": build_py, "sdist": sdist}, @@ -91,7 +72,6 @@ def build_py_modules(basedir: str, excludes: Sequence = ()) -> Sequence: url="https://github.com/gitpython-developers/GitPython", packages=find_packages(exclude=["test", "test.*"]), include_package_data=True, - py_modules=build_py_modules("./git", excludes=["git.ext.*"]), package_dir={"git": "git"}, python_requires=">=3.7", install_requires=requirements, diff --git a/test-requirements.txt b/test-requirements.txt index 62f409824..a69181be1 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -1,9 +1,11 @@ black coverage[toml] -ddt>=1.1.1, !=1.4.3 +ddt >= 1.1.1, != 1.4.3 +mock ; python_version < "3.8" mypy pre-commit pytest pytest-cov +pytest-instafail +pytest-subtests pytest-sugar -virtualenv diff --git a/test/__init__.py b/test/__init__.py index 757cbad1f..a3d514523 100644 --- a/test/__init__.py +++ b/test/__init__.py @@ -2,4 +2,4 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ diff --git a/test/lib/__init__.py b/test/lib/__init__.py index a4e57b8e0..299317c0b 100644 --- a/test/lib/__init__.py +++ b/test/lib/__init__.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ # flake8: noqa import inspect diff --git a/test/lib/helper.py b/test/lib/helper.py index c04c5cd90..d415ba2e7 100644 --- a/test/lib/helper.py +++ b/test/lib/helper.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import contextlib from functools import wraps import gc @@ -94,17 +94,16 @@ def wrapper(self): os.mkdir(path) keep = False try: - try: - return func(self, path) - except Exception: - log.info( - "Test %s.%s failed, output is at %r\n", - type(self).__name__, - func.__name__, - path, - ) - keep = True - raise + return func(self, path) + except Exception: + log.info( + "Test %s.%s failed, output is at %r\n", + type(self).__name__, + func.__name__, + path, + ) + keep = True + raise finally: # Need to collect here to be sure all handles have been closed. It appears # a windows-only issue. In fact things should be deleted, as well as @@ -147,12 +146,11 @@ def repo_creator(self): prev_cwd = os.getcwd() os.chdir(rw_repo.working_dir) try: - try: - return func(self, rw_repo) - except: # noqa E722 - log.info("Keeping repo after failure: %s", repo_dir) - repo_dir = None - raise + return func(self, rw_repo) + except: # noqa E722 + log.info("Keeping repo after failure: %s", repo_dir) + repo_dir = None + raise finally: os.chdir(prev_cwd) rw_repo.git.clear_cache() @@ -179,14 +177,12 @@ def git_daemon_launched(base_path, ip, port): gd = None try: if is_win: - ## On MINGW-git, daemon exists in .\Git\mingw64\libexec\git-core\, - # but if invoked as 'git daemon', it detaches from parent `git` cmd, - # and then CANNOT DIE! - # So, invoke it as a single command. - ## Cygwin-git has no daemon. But it can use MINGW's. - # + # On MINGW-git, daemon exists in Git\mingw64\libexec\git-core\, + # but if invoked as 'git daemon', it detaches from parent `git` cmd, + # and then CANNOT DIE! + # So, invoke it as a single command. daemon_cmd = [ - "git-daemon", + osp.join(Git()._call_process("--exec-path"), "git-daemon"), "--enable=receive-pack", "--listen=%s" % ip, "--port=%s" % port, @@ -219,12 +215,11 @@ def git_daemon_launched(base_path, ip, port): ) if is_win: msg += textwrap.dedent( - r""" + R""" On Windows, the `git-daemon.exe` must be in PATH. - For MINGW, look into .\Git\mingw64\libexec\git-core\), but problems with paths might appear. - CYGWIN has no daemon, but if one exists, it gets along fine (but has also paths problems).""" + For MINGW, look into \Git\mingw64\libexec\git-core\, but problems with paths might appear.""" ) log.warning(msg, ex, ip, port, base_path, base_path, exc_info=1) @@ -307,7 +302,7 @@ def remote_repo_creator(self): cw.set("url", remote_repo_url) with git_daemon_launched( - Git.polish_url(base_daemon_path, is_cygwin=False), # No daemon in Cygwin. + Git.polish_url(base_daemon_path), "127.0.0.1", GIT_DAEMON_PORT, ): diff --git a/test/performance/test_commit.py b/test/performance/test_commit.py index 38b529af7..dbe2ad43e 100644 --- a/test/performance/test_commit.py +++ b/test/performance/test_commit.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from io import BytesIO from time import time import sys diff --git a/test/performance/test_streams.py b/test/performance/test_streams.py index 5588212e0..25e081578 100644 --- a/test/performance/test_streams.py +++ b/test/performance/test_streams.py @@ -15,7 +15,6 @@ class TestObjDBPerformance(TestBigRepoR): - large_data_size_bytes = 1000 * 1000 * 10 # some MiB should do it moderate_data_size_bytes = 1000 * 1000 * 1 # just 1 MiB diff --git a/test/test_actor.py b/test/test_actor.py index ce0c74fc9..f495ac084 100644 --- a/test/test_actor.py +++ b/test/test_actor.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from test.lib import TestBase from git import Actor diff --git a/test/test_base.py b/test/test_base.py index 30029367d..90e701c4b 100644 --- a/test/test_base.py +++ b/test/test_base.py @@ -3,11 +3,11 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import os import sys import tempfile -from unittest import SkipTest, skipIf +from unittest import skipIf from git import Repo from git.objects import Blob, Tree, Commit, TagObject @@ -126,7 +126,7 @@ def test_add_unicode(self, rw_repo): try: file_path.encode(sys.getfilesystemencoding()) except UnicodeEncodeError as e: - raise SkipTest("Environment doesn't support unicode filenames") from e + raise RuntimeError("Environment doesn't support unicode filenames") from e with open(file_path, "wb") as fp: fp.write(b"something") diff --git a/test/test_blob.py b/test/test_blob.py index b94dcec23..692522b52 100644 --- a/test/test_blob.py +++ b/test/test_blob.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from test.lib import TestBase from git import Blob diff --git a/test/test_clone.py b/test/test_clone.py index 304ab33cb..1b4a6c332 100644 --- a/test/test_clone.py +++ b/test/test_clone.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from pathlib import Path import re diff --git a/test/test_commit.py b/test/test_commit.py index 4871902ec..527aea334 100644 --- a/test/test_commit.py +++ b/test/test_commit.py @@ -3,7 +3,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import copy from datetime import datetime from io import BytesIO @@ -93,7 +93,6 @@ def assert_commit_serialization(self, rwrepo, commit_id, print_performance_info= class TestCommit(TestCommitSerialization): def test_bake(self): - commit = self.rorepo.commit("2454ae89983a4496a445ce347d7a41c0bb0ea7ae") # commits have no dict self.assertRaises(AttributeError, setattr, commit, "someattr", 1) @@ -170,15 +169,15 @@ def test_renames(self): def check_entries(path, changes): expected = { - ".github/workflows/Future.yml" : { - 'insertions': 57, - 'deletions': 0, - 'lines': 57 + ".github/workflows/Future.yml": { + "insertions": 57, + "deletions": 0, + "lines": 57, }, - ".github/workflows/test_pytest.yml" : { - 'insertions': 0, - 'deletions': 55, - 'lines': 55 + ".github/workflows/test_pytest.yml": { + "insertions": 0, + "deletions": 55, + "lines": 55, }, } assert path in expected @@ -278,7 +277,7 @@ def __init__(self, *args, **kwargs): super(Child, self).__init__(*args, **kwargs) child_commits = list(Child.iter_items(self.rorepo, "master", paths=("CHANGES", "AUTHORS"))) - assert type(child_commits[0]) == Child + assert type(child_commits[0]) is Child def test_iter_items(self): # pretty not allowed @@ -526,12 +525,12 @@ def test_trailers(self): # check that trailer stays empty for multiple msg combinations msgs = [ - f"Subject\n", - f"Subject\n\nBody with some\nText\n", - f"Subject\n\nBody with\nText\n\nContinuation but\n doesn't contain colon\n", - f"Subject\n\nBody with\nText\n\nContinuation but\n only contains one :\n", - f"Subject\n\nBody with\nText\n\nKey: Value\nLine without colon\n", - f"Subject\n\nBody with\nText\n\nLine without colon\nKey: Value\n", + "Subject\n", + "Subject\n\nBody with some\nText\n", + "Subject\n\nBody with\nText\n\nContinuation but\n doesn't contain colon\n", + "Subject\n\nBody with\nText\n\nContinuation but\n only contains one :\n", + "Subject\n\nBody with\nText\n\nKey: Value\nLine without colon\n", + "Subject\n\nBody with\nText\n\nLine without colon\nKey: Value\n", ] for msg in msgs: diff --git a/test/test_config.py b/test/test_config.py index b159ebe2d..f805570d5 100644 --- a/test/test_config.py +++ b/test/test_config.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import glob import io @@ -100,6 +100,7 @@ def test_includes_order(self): # values must be considered as soon as they get them assert r_config.get_value("diff", "tool") == "meld" try: + # FIXME: Split this assertion out somehow and mark it xfail (or fix it). assert r_config.get_value("sec", "var1") == "value1_main" except AssertionError as e: raise SkipTest("Known failure -- included values are not in effect right away") from e diff --git a/test/test_db.py b/test/test_db.py index 228c70e7c..ebf73b535 100644 --- a/test/test_db.py +++ b/test/test_db.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from git.db import GitCmdObjectDB from git.exc import BadObject from test.lib import TestBase diff --git a/test/test_diff.py b/test/test_diff.py index 504337744..5aa4408bf 100644 --- a/test/test_diff.py +++ b/test/test_diff.py @@ -3,11 +3,10 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import ddt import shutil import tempfile -import unittest from git import ( Repo, GitCommandError, @@ -414,12 +413,12 @@ def test_diff_interface(self): @with_rw_directory def test_rename_override(self, rw_dir): - """Test disabling of diff rename detection""" + """Test disabling of diff rename detection""" # create and commit file_a.txt repo = Repo.init(rw_dir) file_a = osp.join(rw_dir, "file_a.txt") - with open(file_a, "w", encoding='utf-8') as outfile: + with open(file_a, "w", encoding="utf-8") as outfile: outfile.write("hello world\n") repo.git.add(Git.polish_url(file_a)) repo.git.commit(message="Added file_a.txt") @@ -429,21 +428,21 @@ def test_rename_override(self, rw_dir): # create and commit file_b.txt with similarity index of 52 file_b = osp.join(rw_dir, "file_b.txt") - with open(file_b, "w", encoding='utf-8') as outfile: + with open(file_b, "w", encoding="utf-8") as outfile: outfile.write("hello world\nhello world") repo.git.add(Git.polish_url(file_b)) repo.git.commit(message="Removed file_a.txt. Added file_b.txt") - commit_a = repo.commit('HEAD') - commit_b = repo.commit('HEAD~1') + commit_a = repo.commit("HEAD") + commit_b = repo.commit("HEAD~1") # check default diff command with renamed files enabled diffs = commit_b.diff(commit_a) self.assertEqual(1, len(diffs)) diff = diffs[0] self.assertEqual(True, diff.renamed_file) - self.assertEqual('file_a.txt', diff.rename_from) - self.assertEqual('file_b.txt', diff.rename_to) + self.assertEqual("file_a.txt", diff.rename_from) + self.assertEqual("file_b.txt", diff.rename_to) # check diff with rename files disabled diffs = commit_b.diff(commit_a, no_renames=True) @@ -452,32 +451,31 @@ def test_rename_override(self, rw_dir): # check fileA.txt deleted diff = diffs[0] self.assertEqual(True, diff.deleted_file) - self.assertEqual('file_a.txt', diff.a_path) + self.assertEqual("file_a.txt", diff.a_path) # check fileB.txt added diff = diffs[1] self.assertEqual(True, diff.new_file) - self.assertEqual('file_b.txt', diff.a_path) + self.assertEqual("file_b.txt", diff.a_path) # check diff with high similarity index - diffs = commit_b.diff(commit_a, split_single_char_options=False, M='75%') + diffs = commit_b.diff(commit_a, split_single_char_options=False, M="75%") self.assertEqual(2, len(diffs)) # check fileA.txt deleted diff = diffs[0] self.assertEqual(True, diff.deleted_file) - self.assertEqual('file_a.txt', diff.a_path) + self.assertEqual("file_a.txt", diff.a_path) # check fileB.txt added diff = diffs[1] self.assertEqual(True, diff.new_file) - self.assertEqual('file_b.txt', diff.a_path) + self.assertEqual("file_b.txt", diff.a_path) # check diff with low similarity index - diffs = commit_b.diff(commit_a, split_single_char_options=False, M='40%') + diffs = commit_b.diff(commit_a, split_single_char_options=False, M="40%") self.assertEqual(1, len(diffs)) diff = diffs[0] self.assertEqual(True, diff.renamed_file) - self.assertEqual('file_a.txt', diff.rename_from) - self.assertEqual('file_b.txt', diff.rename_to) - + self.assertEqual("file_a.txt", diff.rename_from) + self.assertEqual("file_b.txt", diff.rename_to) diff --git a/test/test_docs.py b/test/test_docs.py index 20027c191..d1ed46926 100644 --- a/test/test_docs.py +++ b/test/test_docs.py @@ -3,7 +3,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import os import sys @@ -21,7 +21,10 @@ def tearDown(self): gc.collect() - # @skipIf(HIDE_WINDOWS_KNOWN_ERRORS, ## ACTUALLY skipped by `git.submodule.base#L869`. + # ACTUALLY skipped by git.util.rmtree (in local onerror function), from the last call to it via + # git.objects.submodule.base.Submodule.remove (at "handle separate bare repository"), line 1062. + # + # @skipIf(HIDE_WINDOWS_KNOWN_ERRORS, # "FIXME: helper.wrapper fails with: PermissionError: [WinError 5] Access is denied: " # "'C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\test_work_tree_unsupportedryfa60di\\master_repo\\.git\\objects\\pack\\pack-bc9e0787aef9f69e1591ef38ea0a6f566ec66fe3.idx") # noqa E501 @with_rw_directory @@ -167,7 +170,7 @@ def update(self, op_code, cur_count, max_count=None, message=""): open(new_file_path, "wb").close() # create new file in working tree cloned_repo.index.add([new_file_path]) # add it to the index # Commit the changes to deviate masters history - cloned_repo.index.commit("Added a new file in the past - for later merege") + cloned_repo.index.commit("Added a new file in the past - for later merge") # prepare a merge master = cloned_repo.heads.master # right-hand side is ahead of us, in the future @@ -198,7 +201,7 @@ def update(self, op_code, cur_count, max_count=None, message=""): # .gitmodules was written and added to the index, which is now being committed cloned_repo.index.commit("Added submodule") - assert sm.exists() and sm.module_exists() # this submodule is defintely available + assert sm.exists() and sm.module_exists() # this submodule is definitely available sm.remove(module=True, configuration=False) # remove the working tree assert sm.exists() and not sm.module_exists() # the submodule itself is still available @@ -263,9 +266,9 @@ def test_references_and_objects(self, rw_dir): # [8-test_references_and_objects] hc = repo.head.commit hct = hc.tree - hc != hct # @NoEffect - hc != repo.tags[0] # @NoEffect - hc == repo.head.reference.commit # @NoEffect + assert hc != hct + assert hc != repo.tags[0] + assert hc == repo.head.reference.commit # ![8-test_references_and_objects] # [9-test_references_and_objects] @@ -369,7 +372,7 @@ def test_references_and_objects(self, rw_dir): # The index contains all blobs in a flat list assert len(list(index.iter_blobs())) == len([o for o in repo.head.commit.tree.traverse() if o.type == "blob"]) # Access blob objects - for (_path, _stage), entry in index.entries.items(): + for (_path, _stage), _entry in index.entries.items(): pass new_file_path = os.path.join(repo.working_tree_dir, "new-file-name") open(new_file_path, "w").close() @@ -481,7 +484,7 @@ def test_references_and_objects(self, rw_dir): @pytest.mark.xfail( sys.platform == "cygwin", reason="Cygwin GitPython can't find SHA for submodule", - raises=ValueError + raises=ValueError, ) def test_submodules(self): # [1-test_submodules] diff --git a/test/test_exc.py b/test/test_exc.py index f998ff4d5..9e125d246 100644 --- a/test/test_exc.py +++ b/test/test_exc.py @@ -3,7 +3,7 @@ # Copyright (C) 2008, 2009, 2016 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import re diff --git a/test/test_fun.py b/test/test_fun.py index d76e189ed..f39955aa0 100644 --- a/test/test_fun.py +++ b/test/test_fun.py @@ -2,7 +2,6 @@ from stat import S_IFDIR, S_IFREG, S_IFLNK, S_IXUSR from os import stat import os.path as osp -from unittest import SkipTest from git import Git from git.index import IndexFile @@ -279,7 +278,7 @@ def test_linked_worktree_traversal(self, rw_dir): """Check that we can identify a linked worktree based on a .git file""" git = Git(rw_dir) if git.version_info[:3] < (2, 5, 1): - raise SkipTest("worktree feature unsupported") + raise RuntimeError("worktree feature unsupported (test needs git 2.5.1 or later)") rw_master = self.rorepo.clone(join_path_native(rw_dir, "master_repo")) branch = rw_master.create_head("aaaaaaaa") diff --git a/test/test_git.py b/test/test_git.py index 4d57a2d86..cf82d9ac7 100644 --- a/test/test_git.py +++ b/test/test_git.py @@ -3,24 +3,32 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ +import inspect +import logging import os +import os.path as osp +import re import shutil import subprocess import sys from tempfile import TemporaryDirectory, TemporaryFile -from unittest import mock, skipUnless +from unittest import skipUnless -from git import Git, refresh, GitCommandError, GitCommandNotFound, Repo, cmd -from test.lib import TestBase, fixture_path -from test.lib import with_rw_directory -from git.util import cwd, finalize_process +if sys.version_info >= (3, 8): + from unittest import mock +else: + import mock # To be able to examine call_args.kwargs on a mock. -import os.path as osp +import ddt +from git import Git, refresh, GitCommandError, GitCommandNotFound, Repo, cmd from git.compat import is_win +from git.util import cwd, finalize_process +from test.lib import TestBase, fixture_path, with_rw_directory +@ddt.ddt class TestGit(TestBase): @classmethod def setUpClass(cls): @@ -32,6 +40,13 @@ def tearDown(self): gc.collect() + def _assert_logged_for_popen(self, log_watcher, name, value): + re_name = re.escape(name) + re_value = re.escape(str(value)) + re_line = re.compile(rf"DEBUG:git.cmd:Popen\(.*\b{re_name}={re_value}[,)]") + match_attempts = [re_line.match(message) for message in log_watcher.output] + self.assertTrue(any(match_attempts), repr(log_watcher.output)) + @mock.patch.object(Git, "execute") def test_call_process_calls_execute(self, git): git.return_value = "" @@ -73,7 +88,53 @@ def test_it_transforms_kwargs_into_git_command_arguments(self): res = self.git.transform_kwargs(**{"s": True, "t": True}) self.assertEqual({"-s", "-t"}, set(res)) - def test_it_executes_git_to_shell_and_returns_result(self): + _shell_cases = ( + # value_in_call, value_from_class, expected_popen_arg + (None, False, False), + (None, True, True), + (False, True, False), + (False, False, False), + (True, False, True), + (True, True, True), + ) + + def _do_shell_combo(self, value_in_call, value_from_class): + with mock.patch.object(Git, "USE_SHELL", value_from_class): + # git.cmd gets Popen via a "from" import, so patch it there. + with mock.patch.object(cmd, "Popen", wraps=cmd.Popen) as mock_popen: + # Use a command with no arguments (besides the program name), so it runs + # with or without a shell, on all OSes, with the same effect. + self.git.execute(["git"], with_exceptions=False, shell=value_in_call) + + return mock_popen + + @ddt.idata(_shell_cases) + def test_it_uses_shell_or_not_as_specified(self, case): + """A bool passed as ``shell=`` takes precedence over `Git.USE_SHELL`.""" + value_in_call, value_from_class, expected_popen_arg = case + mock_popen = self._do_shell_combo(value_in_call, value_from_class) + mock_popen.assert_called_once() + self.assertIs(mock_popen.call_args.kwargs["shell"], expected_popen_arg) + + @ddt.idata(full_case[:2] for full_case in _shell_cases) + def test_it_logs_if_it_uses_a_shell(self, case): + """``shell=`` in the log message agrees with what is passed to `Popen`.""" + value_in_call, value_from_class = case + with self.assertLogs(cmd.log, level=logging.DEBUG) as log_watcher: + mock_popen = self._do_shell_combo(value_in_call, value_from_class) + self._assert_logged_for_popen(log_watcher, "shell", mock_popen.call_args.kwargs["shell"]) + + @ddt.data( + ("None", None), + ("", subprocess.PIPE), + ) + def test_it_logs_istream_summary_for_stdin(self, case): + expected_summary, istream_argument = case + with self.assertLogs(cmd.log, level=logging.DEBUG) as log_watcher: + self.git.execute(["git", "version"], istream=istream_argument) + self._assert_logged_for_popen(log_watcher, "stdin", expected_summary) + + def test_it_executes_git_and_returns_result(self): self.assertRegex(self.git.execute(["git", "version"]), r"^git version [\d\.]{2}.*$") def test_it_executes_git_not_from_cwd(self): @@ -195,17 +256,12 @@ def test_version(self): # END verify number types def test_cmd_override(self): - prev_cmd = self.git.GIT_PYTHON_GIT_EXECUTABLE - exc = GitCommandNotFound - try: - # set it to something that doesn't exist, assure it raises - type(self.git).GIT_PYTHON_GIT_EXECUTABLE = osp.join( - "some", "path", "which", "doesn't", "exist", "gitbinary" - ) - self.assertRaises(exc, self.git.version) - finally: - type(self.git).GIT_PYTHON_GIT_EXECUTABLE = prev_cmd - # END undo adjustment + with mock.patch.object( + type(self.git), + "GIT_PYTHON_GIT_EXECUTABLE", + osp.join("some", "path", "which", "doesn't", "exist", "gitbinary"), + ): + self.assertRaises(GitCommandNotFound, self.git.version) def test_refresh(self): # test a bad git path refresh @@ -250,7 +306,7 @@ def test_insert_after_kwarg_raises(self): def test_env_vars_passed_to_git(self): editor = "non_existent_editor" - with mock.patch.dict("os.environ", {"GIT_EDITOR": editor}): # @UndefinedVariable + with mock.patch.dict(os.environ, {"GIT_EDITOR": editor}): self.assertEqual(self.git.var("GIT_EDITOR"), editor) @with_rw_directory @@ -318,3 +374,11 @@ def counter_stderr(line): self.assertEqual(count[1], line_count) self.assertEqual(count[2], line_count) + + def test_execute_kwargs_set_agrees_with_method(self): + parameter_names = inspect.signature(cmd.Git.execute).parameters.keys() + self_param, command_param, *most_params, extra_kwargs_param = parameter_names + self.assertEqual(self_param, "self") + self.assertEqual(command_param, "command") + self.assertEqual(set(most_params), cmd.execute_kwargs) # Most important. + self.assertEqual(extra_kwargs_param, "subprocess_kwargs") diff --git a/test/test_index.py b/test/test_index.py index 3bebb382b..06db3aedd 100644 --- a/test/test_index.py +++ b/test/test_index.py @@ -3,14 +3,17 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from io import BytesIO import os +import os.path as osp +from pathlib import Path from stat import S_ISLNK, ST_MODE -import tempfile -from unittest import skipIf import shutil +import tempfile + +import pytest from git import ( IndexFile, @@ -28,21 +31,26 @@ from git.index.fun import hook_path from git.index.typ import BaseIndexEntry, IndexEntry from git.objects import Blob -from test.lib import TestBase, fixture_path, fixture, with_rw_repo -from test.lib import with_rw_directory -from git.util import Actor, rmtree -from git.util import HIDE_WINDOWS_KNOWN_ERRORS, hex_to_bin +from test.lib import TestBase, fixture, fixture_path, with_rw_directory, with_rw_repo +from git.util import Actor, hex_to_bin, rmtree from gitdb.base import IStream -import os.path as osp -from git.cmd import Git +HOOKS_SHEBANG = "#!/usr/bin/env sh\n" -from pathlib import Path -HOOKS_SHEBANG = "#!/usr/bin/env sh\n" +def _found_in(cmd, directory): + """Check if a command is resolved in a directory (without following symlinks).""" + path = shutil.which(cmd) + return path and Path(path).parent == Path(directory) + is_win_without_bash = is_win and not shutil.which("bash.exe") +is_win_with_wsl_bash = is_win and _found_in( + cmd="bash.exe", + directory=Path(os.getenv("WINDIR")) / "System32", +) + def _make_hook(git_dir, name, content, make_exec=True): """A helper to create a hook""" @@ -422,14 +430,6 @@ def _count_existing(self, repo, files): # END num existing helper - @skipIf( - HIDE_WINDOWS_KNOWN_ERRORS and Git.is_cygwin(), - """FIXME: File "C:\\projects\\gitpython\\git\\test\\test_index.py", line 642, in test_index_mutation - self.assertEqual(fd.read(), link_target) - AssertionError: '!\xff\xfe/\x00e\x00t\x00c\x00/\x00t\x00h\x00a\x00t\x00\x00\x00' - != '/etc/that' - """, - ) @with_rw_repo("0.1.6") def test_index_mutation(self, rw_repo): index = rw_repo.index @@ -910,7 +910,11 @@ def test_pre_commit_hook_fail(self, rw_repo): else: raise AssertionError("Should have caught a HookExecutionError") - @skipIf(HIDE_WINDOWS_KNOWN_ERRORS, "TODO: fix hooks execution on Windows: #703") + @pytest.mark.xfail( + is_win_without_bash or is_win_with_wsl_bash, + reason="Specifically seems to fail on WSL bash (in spite of #1399)", + raises=AssertionError, + ) @with_rw_repo("HEAD", bare=True) def test_commit_msg_hook_success(self, rw_repo): commit_message = "commit default head by Frèderic Çaufl€" @@ -946,11 +950,11 @@ def test_commit_msg_hook_fail(self, rw_repo): else: raise AssertionError("Should have caught a HookExecutionError") - @with_rw_repo('HEAD') + @with_rw_repo("HEAD") def test_index_add_pathlike(self, rw_repo): git_dir = Path(rw_repo.git_dir) file = git_dir / "file.txt" file.touch() - rw_repo.index.add(file) \ No newline at end of file + rw_repo.index.add(file) diff --git a/test/test_installation.py b/test/test_installation.py index d856ebc94..0cb0c71fa 100644 --- a/test/test_installation.py +++ b/test/test_installation.py @@ -1,9 +1,11 @@ # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import ast import os import subprocess +import sys + from git.compat import is_win from test.lib import TestBase from test.lib.helper import with_rw_directory @@ -12,7 +14,7 @@ class TestInstallation(TestBase): def setUp_venv(self, rw_dir): self.venv = rw_dir - subprocess.run(["virtualenv", self.venv], stdout=subprocess.PIPE) + subprocess.run([sys.executable, "-m", "venv", self.venv], stdout=subprocess.PIPE) bin_name = "Scripts" if is_win else "bin" self.python = os.path.join(self.venv, bin_name, "python") self.pip = os.path.join(self.venv, bin_name, "pip") diff --git a/test/test_quick_doc.py b/test/test_quick_doc.py index eaee4e581..342a7f293 100644 --- a/test/test_quick_doc.py +++ b/test/test_quick_doc.py @@ -1,6 +1,3 @@ -import pytest - - from test.lib import TestBase from test.lib.helper import with_rw_directory @@ -13,24 +10,24 @@ def tearDown(self): @with_rw_directory def test_init_repo_object(self, path_to_dir): - # [1-test_init_repo_object] # $ git init from git import Repo repo = Repo.init(path_to_dir) - # ![1-test_init_repo_object] + # ![1-test_init_repo_object] # [2-test_init_repo_object] repo = Repo(path_to_dir) # ![2-test_init_repo_object] + del repo # Avoids "assigned to but never used" warning. Doesn't go in the docs. + @with_rw_directory def test_cloned_repo_object(self, local_dir): - from git import Repo - import git + # code to clone from url # [1-test_cloned_repo_object] # $ git clone @@ -44,9 +41,9 @@ def test_cloned_repo_object(self, local_dir): # [2-test_cloned_repo_object] # We must make a change to a file so that we can add the update to git - update_file = 'dir1/file2.txt' # we'll use local_dir/dir1/file2.txt - with open(f"{local_dir}/{update_file}", 'a') as f: - f.write('\nUpdate version 2') + update_file = "dir1/file2.txt" # we'll use local_dir/dir1/file2.txt + with open(f"{local_dir}/{update_file}", "a") as f: + f.write("\nUpdate version 2") # ![2-test_cloned_repo_object] # [3-test_cloned_repo_object] @@ -73,7 +70,7 @@ def test_cloned_repo_object(self, local_dir): # [6-test_cloned_repo_object] commits_for_file_generator = repo.iter_commits(all=True, max_count=10, paths=update_file) - commits_for_file = [c for c in commits_for_file_generator] + commits_for_file = list(commits_for_file_generator) commits_for_file # Outputs: [, @@ -82,7 +79,7 @@ def test_cloned_repo_object(self, local_dir): # Untracked files - create new file # [7-test_cloned_repo_object] - f = open(f'{local_dir}/untracked.txt', 'w') # creates an empty file + f = open(f"{local_dir}/untracked.txt", "w") # creates an empty file f.close() # ![7-test_cloned_repo_object] @@ -95,8 +92,8 @@ def test_cloned_repo_object(self, local_dir): # [9-test_cloned_repo_object] # Let's modify one of our tracked files - with open(f'{local_dir}/Downloads/file3.txt', 'w') as f: - f.write('file3 version 2') # overwrite file 3 + with open(f"{local_dir}/Downloads/file3.txt", "w") as f: + f.write("file3 version 2") # overwrite file 3 # ![9-test_cloned_repo_object] # [10-test_cloned_repo_object] @@ -126,7 +123,7 @@ def test_cloned_repo_object(self, local_dir): # ![11.1-test_cloned_repo_object] # [11.2-test_cloned_repo_object] # lets add untracked.txt - repo.index.add(['untracked.txt']) + repo.index.add(["untracked.txt"]) diffs = repo.index.diff(repo.head.commit) for d in diffs: print(d.a_path) @@ -137,7 +134,7 @@ def test_cloned_repo_object(self, local_dir): # Compare commit to commit # [11.3-test_cloned_repo_object] - first_commit = [c for c in repo.iter_commits(all=True)][-1] + first_commit = list(repo.iter_commits(all=True))[-1] diffs = repo.head.commit.diff(first_commit) for d in diffs: print(d.a_path) @@ -146,9 +143,7 @@ def test_cloned_repo_object(self, local_dir): # dir1/file2.txt # ![11.3-test_cloned_repo_object] - - - '''Trees and Blobs''' + """Trees and Blobs""" # Latest commit tree # [12-test_cloned_repo_object] @@ -157,7 +152,7 @@ def test_cloned_repo_object(self, local_dir): # Previous commit tree # [13-test_cloned_repo_object] - prev_commits = [c for c in repo.iter_commits(all=True, max_count=10)] # last 10 commits from all branches + prev_commits = list(repo.iter_commits(all=True, max_count=10)) # last 10 commits from all branches tree = prev_commits[0].tree # ![13-test_cloned_repo_object] @@ -195,7 +190,7 @@ def print_files_from_git(root, level=0): # Printing text files # [17-test_cloned_repo_object] - print_file = 'dir1/file2.txt' + print_file = "dir1/file2.txt" tree[print_file] # the head commit tree # Output @@ -213,7 +208,7 @@ def print_files_from_git(root, level=0): # print previous tree # [18.1-test_cloned_repo_object] - commits_for_file = [c for c in repo.iter_commits(all=True, paths=print_file)] + commits_for_file = list(repo.iter_commits(all=True, paths=print_file)) tree = commits_for_file[-1].tree # gets the first commit tree blob = tree[print_file] @@ -221,4 +216,4 @@ def print_files_from_git(root, level=0): # Output # file 2 version 1 - # ![18.1-test_cloned_repo_object] \ No newline at end of file + # ![18.1-test_cloned_repo_object] diff --git a/test/test_refs.py b/test/test_refs.py index e7526c3b2..f9fc8b0ad 100644 --- a/test/test_refs.py +++ b/test/test_refs.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from itertools import chain from pathlib import Path @@ -386,7 +386,7 @@ def test_head_reset(self, rw_repo): head_tree = head.commit.tree self.assertRaises(ValueError, setattr, head, "commit", head_tree) assert head.commit == old_commit # and the ref did not change - # we allow heds to point to any object + # we allow heads to point to any object head.object = head_tree assert head.object == head_tree # cannot query tree as commit @@ -489,7 +489,7 @@ def test_head_reset(self, rw_repo): cur_head.reference.commit, ) # it works if the new ref points to the same reference - assert SymbolicReference.create(rw_repo, symref.path, symref.reference).path == symref.path # @NoEffect + assert SymbolicReference.create(rw_repo, symref.path, symref.reference).path == symref.path SymbolicReference.delete(rw_repo, symref) # would raise if the symref wouldn't have been deletedpbl symref = SymbolicReference.create(rw_repo, symref_path, cur_head.reference) @@ -631,3 +631,39 @@ def test_refs_outside_repo(self): ref_file.flush() ref_file_name = Path(ref_file.name).name self.assertRaises(BadName, self.rorepo.commit, f"../../{ref_file_name}") + + def test_validity_ref_names(self): + check_ref = SymbolicReference._check_ref_name_valid + # Based on the rules specified in https://git-scm.com/docs/git-check-ref-format/#_description + # Rule 1 + self.assertRaises(ValueError, check_ref, ".ref/begins/with/dot") + self.assertRaises(ValueError, check_ref, "ref/component/.begins/with/dot") + self.assertRaises(ValueError, check_ref, "ref/ends/with/a.lock") + self.assertRaises(ValueError, check_ref, "ref/component/ends.lock/with/period_lock") + # Rule 2 + check_ref("valid_one_level_refname") + # Rule 3 + self.assertRaises(ValueError, check_ref, "ref/contains/../double/period") + # Rule 4 + for c in " ~^:": + self.assertRaises(ValueError, check_ref, f"ref/contains/invalid{c}/character") + for code in range(0, 32): + self.assertRaises(ValueError, check_ref, f"ref/contains/invalid{chr(code)}/ASCII/control_character") + self.assertRaises(ValueError, check_ref, f"ref/contains/invalid{chr(127)}/ASCII/control_character") + # Rule 5 + for c in "*?[": + self.assertRaises(ValueError, check_ref, f"ref/contains/invalid{c}/character") + # Rule 6 + self.assertRaises(ValueError, check_ref, "/ref/begins/with/slash") + self.assertRaises(ValueError, check_ref, "ref/ends/with/slash/") + self.assertRaises(ValueError, check_ref, "ref/contains//double/slash/") + # Rule 7 + self.assertRaises(ValueError, check_ref, "ref/ends/with/dot.") + # Rule 8 + self.assertRaises(ValueError, check_ref, "ref/contains@{/at_brace") + # Rule 9 + self.assertRaises(ValueError, check_ref, "@") + # Rule 10 + self.assertRaises(ValueError, check_ref, "ref/contain\\s/backslash") + # Valid reference name should not raise + check_ref("valid/ref/name") diff --git a/test/test_remote.py b/test/test_remote.py index 9636ca486..7144b2791 100644 --- a/test/test_remote.py +++ b/test/test_remote.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import random import tempfile @@ -160,7 +160,7 @@ def _do_test_push_result(self, results, remote): # END error checking # END for each info - if any([info.flags & info.ERROR for info in results]): + if any(info.flags & info.ERROR for info in results): self.assertRaises(GitCommandError, results.raise_if_error) else: # No errors, so this should do nothing diff --git a/test/test_repo.py b/test/test_repo.py index 08ed13a00..364b895fb 100644 --- a/test/test_repo.py +++ b/test/test_repo.py @@ -3,7 +3,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import glob import io from io import BytesIO @@ -13,7 +13,7 @@ import pickle import sys import tempfile -from unittest import mock, skipIf, SkipTest, skip +from unittest import mock, skip import pytest @@ -41,10 +41,8 @@ UnsafeProtocolError, ) from git.repo.fun import touch -from test.lib import TestBase, with_rw_repo, fixture -from git.util import HIDE_WINDOWS_KNOWN_ERRORS, cygpath -from test.lib import with_rw_directory -from git.util import join_path_native, rmtree, rmfile, bin_to_hex +from git.util import bin_to_hex, cygpath, join_path_native, rmfile, rmtree +from test.lib import TestBase, fixture, with_rw_directory, with_rw_repo import os.path as osp @@ -251,7 +249,10 @@ def test_clone_from_with_path_contains_unicode(self): self.fail("Raised UnicodeEncodeError") @with_rw_directory - @skip("the referenced repository was removed, and one needs to setup a new password controlled repo under the orgs control") + @skip( + """The referenced repository was removed, and one needs to set up a new + password controlled repo under the org's control.""" + ) def test_leaking_password_in_clone_logs(self, rw_dir): password = "fakepassword1234" try: @@ -391,7 +392,9 @@ def test_clone_from_unsafe_options_allowed(self, rw_repo): for i, unsafe_option in enumerate(unsafe_options): destination = tmp_dir / str(i) assert not destination.exists() - Repo.clone_from(rw_repo.working_dir, destination, multi_options=[unsafe_option], allow_unsafe_options=True) + Repo.clone_from( + rw_repo.working_dir, destination, multi_options=[unsafe_option], allow_unsafe_options=True + ) assert destination.exists() @with_rw_repo("HEAD") @@ -754,21 +757,11 @@ def test_blame_complex_revision(self, git): @mock.patch.object(Git, "_call_process") def test_blame_accepts_rev_opts(self, git): - res = self.rorepo.blame("HEAD", "README.md", rev_opts=["-M", "-C", "-C"]) - expected_args = ['blame', 'HEAD', '-M', '-C', '-C', '--', 'README.md'] - boilerplate_kwargs = {"p" : True, "stdout_as_string": False} + expected_args = ["blame", "HEAD", "-M", "-C", "-C", "--", "README.md"] + boilerplate_kwargs = {"p": True, "stdout_as_string": False} + self.rorepo.blame("HEAD", "README.md", rev_opts=["-M", "-C", "-C"]) git.assert_called_once_with(*expected_args, **boilerplate_kwargs) - @skipIf( - HIDE_WINDOWS_KNOWN_ERRORS and Git.is_cygwin(), - """FIXME: File "C:\\projects\\gitpython\\git\\cmd.py", line 671, in execute - raise GitCommandError(command, status, stderr_value, stdout_value) - GitCommandError: Cmd('git') failed due to: exit code(128) - cmdline: git add 1__��ava verb��ten 1_test _myfile 1_test_other_file - 1_��ava-----verb��ten - stderr: 'fatal: pathspec '"1__çava verböten"' did not match any files' - """, - ) @with_rw_repo("HEAD", bare=False) def test_untracked_files(self, rwrepo): for run, repo_add in enumerate((rwrepo.index.add, rwrepo.git.add)): @@ -842,18 +835,13 @@ def test_comparison_and_hash(self): @with_rw_directory def test_tilde_and_env_vars_in_repo_path(self, rw_dir): - ph = os.environ.get("HOME") - try: + with mock.patch.dict(os.environ, {"HOME": rw_dir}): os.environ["HOME"] = rw_dir Repo.init(osp.join("~", "test.git"), bare=True) + with mock.patch.dict(os.environ, {"FOO": rw_dir}): os.environ["FOO"] = rw_dir Repo.init(osp.join("$FOO", "test.git"), bare=True) - finally: - if ph: - os.environ["HOME"] = ph - del os.environ["FOO"] - # end assure HOME gets reset to what it was def test_git_cmd(self): # test CatFileContentStream, just to be very sure we have no fencepost errors @@ -967,7 +955,7 @@ def _assert_rev_parse(self, name): # history with number ni = 11 history = [obj.parents[0]] - for pn in range(ni): + for _ in range(ni): history.append(history[-1].parents[0]) # END get given amount of commits @@ -1115,7 +1103,7 @@ def test_repo_odbtype(self): @pytest.mark.xfail( sys.platform == "cygwin", reason="Cygwin GitPython can't find submodule SHA", - raises=ValueError + raises=ValueError, ) def test_submodules(self): self.assertEqual(len(self.rorepo.submodules), 1) # non-recursive @@ -1245,7 +1233,7 @@ def test_merge_base(self): def test_is_ancestor(self): git = self.rorepo.git if git.version_info[:3] < (1, 8, 0): - raise SkipTest("git merge-base --is-ancestor feature unsupported") + raise RuntimeError("git merge-base --is-ancestor feature unsupported (test needs git 1.8.0 or later)") repo = self.rorepo c1 = "f6aa8d1" @@ -1293,7 +1281,7 @@ def test_git_work_tree_dotgit(self, rw_dir): based on it.""" git = Git(rw_dir) if git.version_info[:3] < (2, 5, 1): - raise SkipTest("worktree feature unsupported") + raise RuntimeError("worktree feature unsupported (test needs git 2.5.1 or later)") rw_master = self.rorepo.clone(join_path_native(rw_dir, "master_repo")) branch = rw_master.create_head("aaaaaaaa") @@ -1325,6 +1313,7 @@ def test_git_work_tree_env(self, rw_dir): # move .git directory to a subdirectory # set GIT_DIR and GIT_WORK_TREE appropriately # check that repo.working_tree_dir == rw_dir + self.rorepo.clone(join_path_native(rw_dir, "master_repo")) repo_dir = join_path_native(rw_dir, "master_repo") @@ -1334,16 +1323,12 @@ def test_git_work_tree_env(self, rw_dir): os.mkdir(new_subdir) os.rename(old_git_dir, new_git_dir) - oldenv = os.environ.copy() - os.environ["GIT_DIR"] = new_git_dir - os.environ["GIT_WORK_TREE"] = repo_dir + to_patch = {"GIT_DIR": new_git_dir, "GIT_WORK_TREE": repo_dir} - try: + with mock.patch.dict(os.environ, to_patch): r = Repo() self.assertEqual(r.working_tree_dir, repo_dir) self.assertEqual(r.working_dir, repo_dir) - finally: - os.environ = oldenv @with_rw_directory def test_rebasing(self, rw_dir): @@ -1415,14 +1400,16 @@ def test_ignored_items_reported(self): gi = tmp_dir / "repo" / ".gitignore" - with open(gi, 'w') as file: - file.write('ignored_file.txt\n') - file.write('ignored_dir/\n') + with open(gi, "w") as file: + file.write("ignored_file.txt\n") + file.write("ignored_dir/\n") - assert temp_repo.ignored(['included_file.txt', 'included_dir/file.txt']) == [] - assert temp_repo.ignored(['ignored_file.txt']) == ['ignored_file.txt'] - assert temp_repo.ignored(['included_file.txt', 'ignored_file.txt']) == ['ignored_file.txt'] - assert temp_repo.ignored(['included_file.txt', 'ignored_file.txt', 'included_dir/file.txt', 'ignored_dir/file.txt']) == ['ignored_file.txt', 'ignored_dir/file.txt'] + assert temp_repo.ignored(["included_file.txt", "included_dir/file.txt"]) == [] + assert temp_repo.ignored(["ignored_file.txt"]) == ["ignored_file.txt"] + assert temp_repo.ignored(["included_file.txt", "ignored_file.txt"]) == ["ignored_file.txt"] + assert temp_repo.ignored( + ["included_file.txt", "ignored_file.txt", "included_dir/file.txt", "ignored_dir/file.txt"] + ) == ["ignored_file.txt", "ignored_dir/file.txt"] def test_ignored_raises_error_w_symlink(self): with tempfile.TemporaryDirectory() as tdir: diff --git a/test/test_stats.py b/test/test_stats.py index 1f6896555..335ce483b 100644 --- a/test/test_stats.py +++ b/test/test_stats.py @@ -2,7 +2,7 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from test.lib import TestBase, fixture from git import Stats diff --git a/test/test_submodule.py b/test/test_submodule.py index 8c98a671e..31a555ce2 100644 --- a/test/test_submodule.py +++ b/test/test_submodule.py @@ -1,13 +1,13 @@ # -*- coding: utf-8 -*- # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ import contextlib import os import shutil import tempfile from pathlib import Path import sys -from unittest import mock, skipIf +from unittest import mock, skipUnless import pytest @@ -39,11 +39,14 @@ def _patch_git_config(name, value): # This is recomputed each time the context is entered, for compatibility with # existing GIT_CONFIG_* environment variables, even if changed in this process. - patcher = mock.patch.dict(os.environ, { - "GIT_CONFIG_COUNT": str(pair_index + 1), - f"GIT_CONFIG_KEY_{pair_index}": name, - f"GIT_CONFIG_VALUE_{pair_index}": value, - }) + patcher = mock.patch.dict( + os.environ, + { + "GIT_CONFIG_COUNT": str(pair_index + 1), + f"GIT_CONFIG_KEY_{pair_index}": name, + f"GIT_CONFIG_VALUE_{pair_index}": value, + }, + ) with patcher: yield @@ -108,7 +111,7 @@ def _do_base_tests(self, rwrepo): # force it to reread its information del smold._url - smold.url == sm.url # @NoEffect + smold.url == sm.url # noqa: B015 # FIXME: Should this be an assertion? # test config_reader/writer methods sm.config_reader() @@ -245,7 +248,7 @@ def _do_base_tests(self, rwrepo): assert csm.module_exists() # tracking branch once again - csm.module().head.ref.tracking_branch() is not None # @NoEffect + assert csm.module().head.ref.tracking_branch() is not None # this flushed in a sub-submodule assert len(list(rwrepo.iter_submodules())) == 2 @@ -454,7 +457,10 @@ def _do_base_tests(self, rwrepo): True, ) - # @skipIf(HIDE_WINDOWS_KNOWN_ERRORS, ## ACTUALLY skipped by `git.submodule.base#L869`. + # ACTUALLY skipped by git.util.rmtree (in local onerror function), called via + # git.objects.submodule.base.Submodule.remove at "method(mp)", line 1011. + # + # @skipIf(HIDE_WINDOWS_KNOWN_ERRORS, # "FIXME: fails with: PermissionError: [WinError 32] The process cannot access the file because" # "it is being used by another process: " # "'C:\\Users\\ankostis\\AppData\\Local\\Temp\\tmp95c3z83bnon_bare_test_base_rw\\git\\ext\\gitdb\\gitdb\\ext\\smmap'") # noqa E501 @@ -469,16 +475,16 @@ def test_base_bare(self, rwrepo): @pytest.mark.xfail( sys.platform == "cygwin", reason="Cygwin GitPython can't find submodule SHA", - raises=ValueError + raises=ValueError, ) - @skipIf( + @pytest.mark.xfail( HIDE_WINDOWS_KNOWN_ERRORS, - """ - File "C:\\projects\\gitpython\\git\\cmd.py", line 559, in execute - raise GitCommandNotFound(command, err) - git.exc.GitCommandNotFound: Cmd('git') not found due to: OSError('[WinError 6] The handle is invalid') - cmdline: git clone -n --shared -v C:\\projects\\gitpython\\.git Users\\appveyor\\AppData\\Local\\Temp\\1\\tmplyp6kr_rnon_bare_test_root_module""", - ) # noqa E501 + reason=( + '"The process cannot access the file because it is being used by another process"' + + " on first call to rm.update" + ), + raises=PermissionError, + ) @with_rw_repo(k_subm_current, bare=False) def test_root_module(self, rwrepo): # Can query everything without problems @@ -745,15 +751,13 @@ def test_list_only_valid_submodules(self, rwdir): repo = git.Repo(repo_path) assert len(repo.submodules) == 0 - @skipIf( + @pytest.mark.xfail( HIDE_WINDOWS_KNOWN_ERRORS, - """FIXME on cygwin: File "C:\\projects\\gitpython\\git\\cmd.py", line 671, in execute - raise GitCommandError(command, status, stderr_value, stdout_value) - GitCommandError: Cmd('git') failed due to: exit code(128) - cmdline: git add 1__Xava verbXXten 1_test _myfile 1_test_other_file 1_XXava-----verbXXten - stderr: 'fatal: pathspec '"1__çava verböten"' did not match any files' - FIXME on appveyor: see https://ci.appveyor.com/project/Byron/gitpython/build/1.0.185 - """, + reason=( + '"The process cannot access the file because it is being used by another process"' + + " on first call to sm.move" + ), + raises=PermissionError, ) @with_rw_directory @_patch_git_config("protocol.file.allow", "always") @@ -818,9 +822,11 @@ def test_git_submodules_and_add_sm_with_new_commit(self, rwdir): assert commit_sm.binsha == sm_too.binsha assert sm_too.binsha != sm.binsha - # @skipIf(HIDE_WINDOWS_KNOWN_ERRORS, ## ACTUALLY skipped by `git.submodule.base#L869`. - # "FIXME: helper.wrapper fails with: PermissionError: [WinError 5] Access is denied: " - # "'C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\test_work_tree_unsupportedryfa60di\\master_repo\\.git\\objects\\pack\\pack-bc9e0787aef9f69e1591ef38ea0a6f566ec66fe3.idx") # noqa E501 + @pytest.mark.xfail( + HIDE_WINDOWS_KNOWN_ERRORS, + reason='"The process cannot access the file because it is being used by another process" on call to sm.move', + raises=PermissionError, + ) @with_rw_directory def test_git_submodule_compatibility(self, rwdir): parent = git.Repo.init(osp.join(rwdir, "parent")) @@ -914,17 +920,17 @@ def test_ignore_non_submodule_file(self, rwdir): os.mkdir(smp) with open(osp.join(smp, "a"), "w", encoding="utf-8") as f: - f.write('test\n') + f.write("test\n") with open(osp.join(rwdir, ".gitmodules"), "w", encoding="utf-8") as f: - f.write("[submodule \"a\"]\n") + f.write('[submodule "a"]\n') f.write(" path = module\n") f.write(" url = https://github.com/chaconinc/DbConnector\n") parent.git.add(Git.polish_url(osp.join(smp, "a"))) parent.git.add(Git.polish_url(osp.join(rwdir, ".gitmodules"))) - parent.git.commit(message='test') + parent.git.commit(message="test") assert len(parent.submodules) == 0 @@ -1027,15 +1033,15 @@ def test_branch_renames(self, rw_dir): # This doesn't fail as our own submodule binsha didn't change, and the reset is only triggered if # to latest revision is True. parent_repo.submodule_update(to_latest_revision=False) - sm_mod.head.ref.name == sm_pfb.name, "should have been switched to past head" - sm_mod.commit() == sm_fb.commit, "Head wasn't reset" + assert sm_mod.head.ref.name == sm_pfb.name, "should have been switched to past head" + assert sm_mod.commit() == sm_fb.commit, "Head wasn't reset" self.assertRaises(RepositoryDirtyError, parent_repo.submodule_update, to_latest_revision=True) parent_repo.submodule_update(to_latest_revision=True, force_reset=True) assert sm_mod.commit() == sm_pfb.commit, "Now head should have been reset" assert sm_mod.head.ref.name == sm_pfb.name - @skipIf(not is_win, "Specifically for Windows.") + @skipUnless(is_win, "Specifically for Windows.") def test_to_relative_path_with_super_at_root_drive(self): class Repo(object): working_tree_dir = "D:\\" @@ -1046,9 +1052,9 @@ class Repo(object): msg = '_to_relative_path should be "submodule_path" but was "%s"' % relative_path assert relative_path == "submodule_path", msg - @skipIf( - True, - "for some unknown reason the assertion fails, even though it in fact is working in more common setup", + @pytest.mark.xfail( + reason="for some unknown reason the assertion fails, even though it in fact is working in more common setup", + raises=AssertionError, ) @with_rw_directory def test_depth(self, rwdir): @@ -1200,7 +1206,12 @@ def test_submodule_add_unsafe_options_allowed(self, rw_repo): # The options will be allowed, but the command will fail. with self.assertRaises(GitCommandError): Submodule.add( - rw_repo, "new", "new", str(tmp_dir), clone_multi_options=[unsafe_option], allow_unsafe_options=True + rw_repo, + "new", + "new", + str(tmp_dir), + clone_multi_options=[unsafe_option], + allow_unsafe_options=True, ) assert not tmp_file.exists() @@ -1211,7 +1222,12 @@ def test_submodule_add_unsafe_options_allowed(self, rw_repo): for unsafe_option in unsafe_options: with self.assertRaises(GitCommandError): Submodule.add( - rw_repo, "new", "new", str(tmp_dir), clone_multi_options=[unsafe_option], allow_unsafe_options=True + rw_repo, + "new", + "new", + str(tmp_dir), + clone_multi_options=[unsafe_option], + allow_unsafe_options=True, ) @with_rw_repo("HEAD") diff --git a/test/test_tree.py b/test/test_tree.py index 22c9c7d78..c5ac8d539 100644 --- a/test/test_tree.py +++ b/test/test_tree.py @@ -2,27 +2,17 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ from io import BytesIO -from unittest import skipIf from git.objects import Tree, Blob from test.lib import TestBase -from git.util import HIDE_WINDOWS_KNOWN_ERRORS import os.path as osp class TestTree(TestBase): - @skipIf( - HIDE_WINDOWS_KNOWN_ERRORS, - """ - File "C:\\projects\\gitpython\\git\\cmd.py", line 559, in execute - raise GitCommandNotFound(command, err) - git.exc.GitCommandNotFound: Cmd('git') not found due to: OSError('[WinError 6] The handle is invalid') - cmdline: git cat-file --batch-check""", - ) def test_serializable(self): # tree at the given commit contains a submodule as well roottree = self.rorepo.tree("6c1faef799095f3990e9970bc2cb10aa0221cf9c") @@ -51,14 +41,6 @@ def test_serializable(self): testtree._deserialize(stream) # END for each item in tree - @skipIf( - HIDE_WINDOWS_KNOWN_ERRORS, - """ - File "C:\\projects\\gitpython\\git\\cmd.py", line 559, in execute - raise GitCommandNotFound(command, err) - git.exc.GitCommandNotFound: Cmd('git') not found due to: OSError('[WinError 6] The handle is invalid') - cmdline: git cat-file --batch-check""", - ) def test_traverse(self): root = self.rorepo.tree("0.1.6") num_recursive = 0 diff --git a/test/test_util.py b/test/test_util.py index c17efce35..f75231c98 100644 --- a/test/test_util.py +++ b/test/test_util.py @@ -2,123 +2,265 @@ # Copyright (C) 2008, 2009 Michael Trier (mtrier@gmail.com) and contributors # # This module is part of GitPython and is released under -# the BSD License: http://www.opensource.org/licenses/bsd-license.php +# the BSD License: https://opensource.org/license/bsd-3-clause/ +import ast +import contextlib +from datetime import datetime import os +import pathlib import pickle +import stat +import subprocess import sys import tempfile import time -from unittest import mock, skipIf -from datetime import datetime +from unittest import SkipTest, mock, skipIf, skipUnless -import pytest import ddt +import pytest from git.cmd import dashify from git.compat import is_win from git.objects.util import ( altz_to_utctz_str, - utctz_to_altz, - verify_utctz, + from_timestamp, parse_date, tzoffset, - from_timestamp, -) -from test.lib import ( - TestBase, - with_rw_repo, + utctz_to_altz, + verify_utctz, ) from git.util import ( - LockFile, - BlockingLockFile, - get_user_id, Actor, + BlockingLockFile, IterableList, + LockFile, cygpath, decygpath, + get_user_id, remove_password_if_present, + rmtree, ) +from test.lib import TestBase, with_rw_repo -_norm_cygpath_pairs = ( - (r"foo\bar", "foo/bar"), - (r"foo/bar", "foo/bar"), - (r"C:\Users", "/cygdrive/c/Users"), - (r"C:\d/e", "/cygdrive/c/d/e"), - ("C:\\", "/cygdrive/c/"), - (r"\\server\C$\Users", "//server/C$/Users"), - (r"\\server\C$", "//server/C$"), - ("\\\\server\\c$\\", "//server/c$/"), - (r"\\server\BAR/", "//server/BAR/"), - (r"D:/Apps", "/cygdrive/d/Apps"), - (r"D:/Apps\fOO", "/cygdrive/d/Apps/fOO"), - (r"D:\Apps/123", "/cygdrive/d/Apps/123"), -) +class _Member: + """A member of an IterableList.""" -_unc_cygpath_pairs = ( - (r"\\?\a:\com", "/cygdrive/a/com"), - (r"\\?\a:/com", "/cygdrive/a/com"), - (r"\\?\UNC\server\D$\Apps", "//server/D$/Apps"), -) + __slots__ = ("name",) + def __init__(self, name): + self.name = name -class TestIterableMember(object): + def __repr__(self): + return f"{type(self).__name__}({self.name!r})" - """A member of an iterable list""" - __slots__ = "name" +@contextlib.contextmanager +def _tmpdir_to_force_permission_error(): + """Context manager to test permission errors in situations where they are not overcome.""" + if sys.platform == "cygwin": + raise SkipTest("Cygwin can't set the permissions that make the test meaningful.") + if sys.version_info < (3, 8): + raise SkipTest("In 3.7, TemporaryDirectory doesn't clean up after weird permissions.") - def __init__(self, name): - self.name = name + with tempfile.TemporaryDirectory() as parent: + td = pathlib.Path(parent, "testdir") + td.mkdir() + (td / "x").write_bytes(b"") + (td / "x").chmod(stat.S_IRUSR) # Set up PermissionError on Windows. + td.chmod(stat.S_IRUSR | stat.S_IXUSR) # Set up PermissionError on Unix. + yield td - def __repr__(self): - return "TestIterableMember(%r)" % self.name + +@contextlib.contextmanager +def _tmpdir_for_file_not_found(): + """Context manager to test errors deleting a directory that are not due to permissions.""" + with tempfile.TemporaryDirectory() as parent: + yield pathlib.Path(parent, "testdir") # It is deliberately never created. @ddt.ddt class TestUtils(TestBase): - def setup(self): - self.testdict = { - "string": "42", - "int": 42, - "array": [42], - } + def test_rmtree_deletes_nested_dir_with_files(self): + with tempfile.TemporaryDirectory() as parent: + td = pathlib.Path(parent, "testdir") + for d in td, td / "q", td / "s": + d.mkdir() + for f in ( + td / "p", + td / "q" / "w", + td / "q" / "x", + td / "r", + td / "s" / "y", + td / "s" / "z", + ): + f.write_bytes(b"") + + try: + rmtree(td) + except SkipTest as ex: + self.fail(f"rmtree unexpectedly attempts skip: {ex!r}") + + self.assertFalse(td.exists()) + + @skipIf( + sys.platform == "cygwin", + "Cygwin can't set the permissions that make the test meaningful.", + ) + def test_rmtree_deletes_dir_with_readonly_files(self): + # Automatically works on Unix, but requires special handling on Windows. + # Not to be confused with what _tmpdir_to_force_permission_error sets up (see below). + with tempfile.TemporaryDirectory() as parent: + td = pathlib.Path(parent, "testdir") + for d in td, td / "sub": + d.mkdir() + for f in td / "x", td / "sub" / "y": + f.write_bytes(b"") + f.chmod(0) + + try: + rmtree(td) + except SkipTest as ex: + self.fail(f"rmtree unexpectedly attempts skip: {ex!r}") + + self.assertFalse(td.exists()) + + def test_rmtree_can_wrap_exceptions(self): + """rmtree wraps PermissionError when HIDE_WINDOWS_KNOWN_ERRORS is true.""" + with _tmpdir_to_force_permission_error() as td: + # Access the module through sys.modules so it is unambiguous which module's + # attribute we patch: the original git.util, not git.index.util even though + # git.index.util "replaces" git.util and is what "import git.util" gives us. + with mock.patch.object(sys.modules["git.util"], "HIDE_WINDOWS_KNOWN_ERRORS", True): + # Disable common chmod functions so the callback can't fix the problem. + with mock.patch.object(os, "chmod"), mock.patch.object(pathlib.Path, "chmod"): + # Now we can see how an intractable PermissionError is treated. + with self.assertRaises(SkipTest): + rmtree(td) + + @ddt.data( + (False, PermissionError, _tmpdir_to_force_permission_error), + (False, FileNotFoundError, _tmpdir_for_file_not_found), + (True, FileNotFoundError, _tmpdir_for_file_not_found), + ) + def test_rmtree_does_not_wrap_unless_called_for(self, case): + """rmtree doesn't wrap non-PermissionError, nor if HIDE_WINDOWS_KNOWN_ERRORS is false.""" + hide_windows_known_errors, exception_type, tmpdir_context_factory = case + + with tmpdir_context_factory() as td: + # See comments in test_rmtree_can_wrap_exceptions regarding the patching done here. + with mock.patch.object( + sys.modules["git.util"], + "HIDE_WINDOWS_KNOWN_ERRORS", + hide_windows_known_errors, + ): + with mock.patch.object(os, "chmod"), mock.patch.object(pathlib.Path, "chmod"): + with self.assertRaises(exception_type): + try: + rmtree(td) + except SkipTest as ex: + self.fail(f"rmtree unexpectedly attempts skip: {ex!r}") + + @ddt.data("HIDE_WINDOWS_KNOWN_ERRORS", "HIDE_WINDOWS_FREEZE_ERRORS") + def test_env_vars_for_windows_tests(self, name): + def run_parse(value): + command = [ + sys.executable, + "-c", + f"from git.util import {name}; print(repr({name}))", + ] + output = subprocess.check_output( + command, + env=None if value is None else dict(os.environ, **{name: value}), + text=True, + ) + return ast.literal_eval(output) + + for env_var_value, expected_truth_value in ( + (None, os.name == "nt"), # True on Windows when the environment variable is unset. + ("", False), + (" ", False), + ("0", False), + ("1", os.name == "nt"), + ("false", False), + ("true", os.name == "nt"), + ("False", False), + ("True", os.name == "nt"), + ("no", False), + ("yes", os.name == "nt"), + ("NO", False), + ("YES", os.name == "nt"), + (" no ", False), + (" yes ", os.name == "nt"), + ): + with self.subTest(env_var_value=env_var_value): + self.assertIs(run_parse(env_var_value), expected_truth_value) + + _norm_cygpath_pairs = ( + (R"foo\bar", "foo/bar"), + (R"foo/bar", "foo/bar"), + (R"C:\Users", "/cygdrive/c/Users"), + (R"C:\d/e", "/cygdrive/c/d/e"), + ("C:\\", "/cygdrive/c/"), + (R"\\server\C$\Users", "//server/C$/Users"), + (R"\\server\C$", "//server/C$"), + ("\\\\server\\c$\\", "//server/c$/"), + (R"\\server\BAR/", "//server/BAR/"), + (R"D:/Apps", "/cygdrive/d/Apps"), + (R"D:/Apps\fOO", "/cygdrive/d/Apps/fOO"), + (R"D:\Apps/123", "/cygdrive/d/Apps/123"), + ) - @skipIf(not is_win, "Paths specifically for Windows.") + _unc_cygpath_pairs = ( + (R"\\?\a:\com", "/cygdrive/a/com"), + (R"\\?\a:/com", "/cygdrive/a/com"), + (R"\\?\UNC\server\D$\Apps", "//server/D$/Apps"), + ) + + # FIXME: Mark only the /proc-prefixing cases xfail, somehow (or fix them). + @pytest.mark.xfail( + reason="Many return paths prefixed /proc/cygdrive instead.", + raises=AssertionError, + ) + @skipUnless(sys.platform == "cygwin", "Paths specifically for Cygwin.") @ddt.idata(_norm_cygpath_pairs + _unc_cygpath_pairs) def test_cygpath_ok(self, case): wpath, cpath = case cwpath = cygpath(wpath) self.assertEqual(cwpath, cpath, wpath) - @skipIf(not is_win, "Paths specifically for Windows.") + @pytest.mark.xfail( + reason=R'2nd example r".\bar" -> "bar" fails, returns "./bar"', + raises=AssertionError, + ) + @skipUnless(sys.platform == "cygwin", "Paths specifically for Cygwin.") @ddt.data( - (r"./bar", "bar"), - (r".\bar", "bar"), - (r"../bar", "../bar"), - (r"..\bar", "../bar"), - (r"../bar/.\foo/../chu", "../bar/chu"), + (R"./bar", "bar"), + (R".\bar", "bar"), # FIXME: Mark only this one xfail, somehow (or fix it). + (R"../bar", "../bar"), + (R"..\bar", "../bar"), + (R"../bar/.\foo/../chu", "../bar/chu"), ) def test_cygpath_norm_ok(self, case): wpath, cpath = case cwpath = cygpath(wpath) self.assertEqual(cwpath, cpath or wpath, wpath) - @skipIf(not is_win, "Paths specifically for Windows.") + @skipUnless(sys.platform == "cygwin", "Paths specifically for Cygwin.") @ddt.data( - r"C:", - r"C:Relative", - r"D:Apps\123", - r"D:Apps/123", - r"\\?\a:rel", - r"\\share\a:rel", + R"C:", + R"C:Relative", + R"D:Apps\123", + R"D:Apps/123", + R"\\?\a:rel", + R"\\share\a:rel", ) def test_cygpath_invalids(self, wpath): cwpath = cygpath(wpath) self.assertEqual(cwpath, wpath.replace("\\", "/"), wpath) - @skipIf(not is_win, "Paths specifically for Windows.") + @skipUnless(sys.platform == "cygwin", "Paths specifically for Cygwin.") @ddt.idata(_norm_cygpath_pairs) def test_decygpath(self, case): wpath, cpath = case @@ -156,11 +298,6 @@ def test_lock_file(self): lock_file._obtain_lock_or_raise() lock_file._release_lock() - @pytest.mark.xfail( - sys.platform == "cygwin", - reason="Cygwin fails here for some reason, always", - raises=AssertionError - ) def test_blocking_lock_file(self): my_file = tempfile.mktemp() lock_file = BlockingLockFile(my_file) @@ -173,9 +310,8 @@ def test_blocking_lock_file(self): self.assertRaises(IOError, wait_lock._obtain_lock) elapsed = time.time() - start extra_time = 0.02 - if is_win: - # for Appveyor - extra_time *= 6 # NOTE: Indeterministic failures here... + if is_win or sys.platform == "cygwin": + extra_time *= 6 # NOTE: Indeterministic failures without this... self.assertLess(elapsed, wait_time + extra_time) def test_user_id(self): @@ -283,15 +419,18 @@ def test_actor_from_string(self): Actor("name last another", "some-very-long-email@example.com"), ) - @ddt.data(("name", ""), ("name", "prefix_")) + @ddt.data( + ("name", ""), + ("name", "prefix_"), + ) def test_iterable_list(self, case): name, prefix = case ilist = IterableList(name, prefix) name1 = "one" name2 = "two" - m1 = TestIterableMember(prefix + name1) - m2 = TestIterableMember(prefix + name2) + m1 = _Member(prefix + name1) + m2 = _Member(prefix + name2) ilist.extend((m1, m2)) diff --git a/tox.ini b/tox.ini new file mode 100644 index 000000000..f9ac25b78 --- /dev/null +++ b/tox.ini @@ -0,0 +1,32 @@ +[tox] +requires = tox>=4 +env_list = py{37,38,39,310,311,312}, lint, mypy, html + +[testenv] +description = Run unit tests +package = wheel +extras = test +pass_env = SSH_* +commands = pytest --color=yes {posargs} + +[testenv:lint] +description = Lint via pre-commit +base_python = py{39,310,311,312,38,37} +set_env = + SKIP = black-format +commands = pre-commit run --all-files --hook-stage manual + +[testenv:mypy] +description = Typecheck with mypy +base_python = py{39,310,311,312,38,37} +commands = mypy -p git +ignore_outcome = true + +[testenv:html] +description = Build HTML documentation +base_python = py{39,310,311,312,38,37} +deps = -r doc/requirements.txt +allowlist_externals = make +commands = + make BUILDDIR={env_tmp_dir}/doc/build -C doc clean + make BUILDDIR={env_tmp_dir}/doc/build -C doc html