diff --git a/.flake8 b/.flake8 index ed5d036bf..1cc049a69 100644 --- a/.flake8 +++ b/.flake8 @@ -1,38 +1,26 @@ [flake8] + show-source = True -count= True +count = True statistics = True -# E265 = comment blocks like @{ section, which it can't handle + # E266 = too many leading '#' for block comment # E731 = do not assign a lambda expression, use a def -# W293 = Blank line contains whitespace -# W504 = Line break after operator -# E704 = multiple statements in one line - used for @override # TC002 = move third party import to TYPE_CHECKING -# ANN = flake8-annotations # TC, TC2 = flake8-type-checking -# D = flake8-docstrings # select = C,E,F,W ANN, TC, TC2 # to enable code. Disabled if not listed, including builtin codes enable-extensions = TC, TC2 # only needed for extensions not enabled by default -ignore = E265,E266,E731,E704, - W293, W504, - ANN0 ANN1 ANN2, - TC002, - TC0, TC1, TC2 - # B, - A, - D, - RST, RST3 +ignore = E266, E731 -exclude = .tox,.venv,build,dist,doc,git/ext/ +exclude = .tox, .venv, build, dist, doc, git/ext/ rst-roles = # for flake8-RST-docstrings - attr,class,func,meth,mod,obj,ref,term,var # used by sphinx + attr, class, func, meth, mod, obj, ref, term, var # used by sphinx min-python-version = 3.7.0 # for `black` compatibility max-line-length = 120 -extend-ignore = E203,W503 +extend-ignore = E203, W503 diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 203f3c889..8c139c7be 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -3,4 +3,4 @@ updates: - package-ecosystem: "github-actions" directory: "/" schedule: - interval: "weekly" + interval: "weekly" diff --git a/.github/workflows/cygwin-test.yml b/.github/workflows/cygwin-test.yml index 962791ae7..cd913385f 100644 --- a/.github/workflows/cygwin-test.yml +++ b/.github/workflows/cygwin-test.yml @@ -5,20 +5,24 @@ on: [push, pull_request, workflow_dispatch] jobs: build: runs-on: windows-latest + strategy: fail-fast: false + env: CHERE_INVOKING: 1 - SHELLOPTS: igncr TMP: "/tmp" TEMP: "/tmp" + defaults: run: - shell: bash.exe --noprofile --norc -exo pipefail -o igncr "{0}" + shell: C:\cygwin\bin\bash.exe --noprofile --norc -exo pipefail -o igncr "{0}" steps: - name: Force LF line endings - run: git config --global core.autocrlf input + run: | + git config --global core.autocrlf false # Affects the non-Cygwin git. + shell: bash - uses: actions/checkout@v4 with: @@ -29,36 +33,42 @@ jobs: with: packages: python39 python39-pip python39-virtualenv git - - name: Show python and git versions + - name: Special configuration for Cygwin's git run: | - /usr/bin/python --version - /usr/bin/git version - - - name: Tell git to trust this repo - run: | - /usr/bin/git config --global --add safe.directory "$(pwd)" + git config --global --add safe.directory "$(pwd)" + git config --global core.autocrlf false - name: Prepare this repo for tests run: | - TRAVIS=yes ./init-tests-after-clone.sh + ./init-tests-after-clone.sh - - name: Further prepare git configuration for tests + - name: Set git user identity and command aliases for the tests run: | - /usr/bin/git config --global user.email "travis@ci.com" - /usr/bin/git config --global user.name "Travis Runner" + git config --global user.email "travis@ci.com" + git config --global user.name "Travis Runner" # If we rewrite the user's config by accident, we will mess it up # and cause subsequent tests to fail cat test/fixtures/.gitconfig >> ~/.gitconfig - name: Update PyPA packages run: | - /usr/bin/python -m pip install --upgrade pip setuptools wheel + # Get the latest pip, wheel, and prior to Python 3.12, setuptools. + python -m pip install -U pip $(pip freeze --all | grep -oF setuptools) wheel - name: Install project and test dependencies run: | - /usr/bin/python -m pip install ".[test]" + python -m pip install ".[test]" + + - name: Show version and platform information + run: | + uname -a + command -v git python + git version + python --version + python -c 'import sys; print(sys.platform)' + python -c 'import os; print(os.name)' + python -c 'import git; print(git.compat.is_win)' - name: Test with pytest run: | - set +x - /usr/bin/python -m pytest + python -m pytest --color=yes -p no:sugar --instafail -vv diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 5e79664a8..91dd919e0 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -7,8 +7,14 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v4 - with: - python-version: "3.x" - - uses: pre-commit/action@v3.0.0 + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v4 + with: + python-version: "3.x" + + - uses: pre-commit/action@v3.0.0 + with: + extra_args: --all-files --hook-stage manual + env: + SKIP: black-format diff --git a/.github/workflows/pythonpackage.yml b/.github/workflows/pythonpackage.yml index a5467ef94..2a82e0e03 100644 --- a/.github/workflows/pythonpackage.yml +++ b/.github/workflows/pythonpackage.yml @@ -10,16 +10,15 @@ permissions: jobs: build: - runs-on: ubuntu-latest + strategy: fail-fast: false matrix: python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"] include: - - experimental: false - - python-version: "3.12" - experimental: true + - experimental: false + defaults: run: shell: /bin/bash --noprofile --norc -exo pipefail {0} @@ -36,16 +35,11 @@ jobs: python-version: ${{ matrix.python-version }} allow-prereleases: ${{ matrix.experimental }} - - name: Show python and git versions - run: | - python --version - git version - - name: Prepare this repo for tests run: | - TRAVIS=yes ./init-tests-after-clone.sh + ./init-tests-after-clone.sh - - name: Prepare git configuration for tests + - name: Set git user identity and command aliases for the tests run: | git config --global user.email "travis@ci.com" git config --global user.name "Travis Runner" @@ -55,17 +49,23 @@ jobs: - name: Update PyPA packages run: | - python -m pip install --upgrade pip - if pip freeze --all | grep --quiet '^setuptools=='; then - # Python prior to 3.12 ships setuptools. Upgrade it if present. - python -m pip install --upgrade setuptools - fi - python -m pip install --upgrade wheel + # Get the latest pip, wheel, and prior to Python 3.12, setuptools. + python -m pip install -U pip $(pip freeze --all | grep -oF setuptools) wheel - name: Install project and test dependencies run: | pip install ".[test]" + - name: Show version and platform information + run: | + uname -a + command -v git python + git version + python --version + python -c 'import sys; print(sys.platform)' + python -c 'import os; print(os.name)' + python -c 'import git; print(git.compat.is_win)' + - name: Check types with mypy run: | mypy -p git @@ -75,7 +75,7 @@ jobs: - name: Test with pytest run: | - pytest + pytest --color=yes -p no:sugar --instafail -vv continue-on-error: false - name: Documentation diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 5a34b8af0..be97d5f9b 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,19 +1,39 @@ repos: - - repo: https://github.com/PyCQA/flake8 - rev: 6.1.0 - hooks: - - id: flake8 - additional_dependencies: - [ - flake8-bugbear==23.9.16, - flake8-comprehensions==3.14.0, - flake8-typing-imports==1.14.0, - ] - exclude: ^doc|^git/ext/ +- repo: https://github.com/psf/black-pre-commit-mirror + rev: 23.9.1 + hooks: + - id: black + alias: black-check + name: black (check) + args: [--check, --diff] + exclude: ^git/ext/ + stages: [manual] - - repo: https://github.com/pre-commit/pre-commit-hooks - rev: v4.4.0 - hooks: - - id: check-merge-conflict - - id: check-toml - - id: check-yaml + - id: black + alias: black-format + name: black (format) + exclude: ^git/ext/ + +- repo: https://github.com/PyCQA/flake8 + rev: 6.1.0 + hooks: + - id: flake8 + additional_dependencies: + - flake8-bugbear==23.9.16 + - flake8-comprehensions==3.14.0 + - flake8-typing-imports==1.14.0 + exclude: ^doc|^git/ext/ + +- repo: https://github.com/shellcheck-py/shellcheck-py + rev: v0.9.0.5 + hooks: + - id: shellcheck + args: [--color] + exclude: ^git/ext/ + +- repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.4.0 + hooks: + - id: check-toml + - id: check-yaml + - id: check-merge-conflict diff --git a/Makefile b/Makefile index 38090244c..839dc9f78 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,10 @@ -.PHONY: all clean release force_release +.PHONY: all lint clean release force_release all: - @grep -Ee '^[a-z].*:' Makefile | cut -d: -f1 | grep -vF all + @awk -F: '/^[[:alpha:]].*:/ && !/^all:/ {print $$1}' Makefile + +lint: + SKIP=black-format pre-commit run --all-files --hook-stage manual clean: rm -rf build/ dist/ .eggs/ .tox/ diff --git a/README.md b/README.md index dbec36024..69fb54c9f 100644 --- a/README.md +++ b/README.md @@ -76,17 +76,20 @@ To clone the [the GitHub repository](https://github.com/gitpython-developers/Git ```bash git clone https://github.com/gitpython-developers/GitPython cd GitPython -git fetch --tags ./init-tests-after-clone.sh ``` +On Windows, `./init-tests-after-clone.sh` can be run in a Git Bash shell. + If you are cloning [your own fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/about-forks), then replace the above `git clone` command with one that gives the URL of your fork. Or use this [`gh`](https://cli.github.com/) command (assuming you have `gh` and your fork is called `GitPython`): ```bash gh repo clone GitPython ``` -Having cloned the repo, create and activate your [virtual environment](https://docs.python.org/3/tutorial/venv.html). Then make an [editable install](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs): +Having cloned the repo, create and activate your [virtual environment](https://docs.python.org/3/tutorial/venv.html). + +Then make an [editable install](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs): ```bash pip install -e ".[test]" @@ -114,13 +117,12 @@ See [Issue #525](https://github.com/gitpython-developers/GitPython/issues/525). ### RUNNING TESTS -_Important_: Right after cloning this repository, please be sure to have -executed `git fetch --tags` followed by the `./init-tests-after-clone.sh` -script in the repository root. Otherwise you will encounter test failures. +_Important_: Right after cloning this repository, please be sure to have executed +the `./init-tests-after-clone.sh` script in the repository root. Otherwise +you will encounter test failures. On _Windows_, make sure you have `git-daemon` in your PATH. For MINGW-git, the `git-daemon.exe` -exists in `Git\mingw64\libexec\git-core\`; CYGWIN has no daemon, but should get along fine -with MINGW's. +exists in `Git\mingw64\libexec\git-core\`. #### Install test dependencies @@ -144,30 +146,41 @@ To test, run: pytest ``` -To lint, run: +To lint, and apply automatic code formatting, run: ```bash pre-commit run --all-files ``` +- Linting without modifying code can be done with: `make lint` +- Auto-formatting without other lint checks can be done with: `black .` + To typecheck, run: ```bash mypy -p git ``` -For automatic code formatting, run: +#### CI (and tox) -```bash -black . -``` +The same linting, and running tests on all the different supported Python versions, will be performed: + +- Upon submitting a pull request. +- On each push, *if* you have a fork with GitHub Actions enabled. +- Locally, if you run [`tox`](https://tox.wiki/) (this skips any Python versions you don't have installed). + +#### Configuration files + +Specific tools: -Configuration for flake8 is in the `./.flake8` file. +- Configurations for `mypy`, `pytest`, `coverage.py`, and `black` are in `./pyproject.toml`. +- Configuration for `flake8` is in the `./.flake8` file. -Configurations for `mypy`, `pytest`, `coverage.py`, and `black` are in `./pyproject.toml`. +Orchestration tools: -The same linting and testing will also be performed against different supported python versions -upon submitting a pull request (or on each push if you have a fork with a "main" branch and actions enabled). +- Configuration for `pre-commit` is in the `./.pre-commit-config.yaml` file. +- Configuration for `tox` is in `./tox.ini`. +- Configuration for GitHub Actions (CI) is in files inside `./.github/workflows/`. ### Contributions @@ -267,7 +280,7 @@ gpg --edit-key 4C08421980C9 ### LICENSE -[New BSD License](https://opensource.org/license/bsd-3-clause/). See the [LICENSE file](https://github.com/gitpython-developers/GitPython/blob/main/license). +[New BSD License](https://opensource.org/license/bsd-3-clause/). See the [LICENSE file][license]. [contributing]: https://github.com/gitpython-developers/GitPython/blob/main/CONTRIBUTING.md -[license]: https://github.com/gitpython-developers/GitPython/blob/main/license +[license]: https://github.com/gitpython-developers/GitPython/blob/main/LICENSE diff --git a/build-release.sh b/build-release.sh index 5840e4472..49c13b93a 100755 --- a/build-release.sh +++ b/build-release.sh @@ -6,21 +6,22 @@ set -eEu function release_with() { - $1 -m build --sdist --wheel + "$1" -m build --sdist --wheel } -if test -n "${VIRTUAL_ENV:-}"; then +function suggest_venv() { + local venv_cmd='python -m venv env && source env/bin/activate' + printf "HELP: To avoid this error, use a virtual-env with '%s' instead.\n" "$venv_cmd" +} + +if test -n "${VIRTUAL_ENV-}"; then deps=(build twine) # Install twine along with build, as we need it later. echo "Virtual environment detected. Adding packages: ${deps[*]}" pip install --quiet --upgrade "${deps[@]}" echo 'Starting the build.' release_with python else - function suggest_venv() { - venv_cmd='python -m venv env && source env/bin/activate' - printf "HELP: To avoid this error, use a virtual-env with '%s' instead.\n" "$venv_cmd" - } trap suggest_venv ERR # This keeps the original exit (error) code. echo 'Starting the build.' - release_with python3 # Outside a venv, use python3. + release_with python3 # Outside a venv, use python3. fi diff --git a/check-version.sh b/check-version.sh index c50bf498b..dac386e46 100755 --- a/check-version.sh +++ b/check-version.sh @@ -10,29 +10,39 @@ trap 'echo "$0: Check failed. Stopping." >&2' ERR readonly version_path='VERSION' readonly changes_path='doc/source/changes.rst' +function check_status() { + git status -s "$@" + test -z "$(git status -s "$@")" +} + +function get_latest_tag() { + local config_opts + printf -v config_opts ' -c versionsort.suffix=-%s' alpha beta pre rc RC + # shellcheck disable=SC2086 # Deliberately word-splitting the arguments. + git $config_opts tag -l '[0-9]*' --sort=-v:refname | head -n1 +} + echo 'Checking current directory.' test "$(cd -- "$(dirname -- "$0")" && pwd)" = "$(pwd)" # Ugly, but portable. echo "Checking that $version_path and $changes_path exist and have no uncommitted changes." test -f "$version_path" test -f "$changes_path" -git status -s -- "$version_path" "$changes_path" -test -z "$(git status -s -- "$version_path" "$changes_path")" +check_status -- "$version_path" "$changes_path" # This section can be commented out, if absolutely necessary. echo 'Checking that ALL changes are committed.' -git status -s --ignore-submodules -test -z "$(git status -s --ignore-submodules)" +check_status --ignore-submodules -version_version="$(cat "$version_path")" +version_version="$(<"$version_path")" changes_version="$(awk '/^[0-9]/ {print $0; exit}' "$changes_path")" -config_opts="$(printf ' -c versionsort.suffix=-%s' alpha beta pre rc RC)" -latest_tag="$(git $config_opts tag -l '[0-9]*' --sort=-v:refname | head -n1)" +latest_tag="$(get_latest_tag)" head_sha="$(git rev-parse HEAD)" latest_tag_sha="$(git rev-parse "${latest_tag}^{commit}")" # Display a table of all the current version, tag, and HEAD commit information. -echo $'\nThe VERSION must be the same in all locations, and so must the HEAD and tag SHA' +echo +echo 'The VERSION must be the same in all locations, and so must the HEAD and tag SHA' printf '%-14s = %s\n' 'VERSION file' "$version_version" \ 'changes.rst' "$changes_version" \ 'Latest tag' "$latest_tag" \ diff --git a/doc/Makefile b/doc/Makefile index ef2d60e5f..ddeadbd7e 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -2,6 +2,7 @@ # # You can set these variables from the command line. +BUILDDIR = build SPHINXOPTS = -W SPHINXBUILD = sphinx-build PAPER = @@ -9,7 +10,7 @@ PAPER = # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter -ALLSPHINXOPTS = -d build/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source +ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source .PHONY: help clean html web pickle htmlhelp latex changes linkcheck @@ -24,52 +25,52 @@ help: @echo " linkcheck to check all external links for integrity" clean: - -rm -rf build/* + -rm -rf $(BUILDDIR)/* html: - mkdir -p build/html build/doctrees - $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) build/html + mkdir -p $(BUILDDIR)/html $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo - @echo "Build finished. The HTML pages are in build/html." + @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." pickle: - mkdir -p build/pickle build/doctrees - $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) build/pickle + mkdir -p $(BUILDDIR)/pickle $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." web: pickle json: - mkdir -p build/json build/doctrees - $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) build/json + mkdir -p $(BUILDDIR)/json $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: - mkdir -p build/htmlhelp build/doctrees - $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) build/htmlhelp + mkdir -p $(BUILDDIR)/htmlhelp $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ - ".hhp project file in build/htmlhelp." + ".hhp project file in $(BUILDDIR)/htmlhelp." latex: - mkdir -p build/latex build/doctrees - $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) build/latex + mkdir -p $(BUILDDIR)/latex $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo - @echo "Build finished; the LaTeX files are in build/latex." + @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \ "run these through (pdf)latex." changes: - mkdir -p build/changes build/doctrees - $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) build/changes + mkdir -p $(BUILDDIR)/changes $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo - @echo "The overview file is in build/changes." + @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: - mkdir -p build/linkcheck build/doctrees - $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) build/linkcheck + mkdir -p $(BUILDDIR)/linkcheck $(BUILDDIR)/doctrees + $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ - "or in build/linkcheck/output.txt." + "or in $(BUILDDIR)/linkcheck/output.txt." diff --git a/git/cmd.py b/git/cmd.py index 9921dd6c9..7c448e3f2 100644 --- a/git/cmd.py +++ b/git/cmd.py @@ -66,10 +66,10 @@ "with_extended_output", "with_exceptions", "as_process", - "stdout_as_string", "output_stream", - "with_stdout", + "stdout_as_string", "kill_after_timeout", + "with_stdout", "universal_newlines", "shell", "env", @@ -105,7 +105,7 @@ def handle_process_output( ) -> None: """Registers for notifications to learn that process output is ready to read, and dispatches lines to the respective line handlers. - This function returns once the finalizer returns + This function returns once the finalizer returns. :return: result of finalizer :param process: subprocess.Popen instance @@ -294,9 +294,7 @@ def __setstate__(self, d: Dict[str, Any]) -> None: @classmethod def refresh(cls, path: Union[None, PathLike] = None) -> bool: - """This gets called by the refresh function (see the top level - __init__). - """ + """This gets called by the refresh function (see the top level __init__).""" # discern which path to refresh with if path is not None: new_git = os.path.expanduser(path) @@ -446,9 +444,9 @@ def polish_url(cls, url: str, is_cygwin: Union[None, bool] = None) -> PathLike: if is_cygwin: url = cygpath(url) else: - """Remove any backslahes from urls to be written in config files. + """Remove any backslashes from urls to be written in config files. - Windows might create config-files containing paths with backslashed, + Windows might create config files containing paths with backslashes, but git stops liking them as it will escape the backslashes. Hence we undo the escaping just to be sure. """ @@ -464,8 +462,8 @@ def check_unsafe_protocols(cls, url: str) -> None: Check for unsafe protocols. Apart from the usual protocols (http, git, ssh), - Git allows "remote helpers" that have the form `::
`, - one of these helpers (`ext::`) can be used to invoke any arbitrary command. + Git allows "remote helpers" that have the form ``::
``, + one of these helpers (``ext::``) can be used to invoke any arbitrary command. See: @@ -517,7 +515,7 @@ def __init__(self, proc: Union[None, subprocess.Popen], args: Any) -> None: self.status: Union[int, None] = None def _terminate(self) -> None: - """Terminate the underlying process""" + """Terminate the underlying process.""" if self.proc is None: return @@ -572,7 +570,7 @@ def wait(self, stderr: Union[None, str, bytes] = b"") -> int: """Wait for the process and return its status code. :param stderr: Previously read value of stderr, in case stderr is already closed. - :warn: may deadlock if output or error pipes are used and not handled separately. + :warn: May deadlock if output or error pipes are used and not handled separately. :raise GitCommandError: if the return status is not 0""" if stderr is None: stderr_b = b"" @@ -605,13 +603,12 @@ def read_all_from_possibly_closed_stream(stream: Union[IO[bytes], None]) -> byte # END auto interrupt class CatFileContentStream(object): - """Object representing a sized read-only stream returning the contents of an object. It behaves like a stream, but counts the data read and simulates an empty stream once our sized content region is empty. - If not all data is read to the end of the objects's lifetime, we read the - rest to assure the underlying stream continues to work""" + If not all data is read to the end of the object's lifetime, we read the + rest to assure the underlying stream continues to work.""" __slots__: Tuple[str, ...] = ("_stream", "_nbr", "_size") @@ -740,11 +737,11 @@ def __getattr__(self, name: str) -> Any: def set_persistent_git_options(self, **kwargs: Any) -> None: """Specify command line options to the git executable - for subsequent subcommand calls + for subsequent subcommand calls. :param kwargs: is a dict of keyword arguments. - these arguments are passed as in _call_process + These arguments are passed as in _call_process but will be passed to the git command rather than the subcommand. """ @@ -775,7 +772,7 @@ def version_info(self) -> Tuple[int, int, int, int]: """ :return: tuple(int, int, int, int) tuple with integers representing the major, minor and additional version numbers as parsed from git version. - This value is generated on demand and is cached""" + This value is generated on demand and is cached.""" return self._version_info @overload @@ -842,16 +839,16 @@ def execute( strip_newline_in_stdout: bool = True, **subprocess_kwargs: Any, ) -> Union[str, bytes, Tuple[int, Union[str, bytes], str], AutoInterrupt]: - """Handles executing the command on the shell and consumes and returns - the returned information (stdout) + """Handles executing the command and consumes and returns the returned + information (stdout). :param command: The command argument list to execute. - It should be a string, or a sequence of program arguments. The + It should be a sequence of program arguments, or a string. The program to execute is the first item in the args sequence or string. :param istream: - Standard input filehandle passed to subprocess.Popen. + Standard input filehandle passed to `subprocess.Popen`. :param with_extended_output: Whether to return a (status, stdout, stderr) tuple. @@ -862,8 +859,7 @@ def execute( :param as_process: Whether to return the created process instance directly from which streams can be read on demand. This will render with_extended_output and - with_exceptions ineffective - the caller will have - to deal with the details himself. + with_exceptions ineffective - the caller will have to deal with the details. It is important to note that the process will be placed into an AutoInterrupt wrapper that will interrupt the process once it goes out of scope. If you use the command in iterators, you should pass the whole process instance @@ -876,13 +872,34 @@ def execute( always be created with a pipe due to issues with subprocess. This merely is a workaround as data will be copied from the output pipe to the given output stream directly. - Judging from the implementation, you shouldn't use this flag ! + Judging from the implementation, you shouldn't use this parameter! :param stdout_as_string: - if False, the commands standard output will be bytes. Otherwise, it will be - decoded into a string using the default encoding (usually utf-8). + If False, the command's standard output will be bytes. Otherwise, it will be + decoded into a string using the default encoding (usually UTF-8). The latter can fail, if the output contains binary data. + :param kill_after_timeout: + Specifies a timeout in seconds for the git command, after which the process + should be killed. This will have no effect if as_process is set to True. It is + set to None by default and will let the process run until the timeout is + explicitly specified. This feature is not supported on Windows. It's also worth + noting that kill_after_timeout uses SIGKILL, which can have negative side + effects on a repository. For example, stale locks in case of ``git gc`` could + render the repository incapable of accepting changes until the lock is manually + removed. + + :param with_stdout: + If True, default True, we open stdout on the created process. + + :param universal_newlines: + if True, pipes will be opened as text, and lines are split at + all known line endings. + + :param shell: + Whether to invoke commands through a shell (see `Popen(..., shell=True)`). + It overrides :attr:`USE_SHELL` if it is not `None`. + :param env: A dictionary of environment variables to be passed to `subprocess.Popen`. @@ -891,38 +908,23 @@ def execute( one invocation of write() method. If the given number is not positive then the default value is used. + :param strip_newline_in_stdout: + Whether to strip the trailing ``\\n`` of the command stdout. + :param subprocess_kwargs: - Keyword arguments to be passed to subprocess.Popen. Please note that - some of the valid kwargs are already set by this method, the ones you + Keyword arguments to be passed to `subprocess.Popen`. Please note that + some of the valid kwargs are already set by this method; the ones you specify may not be the same ones. - :param with_stdout: If True, default True, we open stdout on the created process - :param universal_newlines: - if True, pipes will be opened as text, and lines are split at - all known line endings. - :param shell: - Whether to invoke commands through a shell (see `Popen(..., shell=True)`). - It overrides :attr:`USE_SHELL` if it is not `None`. - :param kill_after_timeout: - To specify a timeout in seconds for the git command, after which the process - should be killed. This will have no effect if as_process is set to True. It is - set to None by default and will let the process run until the timeout is - explicitly specified. This feature is not supported on Windows. It's also worth - noting that kill_after_timeout uses SIGKILL, which can have negative side - effects on a repository. For example, stale locks in case of git gc could - render the repository incapable of accepting changes until the lock is manually - removed. - :param strip_newline_in_stdout: - Whether to strip the trailing ``\\n`` of the command stdout. :return: * str(output) if extended_output = False (Default) * tuple(int(status), str(stdout), str(stderr)) if extended_output = True - if output_stream is True, the stdout value will be your output stream: + If output_stream is True, the stdout value will be your output stream: * output_stream if extended_output = False * tuple(int(status), output_stream, str(stderr)) if extended_output = True - Note git is executed with LC_MESSAGES="C" to ensure consistent + Note that git is executed with ``LC_MESSAGES="C"`` to ensure consistent output regardless of system language. :raise GitCommandError: @@ -971,16 +973,15 @@ def execute( # end handle stdout_sink = PIPE if with_stdout else getattr(subprocess, "DEVNULL", None) or open(os.devnull, "wb") - istream_ok = "None" - if istream: - istream_ok = "" + if shell is None: + shell = self.USE_SHELL log.debug( - "Popen(%s, cwd=%s, universal_newlines=%s, shell=%s, istream=%s)", + "Popen(%s, cwd=%s, stdin=%s, shell=%s, universal_newlines=%s)", redacted_command, cwd, - universal_newlines, + "" if istream else "None", shell, - istream_ok, + universal_newlines, ) try: with maybe_patch_caller_env: @@ -992,7 +993,7 @@ def execute( stdin=istream or DEVNULL, stderr=PIPE, stdout=stdout_sink, - shell=shell is not None and shell or self.USE_SHELL, + shell=shell, close_fds=is_posix, # unsupported on windows universal_newlines=universal_newlines, creationflags=PROC_CREATIONFLAGS, @@ -1008,8 +1009,8 @@ def execute( if as_process: return self.AutoInterrupt(proc, command) - def _kill_process(pid: int) -> None: - """Callback method to kill a process.""" + def kill_process(pid: int) -> None: + """Callback to kill a process.""" p = Popen( ["ps", "--ppid", str(pid)], stdout=PIPE, @@ -1042,7 +1043,7 @@ def _kill_process(pid: int) -> None: if kill_after_timeout is not None: kill_check = threading.Event() - watchdog = threading.Timer(kill_after_timeout, _kill_process, args=(proc.pid,)) + watchdog = threading.Timer(kill_after_timeout, kill_process, args=(proc.pid,)) # Wait for the process to return status = 0 @@ -1206,7 +1207,7 @@ def _unpack_args(cls, arg_list: Sequence[str]) -> List[str]: def __call__(self, **kwargs: Any) -> "Git": """Specify command line options to the git executable - for a subcommand call + for a subcommand call. :param kwargs: is a dict of keyword arguments. @@ -1244,7 +1245,7 @@ def _call_process( self, method: str, *args: Any, **kwargs: Any ) -> Union[str, bytes, Tuple[int, Union[str, bytes], str], "Git.AutoInterrupt"]: """Run the given git command with the specified arguments and return - the result as a String + the result as a string. :param method: is the command. Contained "_" characters will be converted to dashes, @@ -1253,7 +1254,7 @@ def _call_process( :param args: is the list of arguments. If None is included, it will be pruned. This allows your commands to call git more conveniently as None - is realized as non-existent + is realized as non-existent. :param kwargs: It contains key-values for the following: @@ -1383,7 +1384,7 @@ def get_object_header(self, ref: str) -> Tuple[str, str, int]: return self.__get_object_header(cmd, ref) def get_object_data(self, ref: str) -> Tuple[str, str, int, bytes]: - """As get_object_header, but returns object data as well + """As get_object_header, but returns object data as well. :return: (hexsha, type_string, size_as_int, data_string) :note: not threadsafe""" @@ -1393,10 +1394,10 @@ def get_object_data(self, ref: str) -> Tuple[str, str, int, bytes]: return (hexsha, typename, size, data) def stream_object_data(self, ref: str) -> Tuple[str, str, int, "Git.CatFileContentStream"]: - """As get_object_header, but returns the data as a stream + """As get_object_header, but returns the data as a stream. :return: (hexsha, type_string, size_as_int, stream) - :note: This method is not threadsafe, you need one independent Command instance per thread to be safe !""" + :note: This method is not threadsafe, you need one independent Command instance per thread to be safe!""" cmd = self._get_persistent_cmd("cat_file_all", "cat_file", batch=True) hexsha, typename, size = self.__get_object_header(cmd, ref) cmd_stdout = cmd.stdout if cmd.stdout is not None else io.BytesIO() diff --git a/init-tests-after-clone.sh b/init-tests-after-clone.sh index 95ced98b7..21d1f86d8 100755 --- a/init-tests-after-clone.sh +++ b/init-tests-after-clone.sh @@ -1,18 +1,73 @@ -#!/usr/bin/env bash +#!/bin/sh -set -e +set -eu -if [[ -z "$TRAVIS" ]]; then - read -rp "This operation will destroy locally modified files. Continue ? [N/y]: " answer - if [[ ! $answer =~ [yY] ]]; then - exit 2 - fi +fallback_repo_for_tags='https://github.com/gitpython-developers/GitPython.git' + +ci() { + # For now, check just these, as a false positive could lead to data loss. + test -n "${TRAVIS-}" || test -n "${GITHUB_ACTIONS-}" +} + +no_version_tags() { + test -z "$(git tag -l '[0-9]*' 'v[0-9]*')" +} + +warn() { + if test -n "${GITHUB_ACTIONS-}"; then + printf '::warning ::%s\n' "$*" >&2 # Annotate workflow. + else + printf '%s\n' "$@" >&2 + fi +} + +if ! ci; then + printf 'This operation will destroy locally modified files. Continue ? [N/y]: ' >&2 + read -r answer + case "$answer" in + [yY]) + ;; + *) + exit 2 ;; + esac fi +# Stop if we have run this. (You can delete __testing_point__ to let it rerun.) +# This also keeps track of where we are, so we can get back here. git tag __testing_point__ -git checkout master || git checkout -b master + +# The tests need a branch called master. +git checkout master -- || git checkout -b master + +# The tests need a reflog history on the master branch. git reset --hard HEAD~1 git reset --hard HEAD~1 git reset --hard HEAD~1 + +# Point the master branch where we started, so we test the correct code. git reset --hard __testing_point__ -git submodule update --init --recursive + +# The tests need submodules. (On CI, they would already have been checked out.) +if ! ci; then + git submodule update --init --recursive +fi + +# The tests need some version tags. Try to get them even in forks. This fetches +# other objects too. So, locally, we always do it, for a consistent experience. +if ! ci || no_version_tags; then + git fetch --all --tags +fi + +# If we still have no version tags, try to get them from the original repo. +if no_version_tags; then + warn 'No local or remote version tags found. Trying fallback remote:' \ + "$fallback_repo_for_tags" + + # git fetch supports * but not [], and --no-tags means no *other* tags, so... + printf 'refs/tags/%d*:refs/tags/%d*\n' 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 | + xargs git fetch --no-tags "$fallback_repo_for_tags" + + if no_version_tags; then + warn 'No version tags found anywhere. Some tests will fail.' + fi +fi diff --git a/pyproject.toml b/pyproject.toml index fa06458eb..f4fc33fec 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta" [tool.pytest.ini_options] python_files = 'test_*.py' testpaths = 'test' # space separated list of paths from root e.g test tests doc/testing -addopts = '--cov=git --cov-report=term --maxfail=10 --force-sugar --disable-warnings' +addopts = '--cov=git --cov-report=term --disable-warnings' filterwarnings = 'ignore::DeprecationWarning' # --cov coverage # --cov-report term # send report to terminal term-missing -> terminal with line numbers html xml diff --git a/test-requirements.txt b/test-requirements.txt index b00dd6f06..9414da09c 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -1,8 +1,10 @@ black coverage[toml] -ddt>=1.1.1, !=1.4.3 +ddt >= 1.1.1, != 1.4.3 +mock ; python_version < "3.8" mypy pre-commit pytest pytest-cov +pytest-instafail pytest-sugar diff --git a/test/lib/helper.py b/test/lib/helper.py index e8464b7d4..1de904610 100644 --- a/test/lib/helper.py +++ b/test/lib/helper.py @@ -177,12 +177,10 @@ def git_daemon_launched(base_path, ip, port): gd = None try: if is_win: - ## On MINGW-git, daemon exists in .\Git\mingw64\libexec\git-core\, - # but if invoked as 'git daemon', it detaches from parent `git` cmd, - # and then CANNOT DIE! - # So, invoke it as a single command. - ## Cygwin-git has no daemon. But it can use MINGW's. - # + # On MINGW-git, daemon exists in Git\mingw64\libexec\git-core\, + # but if invoked as 'git daemon', it detaches from parent `git` cmd, + # and then CANNOT DIE! + # So, invoke it as a single command. daemon_cmd = [ "git-daemon", "--enable=receive-pack", @@ -217,12 +215,11 @@ def git_daemon_launched(base_path, ip, port): ) if is_win: msg += textwrap.dedent( - r""" + R""" On Windows, the `git-daemon.exe` must be in PATH. - For MINGW, look into .\Git\mingw64\libexec\git-core\), but problems with paths might appear. - CYGWIN has no daemon, but if one exists, it gets along fine (but has also paths problems).""" + For MINGW, look into \Git\mingw64\libexec\git-core\, but problems with paths might appear.""" ) log.warning(msg, ex, ip, port, base_path, base_path, exc_info=1) @@ -305,7 +302,7 @@ def remote_repo_creator(self): cw.set("url", remote_repo_url) with git_daemon_launched( - Git.polish_url(base_daemon_path, is_cygwin=False), # No daemon in Cygwin. + Git.polish_url(base_daemon_path), "127.0.0.1", GIT_DAEMON_PORT, ): diff --git a/test/test_base.py b/test/test_base.py index b77c8117d..90e701c4b 100644 --- a/test/test_base.py +++ b/test/test_base.py @@ -7,7 +7,7 @@ import os import sys import tempfile -from unittest import SkipTest, skipIf +from unittest import skipIf from git import Repo from git.objects import Blob, Tree, Commit, TagObject @@ -126,7 +126,7 @@ def test_add_unicode(self, rw_repo): try: file_path.encode(sys.getfilesystemencoding()) except UnicodeEncodeError as e: - raise SkipTest("Environment doesn't support unicode filenames") from e + raise RuntimeError("Environment doesn't support unicode filenames") from e with open(file_path, "wb") as fp: fp.write(b"something") diff --git a/test/test_config.py b/test/test_config.py index 481e129c6..f805570d5 100644 --- a/test/test_config.py +++ b/test/test_config.py @@ -100,6 +100,7 @@ def test_includes_order(self): # values must be considered as soon as they get them assert r_config.get_value("diff", "tool") == "meld" try: + # FIXME: Split this assertion out somehow and mark it xfail (or fix it). assert r_config.get_value("sec", "var1") == "value1_main" except AssertionError as e: raise SkipTest("Known failure -- included values are not in effect right away") from e diff --git a/test/test_fun.py b/test/test_fun.py index d76e189ed..f39955aa0 100644 --- a/test/test_fun.py +++ b/test/test_fun.py @@ -2,7 +2,6 @@ from stat import S_IFDIR, S_IFREG, S_IFLNK, S_IXUSR from os import stat import os.path as osp -from unittest import SkipTest from git import Git from git.index import IndexFile @@ -279,7 +278,7 @@ def test_linked_worktree_traversal(self, rw_dir): """Check that we can identify a linked worktree based on a .git file""" git = Git(rw_dir) if git.version_info[:3] < (2, 5, 1): - raise SkipTest("worktree feature unsupported") + raise RuntimeError("worktree feature unsupported (test needs git 2.5.1 or later)") rw_master = self.rorepo.clone(join_path_native(rw_dir, "master_repo")) branch = rw_master.create_head("aaaaaaaa") diff --git a/test/test_git.py b/test/test_git.py index 481309538..cf82d9ac7 100644 --- a/test/test_git.py +++ b/test/test_git.py @@ -4,23 +4,31 @@ # # This module is part of GitPython and is released under # the BSD License: https://opensource.org/license/bsd-3-clause/ +import inspect +import logging import os +import os.path as osp +import re import shutil import subprocess import sys from tempfile import TemporaryDirectory, TemporaryFile -from unittest import mock, skipUnless +from unittest import skipUnless -from git import Git, refresh, GitCommandError, GitCommandNotFound, Repo, cmd -from test.lib import TestBase, fixture_path -from test.lib import with_rw_directory -from git.util import cwd, finalize_process +if sys.version_info >= (3, 8): + from unittest import mock +else: + import mock # To be able to examine call_args.kwargs on a mock. -import os.path as osp +import ddt +from git import Git, refresh, GitCommandError, GitCommandNotFound, Repo, cmd from git.compat import is_win +from git.util import cwd, finalize_process +from test.lib import TestBase, fixture_path, with_rw_directory +@ddt.ddt class TestGit(TestBase): @classmethod def setUpClass(cls): @@ -32,6 +40,13 @@ def tearDown(self): gc.collect() + def _assert_logged_for_popen(self, log_watcher, name, value): + re_name = re.escape(name) + re_value = re.escape(str(value)) + re_line = re.compile(rf"DEBUG:git.cmd:Popen\(.*\b{re_name}={re_value}[,)]") + match_attempts = [re_line.match(message) for message in log_watcher.output] + self.assertTrue(any(match_attempts), repr(log_watcher.output)) + @mock.patch.object(Git, "execute") def test_call_process_calls_execute(self, git): git.return_value = "" @@ -73,7 +88,53 @@ def test_it_transforms_kwargs_into_git_command_arguments(self): res = self.git.transform_kwargs(**{"s": True, "t": True}) self.assertEqual({"-s", "-t"}, set(res)) - def test_it_executes_git_to_shell_and_returns_result(self): + _shell_cases = ( + # value_in_call, value_from_class, expected_popen_arg + (None, False, False), + (None, True, True), + (False, True, False), + (False, False, False), + (True, False, True), + (True, True, True), + ) + + def _do_shell_combo(self, value_in_call, value_from_class): + with mock.patch.object(Git, "USE_SHELL", value_from_class): + # git.cmd gets Popen via a "from" import, so patch it there. + with mock.patch.object(cmd, "Popen", wraps=cmd.Popen) as mock_popen: + # Use a command with no arguments (besides the program name), so it runs + # with or without a shell, on all OSes, with the same effect. + self.git.execute(["git"], with_exceptions=False, shell=value_in_call) + + return mock_popen + + @ddt.idata(_shell_cases) + def test_it_uses_shell_or_not_as_specified(self, case): + """A bool passed as ``shell=`` takes precedence over `Git.USE_SHELL`.""" + value_in_call, value_from_class, expected_popen_arg = case + mock_popen = self._do_shell_combo(value_in_call, value_from_class) + mock_popen.assert_called_once() + self.assertIs(mock_popen.call_args.kwargs["shell"], expected_popen_arg) + + @ddt.idata(full_case[:2] for full_case in _shell_cases) + def test_it_logs_if_it_uses_a_shell(self, case): + """``shell=`` in the log message agrees with what is passed to `Popen`.""" + value_in_call, value_from_class = case + with self.assertLogs(cmd.log, level=logging.DEBUG) as log_watcher: + mock_popen = self._do_shell_combo(value_in_call, value_from_class) + self._assert_logged_for_popen(log_watcher, "shell", mock_popen.call_args.kwargs["shell"]) + + @ddt.data( + ("None", None), + ("", subprocess.PIPE), + ) + def test_it_logs_istream_summary_for_stdin(self, case): + expected_summary, istream_argument = case + with self.assertLogs(cmd.log, level=logging.DEBUG) as log_watcher: + self.git.execute(["git", "version"], istream=istream_argument) + self._assert_logged_for_popen(log_watcher, "stdin", expected_summary) + + def test_it_executes_git_and_returns_result(self): self.assertRegex(self.git.execute(["git", "version"]), r"^git version [\d\.]{2}.*$") def test_it_executes_git_not_from_cwd(self): @@ -313,3 +374,11 @@ def counter_stderr(line): self.assertEqual(count[1], line_count) self.assertEqual(count[2], line_count) + + def test_execute_kwargs_set_agrees_with_method(self): + parameter_names = inspect.signature(cmd.Git.execute).parameters.keys() + self_param, command_param, *most_params, extra_kwargs_param = parameter_names + self.assertEqual(self_param, "self") + self.assertEqual(command_param, "command") + self.assertEqual(set(most_params), cmd.execute_kwargs) # Most important. + self.assertEqual(extra_kwargs_param, "subprocess_kwargs") diff --git a/test/test_index.py b/test/test_index.py index fba9c78ec..06db3aedd 100644 --- a/test/test_index.py +++ b/test/test_index.py @@ -7,10 +7,13 @@ from io import BytesIO import os +import os.path as osp +from pathlib import Path from stat import S_ISLNK, ST_MODE -import tempfile -from unittest import skipIf import shutil +import tempfile + +import pytest from git import ( IndexFile, @@ -28,21 +31,26 @@ from git.index.fun import hook_path from git.index.typ import BaseIndexEntry, IndexEntry from git.objects import Blob -from test.lib import TestBase, fixture_path, fixture, with_rw_repo -from test.lib import with_rw_directory -from git.util import Actor, rmtree -from git.util import HIDE_WINDOWS_KNOWN_ERRORS, hex_to_bin +from test.lib import TestBase, fixture, fixture_path, with_rw_directory, with_rw_repo +from git.util import Actor, hex_to_bin, rmtree from gitdb.base import IStream -import os.path as osp -from git.cmd import Git +HOOKS_SHEBANG = "#!/usr/bin/env sh\n" -from pathlib import Path -HOOKS_SHEBANG = "#!/usr/bin/env sh\n" +def _found_in(cmd, directory): + """Check if a command is resolved in a directory (without following symlinks).""" + path = shutil.which(cmd) + return path and Path(path).parent == Path(directory) + is_win_without_bash = is_win and not shutil.which("bash.exe") +is_win_with_wsl_bash = is_win and _found_in( + cmd="bash.exe", + directory=Path(os.getenv("WINDIR")) / "System32", +) + def _make_hook(git_dir, name, content, make_exec=True): """A helper to create a hook""" @@ -422,14 +430,6 @@ def _count_existing(self, repo, files): # END num existing helper - @skipIf( - HIDE_WINDOWS_KNOWN_ERRORS and Git.is_cygwin(), - """FIXME: File "C:\\projects\\gitpython\\git\\test\\test_index.py", line 642, in test_index_mutation - self.assertEqual(fd.read(), link_target) - AssertionError: '!\xff\xfe/\x00e\x00t\x00c\x00/\x00t\x00h\x00a\x00t\x00\x00\x00' - != '/etc/that' - """, - ) @with_rw_repo("0.1.6") def test_index_mutation(self, rw_repo): index = rw_repo.index @@ -910,7 +910,11 @@ def test_pre_commit_hook_fail(self, rw_repo): else: raise AssertionError("Should have caught a HookExecutionError") - @skipIf(HIDE_WINDOWS_KNOWN_ERRORS, "TODO: fix hooks execution on Windows: #703") + @pytest.mark.xfail( + is_win_without_bash or is_win_with_wsl_bash, + reason="Specifically seems to fail on WSL bash (in spite of #1399)", + raises=AssertionError, + ) @with_rw_repo("HEAD", bare=True) def test_commit_msg_hook_success(self, rw_repo): commit_message = "commit default head by Frèderic Çaufl€" diff --git a/test/test_repo.py b/test/test_repo.py index 15899ec50..364b895fb 100644 --- a/test/test_repo.py +++ b/test/test_repo.py @@ -13,7 +13,7 @@ import pickle import sys import tempfile -from unittest import mock, skipIf, SkipTest, skip +from unittest import mock, skip import pytest @@ -41,10 +41,8 @@ UnsafeProtocolError, ) from git.repo.fun import touch -from test.lib import TestBase, with_rw_repo, fixture -from git.util import HIDE_WINDOWS_KNOWN_ERRORS, cygpath -from test.lib import with_rw_directory -from git.util import join_path_native, rmtree, rmfile, bin_to_hex +from git.util import bin_to_hex, cygpath, join_path_native, rmfile, rmtree +from test.lib import TestBase, fixture, with_rw_directory, with_rw_repo import os.path as osp @@ -764,16 +762,6 @@ def test_blame_accepts_rev_opts(self, git): self.rorepo.blame("HEAD", "README.md", rev_opts=["-M", "-C", "-C"]) git.assert_called_once_with(*expected_args, **boilerplate_kwargs) - @skipIf( - HIDE_WINDOWS_KNOWN_ERRORS and Git.is_cygwin(), - """FIXME: File "C:\\projects\\gitpython\\git\\cmd.py", line 671, in execute - raise GitCommandError(command, status, stderr_value, stdout_value) - GitCommandError: Cmd('git') failed due to: exit code(128) - cmdline: git add 1__��ava verb��ten 1_test _myfile 1_test_other_file - 1_��ava-----verb��ten - stderr: 'fatal: pathspec '"1__çava verböten"' did not match any files' - """, - ) @with_rw_repo("HEAD", bare=False) def test_untracked_files(self, rwrepo): for run, repo_add in enumerate((rwrepo.index.add, rwrepo.git.add)): @@ -1245,7 +1233,7 @@ def test_merge_base(self): def test_is_ancestor(self): git = self.rorepo.git if git.version_info[:3] < (1, 8, 0): - raise SkipTest("git merge-base --is-ancestor feature unsupported") + raise RuntimeError("git merge-base --is-ancestor feature unsupported (test needs git 1.8.0 or later)") repo = self.rorepo c1 = "f6aa8d1" @@ -1293,7 +1281,7 @@ def test_git_work_tree_dotgit(self, rw_dir): based on it.""" git = Git(rw_dir) if git.version_info[:3] < (2, 5, 1): - raise SkipTest("worktree feature unsupported") + raise RuntimeError("worktree feature unsupported (test needs git 2.5.1 or later)") rw_master = self.rorepo.clone(join_path_native(rw_dir, "master_repo")) branch = rw_master.create_head("aaaaaaaa") diff --git a/test/test_submodule.py b/test/test_submodule.py index 4a9c9c582..79ff2c5f2 100644 --- a/test/test_submodule.py +++ b/test/test_submodule.py @@ -7,7 +7,7 @@ import tempfile from pathlib import Path import sys -from unittest import mock, skipIf +from unittest import mock, skipUnless import pytest @@ -474,14 +474,13 @@ def test_base_bare(self, rwrepo): reason="Cygwin GitPython can't find submodule SHA", raises=ValueError, ) - @skipIf( + @pytest.mark.xfail( HIDE_WINDOWS_KNOWN_ERRORS, - """ - File "C:\\projects\\gitpython\\git\\cmd.py", line 559, in execute - raise GitCommandNotFound(command, err) - git.exc.GitCommandNotFound: Cmd('git') not found due to: OSError('[WinError 6] The handle is invalid') - cmdline: git clone -n --shared -v C:\\projects\\gitpython\\.git Users\\appveyor\\AppData\\Local\\Temp\\1\\tmplyp6kr_rnon_bare_test_root_module - """, # noqa E501 + reason=( + '"The process cannot access the file because it is being used by another process"' + + " on first call to rm.update" + ), + raises=PermissionError, ) @with_rw_repo(k_subm_current, bare=False) def test_root_module(self, rwrepo): @@ -749,15 +748,13 @@ def test_list_only_valid_submodules(self, rwdir): repo = git.Repo(repo_path) assert len(repo.submodules) == 0 - @skipIf( + @pytest.mark.xfail( HIDE_WINDOWS_KNOWN_ERRORS, - """FIXME on cygwin: File "C:\\projects\\gitpython\\git\\cmd.py", line 671, in execute - raise GitCommandError(command, status, stderr_value, stdout_value) - GitCommandError: Cmd('git') failed due to: exit code(128) - cmdline: git add 1__Xava verbXXten 1_test _myfile 1_test_other_file 1_XXava-----verbXXten - stderr: 'fatal: pathspec '"1__çava verböten"' did not match any files' - FIXME on appveyor: see https://ci.appveyor.com/project/Byron/gitpython/build/1.0.185 - """, + reason=( + '"The process cannot access the file because it is being used by another process"' + + " on first call to sm.move" + ), + raises=PermissionError, ) @with_rw_directory @_patch_git_config("protocol.file.allow", "always") @@ -1031,15 +1028,15 @@ def test_branch_renames(self, rw_dir): # This doesn't fail as our own submodule binsha didn't change, and the reset is only triggered if # to latest revision is True. parent_repo.submodule_update(to_latest_revision=False) - sm_mod.head.ref.name == sm_pfb.name, "should have been switched to past head" - sm_mod.commit() == sm_fb.commit, "Head wasn't reset" + assert sm_mod.head.ref.name == sm_pfb.name, "should have been switched to past head" + assert sm_mod.commit() == sm_fb.commit, "Head wasn't reset" self.assertRaises(RepositoryDirtyError, parent_repo.submodule_update, to_latest_revision=True) parent_repo.submodule_update(to_latest_revision=True, force_reset=True) assert sm_mod.commit() == sm_pfb.commit, "Now head should have been reset" assert sm_mod.head.ref.name == sm_pfb.name - @skipIf(not is_win, "Specifically for Windows.") + @skipUnless(is_win, "Specifically for Windows.") def test_to_relative_path_with_super_at_root_drive(self): class Repo(object): working_tree_dir = "D:\\" @@ -1050,9 +1047,9 @@ class Repo(object): msg = '_to_relative_path should be "submodule_path" but was "%s"' % relative_path assert relative_path == "submodule_path", msg - @skipIf( - True, - "for some unknown reason the assertion fails, even though it in fact is working in more common setup", + @pytest.mark.xfail( + reason="for some unknown reason the assertion fails, even though it in fact is working in more common setup", + raises=AssertionError, ) @with_rw_directory def test_depth(self, rwdir): diff --git a/test/test_tree.py b/test/test_tree.py index e59705645..c5ac8d539 100644 --- a/test/test_tree.py +++ b/test/test_tree.py @@ -5,24 +5,14 @@ # the BSD License: https://opensource.org/license/bsd-3-clause/ from io import BytesIO -from unittest import skipIf from git.objects import Tree, Blob from test.lib import TestBase -from git.util import HIDE_WINDOWS_KNOWN_ERRORS import os.path as osp class TestTree(TestBase): - @skipIf( - HIDE_WINDOWS_KNOWN_ERRORS, - """ - File "C:\\projects\\gitpython\\git\\cmd.py", line 559, in execute - raise GitCommandNotFound(command, err) - git.exc.GitCommandNotFound: Cmd('git') not found due to: OSError('[WinError 6] The handle is invalid') - cmdline: git cat-file --batch-check""", - ) def test_serializable(self): # tree at the given commit contains a submodule as well roottree = self.rorepo.tree("6c1faef799095f3990e9970bc2cb10aa0221cf9c") @@ -51,14 +41,6 @@ def test_serializable(self): testtree._deserialize(stream) # END for each item in tree - @skipIf( - HIDE_WINDOWS_KNOWN_ERRORS, - """ - File "C:\\projects\\gitpython\\git\\cmd.py", line 559, in execute - raise GitCommandNotFound(command, err) - git.exc.GitCommandNotFound: Cmd('git') not found due to: OSError('[WinError 6] The handle is invalid') - cmdline: git cat-file --batch-check""", - ) def test_traverse(self): root = self.rorepo.tree("0.1.6") num_recursive = 0 diff --git a/test/test_util.py b/test/test_util.py index 42edc57cf..2b1e518ed 100644 --- a/test/test_util.py +++ b/test/test_util.py @@ -9,11 +9,11 @@ import sys import tempfile import time -from unittest import mock, skipIf +from unittest import mock, skipUnless from datetime import datetime -import pytest import ddt +import pytest from git.cmd import dashify from git.compat import is_win @@ -85,17 +85,26 @@ def setup(self): "array": [42], } - @skipIf(not is_win, "Paths specifically for Windows.") + # FIXME: Mark only the /proc-prefixing cases xfail, somehow (or fix them). + @pytest.mark.xfail( + reason="Many return paths prefixed /proc/cygdrive instead.", + raises=AssertionError, + ) + @skipUnless(sys.platform == "cygwin", "Paths specifically for Cygwin.") @ddt.idata(_norm_cygpath_pairs + _unc_cygpath_pairs) def test_cygpath_ok(self, case): wpath, cpath = case cwpath = cygpath(wpath) self.assertEqual(cwpath, cpath, wpath) - @skipIf(not is_win, "Paths specifically for Windows.") + @pytest.mark.xfail( + reason=r'2nd example r".\bar" -> "bar" fails, returns "./bar"', + raises=AssertionError, + ) + @skipUnless(sys.platform == "cygwin", "Paths specifically for Cygwin.") @ddt.data( (r"./bar", "bar"), - (r".\bar", "bar"), + (r".\bar", "bar"), # FIXME: Mark only this one xfail, somehow (or fix it). (r"../bar", "../bar"), (r"..\bar", "../bar"), (r"../bar/.\foo/../chu", "../bar/chu"), @@ -105,7 +114,7 @@ def test_cygpath_norm_ok(self, case): cwpath = cygpath(wpath) self.assertEqual(cwpath, cpath or wpath, wpath) - @skipIf(not is_win, "Paths specifically for Windows.") + @skipUnless(sys.platform == "cygwin", "Paths specifically for Cygwin.") @ddt.data( r"C:", r"C:Relative", @@ -118,7 +127,7 @@ def test_cygpath_invalids(self, wpath): cwpath = cygpath(wpath) self.assertEqual(cwpath, wpath.replace("\\", "/"), wpath) - @skipIf(not is_win, "Paths specifically for Windows.") + @skipUnless(sys.platform == "cygwin", "Paths specifically for Cygwin.") @ddt.idata(_norm_cygpath_pairs) def test_decygpath(self, case): wpath, cpath = case @@ -156,11 +165,6 @@ def test_lock_file(self): lock_file._obtain_lock_or_raise() lock_file._release_lock() - @pytest.mark.xfail( - sys.platform == "cygwin", - reason="Cygwin fails here for some reason, always", - raises=AssertionError, - ) def test_blocking_lock_file(self): my_file = tempfile.mktemp() lock_file = BlockingLockFile(my_file) @@ -173,9 +177,8 @@ def test_blocking_lock_file(self): self.assertRaises(IOError, wait_lock._obtain_lock) elapsed = time.time() - start extra_time = 0.02 - if is_win: - # for Appveyor - extra_time *= 6 # NOTE: Indeterministic failures here... + if is_win or sys.platform == "cygwin": + extra_time *= 6 # NOTE: Indeterministic failures without this... self.assertLess(elapsed, wait_time + extra_time) def test_user_id(self): diff --git a/tox.ini b/tox.ini index 82a41e22c..f9ac25b78 100644 --- a/tox.ini +++ b/tox.ini @@ -1,6 +1,6 @@ [tox] requires = tox>=4 -env_list = py{37,38,39,310,311,312}, lint, mypy, black +env_list = py{37,38,39,310,311,312}, lint, mypy, html [testenv] description = Run unit tests @@ -11,25 +11,22 @@ commands = pytest --color=yes {posargs} [testenv:lint] description = Lint via pre-commit -base_python = py39 -commands = pre-commit run --all-files +base_python = py{39,310,311,312,38,37} +set_env = + SKIP = black-format +commands = pre-commit run --all-files --hook-stage manual [testenv:mypy] description = Typecheck with mypy -base_python = py39 +base_python = py{39,310,311,312,38,37} commands = mypy -p git ignore_outcome = true -[testenv:black] -description = Check style with black -base_python = py39 -commands = black --check --diff . - -# Run "tox -e html" for this. It is deliberately excluded from env_list, as -# unlike the other environments, this one writes outside the .tox/ directory. [testenv:html] description = Build HTML documentation -base_python = py39 +base_python = py{39,310,311,312,38,37} deps = -r doc/requirements.txt allowlist_externals = make -commands = make -C doc html +commands = + make BUILDDIR={env_tmp_dir}/doc/build -C doc clean + make BUILDDIR={env_tmp_dir}/doc/build -C doc html