Merge branch 'develop'

* develop: Check blackness in github actions (pdfminer#711) Changed `log.info` to `log.debug` in six files (pdfminer#690) Update README.md batch for Continuous integration Update actions.yml so that it will run for all PR's Update development tools: travis ci to github actions, tox to nox, nose to pytest (pdfminer#704) Added feature: page labels (pdfminer#680) Remove obsolete returns (pdfminer#707) Revert "Remove obsolete returns" Remove obsolete returns Only use xref fallback if `PDFNoValidXRef` is raised and `fallback` is True (pdfminer#684) Use logger.warn instead of warnings.warn if warning cannot be prevented by user (pdfminer#673) Change log.info into log.debug to make pdfinterp.py less verbose Fix regression in page layout that sometimes returned text lines out of order (pdfminer#659) export type annotations in package (pdfminer#679) fix typos in PR template (pdfminer#681) pdf2txt: clean up construction of LAParams from arguments (pdfminer#682) Fixes jbig2 writer to write valid jb2 files Add support for JPEG2000 image encoding Added test case for CCITTFaxDecoder (pdfminer#700) Attempt to handle decompression error on some broken PDF files (pdfminer#637)
HiTalentAlgorithms · Feb 14, 2022 · 5d96681 · 5d96681
2 parents d2fd331 + b9a8920
commit 5d96681
Show file tree

Hide file tree

Showing 72 changed files with 13,613 additions and 7,776 deletions.
diff --git a/.flake8 b/.flake8
@@ -0,0 +1,5 @@
+[flake8]
+max-line-length = 88
+extend-ignore = 
+    # See https://github.com/PyCQA/pycodestyle/issues/373
+    E203,
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -1,30 +1,25 @@
 **Pull request**
 
-Thanks for improving pdfminer.six! Please include the following information to
-help us discuss and merge this PR:
-
-- A description of why this PR is needed. What does it fix? What does it 
-  improve?
-- A summary of the things that this PR changes.
-- Reference the issues that this PR fixes (use the fixes #(issue nr) syntax). 
-  If this PR does not fix any issue, create the issue first and mention that 
-  you are willing to work on it.
+Please remove this paragraph and replace it with a description of your PR.
+Also include links to the issues that it fixes. 
 
 **How Has This Been Tested?**
 
-Please describe the tests that you ran to verify your changes. Provide 
-instructions so we can reproduce. Include an example pdf if you have one. 
+Please repalce this paragraph with a description of how this PR has been 
+tested. Include the necessary instructions and files such that other can
+reproduce it.
 
 **Checklist**
 
+- [ ] I have formatted my code with [black](https://github.com/psf/black).
 - [ ] I have added tests that prove my fix is effective or that my feature 
   works
 - [ ] I have added docstrings to newly created methods and classes
 - [ ] I have optimized the code at least one time after creating the initial 
   version
-- [ ] I have updated the [README.md](../README.md) or I am verified that this
+- [ ] I have updated the [README.md](../README.md) or verified that this
   is not necessary
-- [ ] I have updated the [readthedocs](../docs/source) documentation or I 
+- [ ] I have updated the [readthedocs](../docs/source) documentation or
   verified that this is not necessary
-- [ ] I have added a consice human-readable description of the change to 
+- [ ] I have added a concise human-readable description of the change to
   [CHANGELOG.md](../CHANGELOG.md)
diff --git a/.github/workflows/actions.yml b/.github/workflows/actions.yml
@@ -0,0 +1,110 @@
+name: Continuous integration
+
+on:
+  push: # run when commits are added to master or develop
+    branches:
+      - master
+      - develop
+  pull_request: # run on pr's against master or develop
+    branches:
+      - master
+      - develop
+
+env:
+  default-python: "3.10"
+
+jobs:
+
+  check-code-formatting:
+    name: Check code formatting
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v2
+      - name: Check code formatting
+        uses: psf/black@stable
+
+  check-coding-style:
+    name: Check coding style
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v2
+      - name: Set up Python ${{ env.default-python }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ env.default-python }}
+      - name: Upgrade pip, Install nox
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install nox
+      - name: Check coding style
+        run: |
+          nox --error-on-missing-interpreters --non-interactive --session lint
+
+  check-static-types:
+    name: Check static types
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v2
+      - name: Set up Python ${{ env.default-python }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ env.default-python }}
+      - name: Upgrade pip, Install nox
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install nox
+      - name: Check static types
+        run: |
+          nox --error-on-missing-interpreters --non-interactive --session types
+
+  tests:
+    name: Run tests
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ ubuntu-latest ]
+        python-version: [ "3.6", "3.7", "3.8", "3.9", "3.10" ]
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v2
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Determine pip cache directory
+        id: pip-cache
+        run: |
+          echo "::set-output name=dir::$(pip cache dir)"
+      - name: Cache pip cache
+        uses: actions/cache@v2
+        with:
+          path: ${{ steps.pip-cache.outputs.dir }}
+          key: ${{ runner.os }}-pip${{ matrix.python-version }}
+      - name: Upgrade pip and install nox
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install nox
+      - name: Run tests
+        run: |
+          nox --non-interactive --session tests-${{ matrix.python-version }}
+
+  build-docs:
+    name: Test building docs
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v2
+      - name: Set up Python ${{ env.default-python }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ env.default-python }}
+      - name: Upgrade pip and install nox
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install nox
+      - name: Build docs
+        run: |
+          nox --error-on-missing-interpreters --non-interactive --session docs
diff --git a/.gitignore b/.gitignore
@@ -17,6 +17,7 @@ tests/*.xml
 tests/*.txt
 .idea/
 .tox/
+.nox/
 
 # python venv management tools
 Pipfile

diff --git a/.travis.yml b/.travis.yml
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -15,7 +15,24 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 ## [20211115]
 
 ### Added
+- Export type annotations from pypi package per PEP561 ([#679](https://github.com/pdfminer/pdfminer.six/pull/679))
 - Support for identity cmap's ([#626](https://github.com/pdfminer/pdfminer.six/pull/626))
+- Add support for PDF page labels ([#680](https://github.com/pdfminer/pdfminer.six/pull/680))
+
+### Fixed
+- Hande decompression error due to CRC checksum error ([#637](https://github.com/pdfminer/pdfminer.six/pull/637))
+- Regression (since 20191107) in `LTLayoutContainer.group_textboxes` that returned some text lines out of order ([#659](https://github.com/pdfminer/pdfminer.six/pull/659))
+- Add handling of JPXDecode filter to enable extraction of images for some pdfs ([#645](https://github.com/pdfminer/pdfminer.six/pull/645))
+- Fix extraction of jbig2 files, which was producing invalid files ([#652](https://github.com/pdfminer/pdfminer.six/pull/653))
+- Crash in `pdf2txt.py --boxes-flow=disabled` ([#682](https://github.com/pdfminer/pdfminer.six/pull/682))
+- Only use xref fallback if `PDFNoValidXRef` is raised and `fallback` is True ([#684](https://github.com/pdfminer/pdfminer.six/pull/684))
+
+### Changed
+- Replace warnings.warn with logging.Logger.warning in line with [recommended use](https://docs.python.org/3/howto/logging.html#when-to-use-logging) ([#673](https://github.com/pdfminer/pdfminer.six/pull/673))
+- Switched from nose to pytest, from tox to nox and from Travis CI to GitHub Actions ([#704](https://github.com/pdfminer/pdfminer.six/pull/704))
+
+### Removed
+- Unnecessary return statements without argument at the end of functions ([#707](https://github.com/pdfminer/pdfminer.six/pull/707))
 
 ### Fixed
 - Fix issue of TypeError: `cid_registry` and `cid_ordering` are not `bytes`

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -31,7 +31,7 @@ Any contribution is appreciated! You might want to:
 * Include unit tests when possible. In case of bugs, this will help to prevent the same mistake in the future. In case 
   of features, this will show that your code works correctly.
 * Code should work for Python 3.6+.
-* Code should conform to PEP8 coding style.
+* Code should be formatted with [black](https://github.com/psf/black). 
 * New features should be well documented using docstrings.
 * Check spelling and grammar.
 * Don't forget to update the [CHANGELOG.md](CHANGELOG.md#[Unreleased])
@@ -60,11 +60,17 @@ Any contribution is appreciated! You might want to:
     On all Python versions:
 
     ```sh
-    tox
+    nox
    ```
 
    Or on a single Python version:
 
    ```sh
-    tox -e py36
+    nox -e py36
+    ```
+
+4. After changing the code, run the black formatter.
+
+    ```sh
+    black .
     ```
diff --git a/Makefile b/Makefile
@@ -53,6 +53,3 @@ $(CMAPDST)/to-unicode-Adobe-Japan1.pickle.gz: $(CMAPDST)
 $(CMAPDST)/to-unicode-Adobe-Korea1.pickle.gz: $(CMAPDST)
 	$(CONV_CMAP) -c KSC-EUC=euc-kr -c KSC-Johab=johab -c KSCms-UHC=cp949 -c UniKS-UTF8=utf-8 \
 		$(CMAPDST) Adobe-Korea1 $(CMAPSRC)/cid2code_Adobe_Korea1.txt
-
-test: cmap
-	nosetests
diff --git a/README.md b/README.md
@@ -1,8 +1,8 @@
 pdfminer.six
 ============
 
-[![Build Status](https://travis-ci.com/pdfminer/pdfminer.six.svg?branch=develop)](https://travis-ci.com/pdfminer/pdfminer.six)
-[![PyPI version](https://img.shields.io/pypi/v/pdfminer.six.svg)](https://pypi.org/project/pdfminer.hitalent/)
+[![Continuous integration](https://github.com/pdfminer/pdfminer.six/actions/workflows/actions.yml/badge.svg)](https://github.com/pdfminer/pdfminer.six/actions/workflows/actions.yml)
+[![PyPI version](https://img.shields.io/pypi/v/pdfminer.six.svg)](https://pypi.python.org/pypi/pdfminer.six/)
 [![gitter](https://badges.gitter.im/pdfminer-six/Lobby.svg)](https://gitter.im/pdfminer-six/Lobby?utm_source=badge&utm_medium)
 
 *We fathom PDF*

diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -16,14 +16,13 @@
 
 import pdfminer
 
-sys.path.insert(0, os.path.join(
-    os.path.abspath(os.path.dirname(__file__)), '../../'))
+sys.path.insert(0, os.path.join(os.path.abspath(os.path.dirname(__file__)), "../../"))
 
 # -- Project information -----------------------------------------------------
 
-project = 'pdfminer.six'
-copyright = '2019, Yusuke Shinyama, Philippe Guglielmetti & Pieter Marsman'
-author = 'Yusuke Shinyama, Philippe Guglielmetti & Pieter Marsman'
+project = "pdfminer.six"
+copyright = "2019, Yusuke Shinyama, Philippe Guglielmetti & Pieter Marsman"
+author = "Yusuke Shinyama, Philippe Guglielmetti & Pieter Marsman"
 
 # The full version, including alpha/beta/rc tags
 release = pdfminer.__version__
@@ -35,16 +34,16 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    'sphinxarg.ext',
-    'sphinx.ext.autodoc',
-    'sphinx.ext.doctest',
+    "sphinxarg.ext",
+    "sphinx.ext.autodoc",
+    "sphinx.ext.doctest",
 ]
 
 # Root rst file
-master_doc = 'index'
+master_doc = "index"
 
 # Add any paths that contain templates here, relative to this directory.
-templates_path = ['_templates']
+templates_path = ["_templates"]
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
@@ -57,9 +56,9 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'alabaster'
+html_theme = "alabaster"
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ['_static']
+html_static_path = ["_static"]
diff --git a/mypy.ini b/mypy.ini
@@ -20,8 +20,11 @@ disallow_untyped_defs = True
 [mypy-cryptography.hazmat.*]
 ignore_missing_imports = True
 
-[mypy-nose.*]
+[mypy-pytest.*]
 ignore_missing_imports = True
 
 [mypy-setuptools]
 ignore_missing_imports = True
+
+[mypy-nox]
+ignore_missing_imports = True
diff --git a/noxfile.py b/noxfile.py
@@ -0,0 +1,35 @@
+import nox
+
+
+PYTHON_ALL_VERSIONS = ["3.6", "3.7", "3.8", "3.9", "3.10"]
+
+
+@nox.session
+def lint(session):
+    session.install("flake8")
+    session.run("flake8", "pdfminer/", "tools/", "tests/", "--count", "--statistics")
+
+
+@nox.session
+def types(session):
+    session.install("mypy")
+    session.run(
+        "mypy", "--install-types", "--non-interactive", "--show-error-codes", "."
+    )
+
+
+@nox.session(python=PYTHON_ALL_VERSIONS)
+def tests(session):
+    session.install("-e", ".[dev]")
+    session.run("pytest")
+
+
+@nox.session
+def docs(session):
+    session.install("-e", ".[docs]")
+    session.run(
+        "python", "-m", "sphinx", "-b", "html", "docs/source", "docs/build/html"
+    )
+    session.run(
+        "python", "-m", "sphinx", "-b", "doctest", "docs/source", "docs/build/doctest"
+    )
diff --git a/pdfminer/__init__.py b/pdfminer/__init__.py
@@ -1,4 +1,4 @@
 __version__ = '20211125'
 
-if __name__ == '__main__':
+if __name__ == "__main__":
     print(__version__)
-Original file line number
+Diff line change
@@ Expand Up / @@ -17,6 +17,7 @@ tests/*.xml @@
     tests/*.txt
     .idea/
     .tox/
+    .nox/
     # python venv management tools
     Pipfile
@@ Expand Down @@