Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import extraction with Rust #581

Merged
merged 79 commits into from
Mar 13, 2024
Merged

Import extraction with Rust #581

merged 79 commits into from
Mar 13, 2024

Conversation

fpgmaas
Copy link
Owner

@fpgmaas fpgmaas commented Mar 9, 2024

PR Checklist

  • A description of the changes is added to the description of this PR.
  • If there is a related issue, make sure it is linked to this PR.
  • If you've fixed a bug or added code that should be tested, add tests!
  • Documentation in docs is updated

Description of changes

This PR adds Rust functionality to deptry. For now, the only part that is handled by Rust is the reading for .py files, parsing their AST and extracting the import statements. This PR also gives us a good starting point to potentially replace more parts of deptry with Rust. For a benchmark comparison of the speed improvement, see this comment.

List of changes:

  • Adds maturin to build a Python/Rust hybrid project.
  • Move from Poetry to PDM, so we have PEP-621 compatible metadata. Change backend to maturin.
  • Adds a 'get_imports_from_py_files' that can be imported in Python as from deptry.rust import get_imports_from_py_files
  • Minimal changes in the unit tests; most changes are an improvement in the 'column' of detected imports; earlier the column was 0 now it is 8.
  • We use ABI to build and publish wheels. For an example of a CI/CD run, see here and for the files on test PyPi see [here](the files on test pypi)
  • Rewrote the entire CI/CD pipeline

Once this PR is approved, I will modify the 'required status checks before merging'.

@fpgmaas fpgmaas marked this pull request as draft March 9, 2024 10:12
@fpgmaas fpgmaas marked this pull request as ready for review March 11, 2024 07:35
@fpgmaas fpgmaas marked this pull request as draft March 11, 2024 07:49
@fpgmaas fpgmaas marked this pull request as ready for review March 12, 2024 10:59
@fpgmaas fpgmaas linked an issue Mar 12, 2024 that may be closed by this pull request
@mkniewallner mkniewallner force-pushed the feature/rust branch 3 times, most recently from 9bd61b3 to 83e4b2a Compare March 12, 2024 20:23
@mkniewallner mkniewallner force-pushed the feature/rust branch 6 times, most recently from 7df67fe to e60c822 Compare March 13, 2024 01:45
@fpgmaas fpgmaas merged commit a13add1 into main Mar 13, 2024
29 checks passed
@fpgmaas fpgmaas deleted the feature/rust branch March 13, 2024 11:13
@fpgmaas fpgmaas changed the title WIP: Import extraction with Rust Import extraction with Rust Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Speed up deptry by using Rust
2 participants