Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conceptual fix for issue #3589 #126

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

brechtm
Copy link

@brechtm brechtm commented Jan 21, 2021

This makes just enough changes to the code to make python-poetry/poetry#3589 work. This is mostly intended to start discussion and collect feedback.

There are (at least) two outstanding issues:

  1. with a from = "src" packages setup, this adds the files in src twice. The resulting tar.gz does seem to be fine, however.
  2. Performance is abysmal when using include = [{ path = "**/*", format = "sdist" }] in a sizeable project. I believe this is because poetry checks for excluded files on a file-per-file bases and doesn't take shortcuts when entire directories are excluded.

Summary of commits:

  • Log all files being added
  • Pass unresolved paths to is_excluded()
  • 'exclude' should be able to override 'include' matches
  • Some remarks
  • Exclude .git directories and files in them by default

Partly resolves: python-poetry/poetry#3589

  • Added tests for changed code.
  • Updated documentation for changed code.

This makes is_excluded work with symlinks.
Note that this behavior was different for files in dirs; see
"if file.is_dir():" block a bit higher in the code.

Related: python-poetry/poetry#1336
Note that this makes a list of all files in .git directories and is
thus very slow.
if self.is_excluded(
include_file.relative_to_project_root()
) and isinstance(include, PackageInclude):
if self.is_excluded(file.relative_to(source_root)):
Copy link
Author

@brechtm brechtm Jan 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes include/exclude behavior, breaking three tests. Currently, include always trumps exclude (see also python-poetry/poetry#1336 and python-poetry/poetry#1750), which is too limiting for some use cases.

One solution, without introducing order-dependent include/exclude rules as in a a MANIFEST.in, is to give globbed paths a lower priority than fully-qualified paths. I feel this offers more flexibility while being intuitive at the same time. This would work for the examples given in python-poetry/poetry#1336, such as:

exclude = [
    "assets/sample/**/*"
]
include = [
    "assets/sample/subdir/one.yaml",
    "assets/sample/subdir/two.yaml"
]

But the following would now also work:

include = [
    "**/*"
]
exclude = [
    ".travis.yml"
]

It might be worth looking into assigning different priorities to glob patterns based on how specific they are.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

poetry build: crash on encountering a symlink
1 participant