Skip to content

Commit

Permalink
Rename --add-dll to --include
Browse files Browse the repository at this point in the history
`--add-dll` will continue to be supported as an alias
  • Loading branch information
adang1345 committed Aug 13, 2024
1 parent 72b9d24 commit 02a3c13
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 36 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ The path separator to use in the following options is `';'` on Windows and `':'`

`delvewheel show`
- `--add-path`: additional path(s) to search for DLLs, path-separator-delimited. These paths are searched before those in the `PATH` environment variable.
- `--add-dll`: name(s) of additional DLL(s) to vendor into the wheel, path-separator-delimited. We do not automatically search for dependencies of these DLLs unless another included DLL depends on them. If you use this option, it is your responsibility to ensure that the additional DLL is found at load time.
- `--include`: name(s) of additional DLL(s) to vendor into the wheel, path-separator-delimited. We do not automatically search for dependencies of these DLLs unless another included DLL depends on them. If you use this option, it is your responsibility to ensure that the additional DLL is found at load time.
- `--no-dll`: name(s) of DLL(s) to specifically exclude from the wheel, path-separator-delimited. Dependencies of these DLLs are also automatically excluded if no other included DLL depends on them.
- `--ignore-existing`: don't search for or vendor in DLLs that are already in the wheel. We still search for and vendor in dependencies of these DLLs if they are not in the wheel. This flag is meant for simpler integration with other DLL bundling tools/techniques but is not a catch-all. If you use this flag, it is your responsibility to ensure that the DLLs that are already in the wheel are loaded correctly.
- `--analyze-existing`: analyze and vendor in dependencies of DLLs that are already in the wheel. If you use this option, it is your responsibility to ensure that these dependencies are found at load time.
Expand All @@ -52,7 +52,7 @@ The path separator to use in the following options is `';'` on Windows and `':'`

`delvewheel repair`
- `--add-path`: additional path(s) to search for DLLs, path-separator-delimited. These paths are searched before those in the `PATH` environment variable.
- `--add-dll`: name(s) of additional DLL(s) to vendor into the wheel, path-separator-delimited. We do not automatically search for or vendor in dependencies of these DLLs unless another included DLL depends on them. We do not mangle the names of these DLLs or their direct dependencies. If you use this option, it is your responsibility to ensure that the additional DLL is found at load time.
- `--include`: name(s) of additional DLL(s) to vendor into the wheel, path-separator-delimited. We do not automatically search for or vendor in dependencies of these DLLs unless another included DLL depends on them. We do not mangle the names of these DLLs or their direct dependencies. If you use this option, it is your responsibility to ensure that the additional DLL is found at load time.
- `--no-dll`: name(s) of DLL(s) to specifically exclude from the wheel, path-separator-delimited. Dependencies of these DLLs are also automatically excluded if no other included DLL depends on them.
- `--ignore-existing`: don't search for or vendor in DLLs that are already in the wheel. Don't mangle the names of these DLLs or their direct dependencies. We still search for and vendor in dependencies of these DLLs if they are not in the wheel. This flag is meant for simpler integration with other DLL bundling tools/techniques but is not a catch-all. If you use this flag, it is your responsibility to ensure that the DLLs that are already in the wheel are loaded correctly.
- `--analyze-existing`: analyze and vendor in dependencies of DLLs that are already in the wheel. These dependencies are name-mangled by default. If you use this option, it is your responsibility to ensure that these dependencies are found at load time.
Expand Down Expand Up @@ -108,11 +108,11 @@ So far, we have described the simplest possible example where there exists one P
## Limitations

- `delvewheel` reads DLL file headers to determine which libraries a wheel depends on. DLLs that are loaded at runtime using `ctypes`/`cffi` (from Python) or `LoadLibrary` (from C/C++) will be missed. Support for runtime-loaded DLLs is limited; however, the following options are available.
- Specify additional DLLs to vendor into the wheel using the `--add-dll` option.
- Specify additional DLLs to vendor into the wheel using the `--include` option.
- Include the runtime-loaded DLL into the wheel yourself, and use the `--analyze-existing` option.

If you use any of these options, it is your responsibility to ensure that the runtime-loaded DLLs are found at load time.
- Wheels created using `delvewheel` are not guaranteed to work on systems older than Windows 7 SP1. We avoid vendoring system libraries that are provided by Windows 7 SP1 or later. If you intend to create a wheel for an older Windows system that requires an extra DLL, use the `--add-dll` flag to vendor additional DLLs into the wheel.
- Wheels created using `delvewheel` are not guaranteed to work on systems older than Windows 7 SP1. We avoid vendoring system libraries that are provided by Windows 7 SP1 or later. If you intend to create a wheel for an older Windows system that requires an extra DLL, use the `--include` flag to vendor additional DLLs into the wheel.
- Due to a limitation in how name-mangling is performed, `delvewheel` is unable to name-mangle DLLs whose dependents contain insufficient internal padding to fit the mangled names and contain an overlay at the end of the binary. An exception will be raised if such a DLL is encountered. Commonly, the overlay consists of symbols that can be safely removed using the GNU `strip` utility, although there exist situations where the data must be present for the DLL to function properly. To remove the overlay, execute `strip -s EXAMPLE.dll` or use the `--strip` flag. To keep the overlay and skip name mangling, use the `--no-mangle` or `--no-mangle-all` flag.
- Any DLL containing an Authenticode signature will have its signature cleared if its dependencies are name-mangled or if it was built with a non-`0` value for the [`/DEPENDENTLOADFLAG`](https://learn.microsoft.com/en-us/cpp/build/reference/dependentloadflag?view=msvc-170) linker flag.
- `delvewheel` cannot repair a wheel that contains extension modules targeting more than one CPU architecture (e.g. both `win32` and `win_amd64`). You should create a separate wheel for each CPU architecture and repair each individually.
Expand Down
8 changes: 4 additions & 4 deletions delvewheel/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def main():
for subparser in (parser_show, parser_repair):
subparser.add_argument('wheel', nargs='+', help='wheel(s) to show or repair')
subparser.add_argument('--add-path', default='', metavar='PATHS', help=f'additional path(s) to search for DLLs, {os.pathsep!r}-delimited')
subparser.add_argument('--add-dll', default='', metavar='DLLS', type=_dll_names, help=f'force inclusion of DLL name(s), {os.pathsep!r}-delimited')
subparser.add_argument('--include', '--add-dll', default='', metavar='DLLS', type=_dll_names, help=f'force inclusion of DLL name(s), {os.pathsep!r}-delimited')
subparser.add_argument('--no-dll', default='', metavar='DLLS', type=_dll_names, help=f'force exclusion of DLL name(s), {os.pathsep!r}-delimited')
subparser.add_argument('--ignore-existing', '--ignore-in-wheel', action='store_true', help="don't search for or vendor in DLLs that are already in the wheel")
subparser.add_argument('--analyze-existing', action='store_true', help='analyze and vendor in dependencies of DLLs that are already in the wheel')
Expand All @@ -66,18 +66,18 @@ def main():
# handle command
if args.command in ('show', 'repair'):
add_paths = dict.fromkeys(os.path.abspath(path) for path in args.add_path.split(os.pathsep) if path)
add_dlls = set(dll_name.lower() for dll_name in args.add_dll.split(os.pathsep) if dll_name)
include = set(dll_name.lower() for dll_name in args.include.split(os.pathsep) if dll_name)
no_dlls = set(dll_name.lower() for dll_name in args.no_dll.split(os.pathsep) if dll_name)

intersection = add_dlls & no_dlls
intersection = include & no_dlls
if intersection:
raise ValueError(f'Cannot force both inclusion and exclusion of {intersection}')

if add_paths:
os.environ['PATH'] = f'{os.pathsep.join(add_paths)}{os.pathsep}{os.environ["PATH"]}'

for wheel in args.wheel:
wr = WheelRepair(wheel, args.extract_dir, add_dlls, no_dlls, args.ignore_existing, args.analyze_existing, args.v, args.test.split(','))
wr = WheelRepair(wheel, args.extract_dir, include, no_dlls, args.ignore_existing, args.analyze_existing, args.v, args.test.split(','))
if args.command == 'show':
wr.show()
else: # args.command == 'repair'
Expand Down
18 changes: 9 additions & 9 deletions delvewheel/_wheel_repair.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ class WheelRepair:
_data_dir: str # extracted path to .data directory, is set even if directory does not exist
_purelib_dir: str # extracted path to .data/purelib directory, is set even if directory does not exist
_platlib_dir: str # extracted path to .data/platlib directory, is set even if directory does not exist
_add_dlls: typing.Set[str] # additional DLLs to addd
_include: typing.Set[str] # additional DLLs to addd
_no_dlls: typing.Set[str] # DLLs to exclude
_wheel_dirs: typing.Optional[typing.List[str]] # extracted directories from inside wheel
_ignore_existing: bool # whether to ignore DLLs that are already inside wheel
Expand All @@ -136,7 +136,7 @@ class WheelRepair:
def __init__(self,
whl_path: str,
extract_dir: typing.Optional[str],
add_dlls: typing.Optional[typing.Set[str]],
include: typing.Optional[typing.Set[str]],
no_dlls: typing.Optional[typing.Set[str]],
ignore_existing: bool,
analyze_existing: bool,
Expand All @@ -146,9 +146,9 @@ def __init__(self,
whl_path: Path to the wheel to repair
extract_dir: Directory where wheel is extracted. If None, a temporary
directory is created.
add_dlls: Set of lowercase DLL names to force inclusion into the wheel
include: Set of lowercase DLL names to force inclusion into the wheel
no_dlls: Set of lowercase DLL names to force exclusion from wheel
(cannot overlap with add_dlls)
(cannot overlap with include)
ignore_existing: whether to ignore DLLs that are already in the wheel
analyze_existing: whether to analyze and vendor in dependencies of DLLs that are already in the wheel
verbose: verbosity level, 0 to 2
Expand Down Expand Up @@ -190,7 +190,7 @@ def __init__(self,
self._purelib_dir = os.path.join(self._data_dir, 'purelib')
self._platlib_dir = os.path.join(self._data_dir, 'platlib')

self._add_dlls = set() if add_dlls is None else add_dlls
self._include = set() if include is None else include
self._no_dlls = set() if no_dlls is None else no_dlls

# Modify self._no_dlls to include those that are already part of every
Expand Down Expand Up @@ -648,7 +648,7 @@ def show(self) -> None:

# find extra dependencies specified with --add-dll
extra_dependency_paths = set()
for dll_name in self._add_dlls:
for dll_name in self._include:
dll_info = _dll_utils.find_library(dll_name, None, self._arch, False, False)
if dll_info:
extra_dependency_paths.add(dll_info[0])
Expand All @@ -658,7 +658,7 @@ def show(self) -> None:
if self._ignore_existing:
dependency_paths_in_wheel, dependency_paths_outside_wheel = self._split_dependency_paths(dependency_paths)
for path in dependency_paths_in_wheel.copy():
if os.path.basename(path).lower() in self._add_dlls:
if os.path.basename(path).lower() in self._include:
dependency_paths_in_wheel.remove(path)
dependency_paths_in_wheel = list(dependency_paths_in_wheel)
dependency_paths_in_wheel.sort()
Expand Down Expand Up @@ -769,7 +769,7 @@ def repair(
name_lower = os.path.basename(p).lower()
no_mangles.add(name_lower)
no_mangles.update(_dll_utils.get_direct_mangleable_needed(p, self._no_dlls, no_mangles, self._verbose))
if name_lower not in self._add_dlls:
if name_lower not in self._include:
ignored_dll_names.add(name_lower)
dependency_paths = dependency_paths_outside_wheel

Expand All @@ -778,7 +778,7 @@ def repair(
dependency_names = {os.path.basename(p) for p in dependency_paths} # this is NOT lowercased
dependency_names_lower = {name.lower() for name in dependency_names}
extra_dependency_paths = set()
for dll_name in self._add_dlls:
for dll_name in self._include:
if dll_name in dependency_names_lower:
continue
dll_info = _dll_utils.find_library(dll_name, None, self._arch, include_symbols, include_imports)
Expand Down
54 changes: 35 additions & 19 deletions tests/run_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -285,8 +285,8 @@ def test_strip_2(self):
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy/trailing_data_1;iknowpy/trailing_data_2;iknowpy', '--strip', '--test', 'not_enough_padding', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
self.assertTrue(import_iknowpy_successful())

def test_add_dll_1(self):
"""--add-dll for 1 DLL, case-insensitive"""
def test_add_dll(self):
"""--add-dll is alias for --include"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--add-dll', 'kernEl32.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
with zipfile.ZipFile('wheelhouse/iknowpy-1.5.3-cp312-cp312-win_amd64.whl') as wheel:
kernel32_found = False
Expand All @@ -301,19 +301,35 @@ def test_add_dll_1(self):
self.assertTrue(kernel32_found, 'kernel32.dll found')
self.assertTrue(import_iknowpy_successful())

def test_add_dll_1_exist(self):
"""-add-dll for 1 DLL that's being added anyway"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--add-dll', 'iKnowEngine.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
def test_include_1(self):
"""--include for 1 DLL, case-insensitive"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--include', 'kernEl32.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
with zipfile.ZipFile('wheelhouse/iknowpy-1.5.3-cp312-cp312-win_amd64.whl') as wheel:
kernel32_found = False
for path in zipfile.Path(wheel, 'iknowpy.libs/').iterdir():
if path.name in ('.load-order-iknowpy-1.5.3',):
continue
if path.name.startswith('kernel32'):
self.assertFalse(is_mangled(path.name), f'{path.name} is not mangled')
kernel32_found = True
else:
self.assertTrue(is_mangled(path.name), f'{path.name} is mangled')
self.assertTrue(kernel32_found, 'kernel32.dll found')
self.assertTrue(import_iknowpy_successful())

def test_include_1_exist(self):
"""-include for 1 DLL that's being added anyway"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--include', 'iKnowEngine.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
with zipfile.ZipFile('wheelhouse/iknowpy-1.5.3-cp312-cp312-win_amd64.whl') as wheel:
for path in zipfile.Path(wheel, 'iknowpy.libs/').iterdir():
if path.name in ('.load-order-iknowpy-1.5.3',):
continue
self.assertTrue(is_mangled(path.name), f'{path.name} is mangled')
self.assertTrue(import_iknowpy_successful())

def test_add_dll_2_repeat(self):
"""--add-dll for 2 DLLs that are the same"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--add-dll', 'kernel32.dll;kernel32.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
def test_include_2_repeat(self):
"""--include for 2 DLLs that are the same"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--include', 'kernel32.dll;kernel32.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
with zipfile.ZipFile('wheelhouse/iknowpy-1.5.3-cp312-cp312-win_amd64.whl') as wheel:
kernel32_found = False
for path in zipfile.Path(wheel, 'iknowpy.libs/').iterdir():
Expand All @@ -327,9 +343,9 @@ def test_add_dll_2_repeat(self):
self.assertTrue(kernel32_found, 'kernel32.dll found')
self.assertTrue(import_iknowpy_successful())

def test_add_dll_2(self):
"""--add-dll for 2 DLLs"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--add-dll', 'kernel32.dll;kernelbase.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
def test_include_2(self):
"""--include for 2 DLLs"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--include', 'kernel32.dll;kernelbase.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
kernel32_found = False
kernelbase_found = False
with zipfile.ZipFile('wheelhouse/iknowpy-1.5.3-cp312-cp312-win_amd64.whl') as wheel:
Expand All @@ -348,10 +364,10 @@ def test_add_dll_2(self):
self.assertTrue(kernelbase_found, 'kernelbase.dll found')
self.assertTrue(import_iknowpy_successful())

def test_add_dll_no_dll_overlap(self):
"""overlap between --add-dll and --no-dll generates an error"""
def test_include_no_dll_overlap(self):
"""overlap between --include and --no-dll generates an error"""
with self.assertRaises(subprocess.CalledProcessError):
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--add-dll', 'kernel32.dll', '--no-dll', 'Kernel32.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--include', 'kernel32.dll', '--no-dll', 'Kernel32.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])

def test_no_dll_irrelevant(self):
"""--no-dll for DLL that's not included anyway"""
Expand Down Expand Up @@ -444,10 +460,10 @@ def test_ignore_in_wheel(self):
self.assertTrue(import_iknowpy_successful('0ignore'))

def test_ignore_existing_override(self):
"""--ignore-existing would ignore iKnowEngine.dll, but --add-dll
"""--ignore-existing would ignore iKnowEngine.dll, but --include
overrides this and prevents it and its direct dependencies from name-
mangling."""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--ignore-existing', '--add-dll', 'iKnowEngine.dll', 'iknowpy/iknowpy-1.5.3-0ignore-cp312-cp312-win_amd64.whl'])
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--ignore-existing', '--include', 'iKnowEngine.dll', 'iknowpy/iknowpy-1.5.3-0ignore-cp312-cp312-win_amd64.whl'])
iknowengine_found = False
with zipfile.ZipFile('wheelhouse/iknowpy-1.5.3-0ignore-cp312-cp312-win_amd64.whl') as wheel:
for path in zipfile.Path(wheel, 'iknowpy.libs/').iterdir():
Expand Down Expand Up @@ -1334,9 +1350,9 @@ def test_repair_no_mangle_2(self):
else:
self.assertTrue(is_mangled(path.name), f'{path.name} is mangled')

def test_add_dll_1(self):
"""--add-dll for 1 DLL, case-insensitive"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--add-dll', 'kernEl32.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
def test_include_1(self):
"""--include for 1 DLL, case-insensitive"""
check_call(['delvewheel', 'repair', '--add-path', 'iknowpy', '--include', 'kernEl32.dll', 'iknowpy/iknowpy-1.5.3-cp312-cp312-win_amd64.whl'])
with zipfile.ZipFile('wheelhouse/iknowpy-1.5.3-cp312-cp312-win_amd64.whl') as wheel:
kernel32_found = False
for path in zipfile.Path(wheel, 'iknowpy.libs/').iterdir():
Expand Down

0 comments on commit 02a3c13

Please sign in to comment.