flit_core does not filter out Python bytecode when not operating on vcs

[nanonyme]

When flit_core is generating allowed files into sdist

flit/flit_core/flit_core/sdist.py

Lines 109 to 118 in ec6e6b7

	def select_files(self):
	"""Pick which files from the source tree will be included in the sdist
	This is overridden in flit itself to use information from a VCS to
	include tests, docs, etc. for a 'gold standard' sdist.
	"""
	cfgdir_s = str(self.cfgdir)
	return [
	osp.relpath(p, cfgdir_s) for p in self.module.iter_files()
	] + self.extra_files

it does not use

flit/flit/sdist.py

Lines 98 to 101 in ec6e6b7

	def include_path(p):
	return not (p.startswith('dist' + os.sep)
	or (os.sep+'__pycache__' in p)
	or p.endswith('.pyc'))

for filtering on output files which results in Python bytecode ending up in package. Looks like there is filtering done through

flit/flit_core/flit_core/common.py

Lines 79 to 83 in ec6e6b7

	def _include(path):
	name = os.path.basename(path)
	if (name == '__pycache__') or name.endswith('.pyc'):
	return False
	return True

but it appears not to cover all cases. This is almost certainly undesired behaviour as there is no portability with Python bytecode across other Python versions.

[takluyver]

Can you describe a specific scenario where this is going wrong? The filtering works in somewhat different contexts, and I often live with little bits of duplication rather than tying modules pieces more closely together.

[layday]

When building with flit-core, __pycache__ folders are only excluded from the "module" folder, so if you were to add e.g. your tests folders in the sdist, tests/__pycache__ won't be filtered out. IIUC, the flit builder excludes these from everywhere in the sdist.

[takluyver]

Aha, I see. So the _include function is fine, because Module.iter_files() is only going through the files in the importable module - other files you configure to include in the sdist are added by SdistBuilder.apply_includes_excludes().

Maybe the simplest way to do it is to always add **/__pycache__ and **.pyc to the exclude patterns? Excludes override inclues, so there would be no way to configure including them, but I can't think of a reason you'd need that (and Flit doesn't aim to cover every possible scenario anyway).

[nanonyme]

@takluyver do we really need also the directory? Does flit package empty directories?