Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add purge_files method to CoverageData + unit tests for it #1547

Merged
merged 4 commits into from
Feb 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions coverage/sqldata.py
Original file line number Diff line number Diff line change
Expand Up @@ -615,6 +615,42 @@ def touch_files(self, filenames: Collection[str], plugin_name: Optional[str] = N
# Set the tracer for this file
self.add_file_tracers({filename: plugin_name})

def purge_files(self, filenames: Iterable[str], context: Optional[str] = None) -> None:
"""Purge any existing coverage data for the given `filenames`.

If `context` is given, purge only data associated with that measurement context.
"""

if self._debug.should("dataop"):
self._debug.write(f"Purging {filenames!r} for context {context}")
self._start_using()
with self._connect() as con:

if context is not None:
context_id = self._context_id(context)
if context_id is None:
raise DataError("Unknown context {context}")
else:
context_id = None

if self._has_lines:
table = 'line_bits'
elif self._has_arcs:
table = 'arcs'
else:
return

for filename in filenames:
file_id = self._file_id(filename, add=False)
if file_id is None:
continue
self._file_map.pop(filename, None)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After I merged, I noticed this line: if you are only removing one context for the file, then we don't want to remove the file from the file_map, correct? Why is it important to remove it from the map, and what should we do if it's only a single context being removed?

Copy link
Contributor

@sdeibel sdeibel Feb 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. It looks like in Wing i'm only calling this with context==None and in all cases I immediately set updated data with set_context and add_lines so all the contexts are restored, but on the right new line numbers. Should I go ahead and just remove the context arg? I think it was a mistake that came from trying to follow what other API calls support for arguments. Now I can't really think of a reason to have that argument.

I do remember popping from _file_map was needed or some aspect of the update did not work because measured_files was called. I'm going to investigate that further since popping here may indeed be the wrong way to solve that part of it.

Another option would be to replace this with a call that purges/replaces data with a new set of lines all in one operation. My initial idea was the low-level approach was a better match for this API, but I could have been wrong.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the original cur.execute call didn't actually remove data because the cursor wasn't iterated. I replaced it with cur.execute_void and it had a better effect.

If you don't need the per-context operation, then let's definitely simplify it to only remove the entire file. I think I'd have it remove the data for the file, but leave the file in the set of measured files, and state that it's possible for a "measured file" to have no data. Would that work for you?

If we find a need for per-context purging later, we could add it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've implemented this in #1551 - wasn't sure how to add to this pull request so ended up with a new one. Hope that is OK...

if context_id is None:
q = f'delete from {table} where file_id={file_id}'
else:
q = f'delete from {table} where file_id={file_id} and context_id={context_id}'
con.execute(q)

def update(self, other_data: CoverageData, aliases: Optional[PathAliases] = None) -> None:
"""Update this data with data from several other :class:`CoverageData` instances.

Expand Down
106 changes: 106 additions & 0 deletions tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -754,6 +754,112 @@ def test_run_debug_sys(self) -> None:
cov.stop() # pragma: nested
assert cast(str, d['data_file']).endswith(".coverage")

def test_purge_filenames(self) -> None:

fn1 = self.make_file("mymain.py", """\
import mymod
a = 1
""")
fn1 = os.path.join(self.temp_dir, fn1)

fn2 = self.make_file("mymod.py", """\
fooey = 17
""")
fn2 = os.path.join(self.temp_dir, fn2)

cov = coverage.Coverage()
self.start_import_stop(cov, "mymain")

data = cov.get_data()

# Initial measurement was for two files
assert len(data.measured_files()) == 2
assert [1, 2] == sorted_lines(data, fn1)
assert [1,] == sorted_lines(data, fn2)

# Purge one file's data and one should remain
data.purge_files([fn1])
assert len(data.measured_files()) == 1
assert [] == sorted_lines(data, fn1)
assert [1,] == sorted_lines(data, fn2)

# Purge second file's data and none should remain
data.purge_files([fn2])
assert len(data.measured_files()) == 0
assert [] == sorted_lines(data, fn1)
assert [] == sorted_lines(data, fn2)

def test_purge_filenames_context(self) -> None:

fn1 = self.make_file("mymain.py", """\
import mymod
a = 1
""")
fn1 = os.path.join(self.temp_dir, fn1)

fn2 = self.make_file("mymod.py", """\
fooey = 17
""")
fn2 = os.path.join(self.temp_dir, fn2)

def dummy_function() -> None:
unused = 42

# Start/stop since otherwise cantext
cov = coverage.Coverage()
cov.start()
cov.switch_context('initialcontext')
dummy_function()
cov.switch_context('testcontext')
cov.stop()
self.start_import_stop(cov, "mymain")

data = cov.get_data()

# Initial measurement was for three files and two contexts
assert len(data.measured_files()) == 3
assert [1, 2] == sorted_lines(data, fn1)
assert [1,] == sorted_lines(data, fn2)
assert len(sorted_lines(data, __file__)) == 1
assert len(data.measured_contexts()) == 2

# Remove specifying wrong context should raise exception and not remove anything
try:
data.purge_files([fn1], 'wrongcontext')
except coverage.sqldata.DataError:
pass
else:
assert 0, "exception expected"
assert len(data.measured_files()) == 3
assert [1, 2] == sorted_lines(data, fn1)
assert [1,] == sorted_lines(data, fn2)
assert len(sorted_lines(data, __file__)) == 1
assert len(data.measured_contexts()) == 2

# Remove one file specifying correct context
data.purge_files([fn1], 'testcontext')
assert len(data.measured_files()) == 2
assert [] == sorted_lines(data, fn1)
assert [1,] == sorted_lines(data, fn2)
assert len(sorted_lines(data, __file__)) == 1
assert len(data.measured_contexts()) == 2

# Remove second file with other correct context
data.purge_files([__file__], 'initialcontext')
assert len(data.measured_files()) == 1
assert [] == sorted_lines(data, fn1)
assert [1,] == sorted_lines(data, fn2)
assert len(sorted_lines(data, __file__)) == 0
assert len(data.measured_contexts()) == 2

# Remove last file specifying correct context
data.purge_files([fn2], 'testcontext')
assert len(data.measured_files()) == 0
assert [] == sorted_lines(data, fn1)
assert [] == sorted_lines(data, fn2)
assert len(sorted_lines(data, __file__)) == 0
assert len(data.measured_contexts()) == 2


class CurrentInstanceTest(CoverageTest):
"""Tests of Coverage.current()."""
Expand Down