Coverage(data_file=None) different behaviour (v5.0b1)

[Matlino]

Describe the bug
In coverage 4.* versions, when i used Coverage(data_file=None), DB file was not created. It seems to be not case in 5.0b1 version. Is there a way how to not create DB file in this new version?

[nedbat]

Using data_file=None was supposed to mean, use the default of ".coverage". The difference in 5.0 is that now the data file (a SQLite database now) is created at the start of the process rather than at the end.

How are you using the Coverage object? Something else must also be a little different to have stopped a data file from being created.

Also, can you say more about your use-case? Why is it important to not have a data file?

[Matlino]

We have one coverage instance and we call start and stop multiple times, always calling erase before start. In coverage version 5 we start to experience invalid data returned from coverage and also error. I can demonstrate our problem using the snippet:

import coverage
cov = coverage.Coverage()

def m1():
    1

def run_one_method(m):
    cov.erase()
    cov.start()
    m()
    cov.stop()

    fs = cov.get_data().measured_files()
    print(cov.get_data().lines(list(fs)[0]))


run_one_method(m1)
run_one_method(m1)
# run_one_method(m1)

After i run the script above i would expect output:
[5]
[5]
But i get output:
[5]
None

When i uncomment last line, i get also this error:

Traceback (most recent call last):
  File "/Users/matlo/.local/share/virtualenvs/tmp_project-xbCq7jN1/lib/python3.7/site-packages/coverage/sqldata.py", line 1025, in execute
    return self.con.execute(sql, parameters)
sqlite3.OperationalError: no such table: context

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "my_cov.py", line 20, in <module>
    run_one_method(m1)
  File "my_cov.py", line 13, in run_one_method
    cov.save()
  File "/Users/matlo/.local/share/virtualenvs/tmp_project-xbCq7jN1/lib/python3.7/site-packages/coverage/control.py", line 595, in save
    data = self.get_data()
  File "/Users/matlo/.local/share/virtualenvs/tmp_project-xbCq7jN1/lib/python3.7/site-packages/coverage/control.py", line 649, in get_data
    if self._collector and self._collector.flush_data():
  File "/Users/matlo/.local/share/virtualenvs/tmp_project-xbCq7jN1/lib/python3.7/site-packages/coverage/collector.py", line 425, in flush_data
    self.covdata.add_lines(abs_file_dict(self.data))
  File "/Users/matlo/.local/share/virtualenvs/tmp_project-xbCq7jN1/lib/python3.7/site-packages/coverage/sqldata.py", line 440, in add_lines
    self._set_context_id()
  File "/Users/matlo/.local/share/virtualenvs/tmp_project-xbCq7jN1/lib/python3.7/site-packages/coverage/sqldata.py", line 399, in _set_context_id
    context_id = self._context_id(context)
  File "/Users/matlo/.local/share/virtualenvs/tmp_project-xbCq7jN1/lib/python3.7/site-packages/coverage/sqldata.py", line 376, in _context_id
    row = con.execute("select id from context where context = ?", (context,)).fetchone()
  File "/Users/matlo/.local/share/virtualenvs/tmp_project-xbCq7jN1/lib/python3.7/site-packages/coverage/sqldata.py", line 1027, in execute
    raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
coverage.misc.CoverageException: Couldn't use data file '/Users/matlo/dev/testmon.io/tmp_project/.coverage': no such table: context

Important thing to note is that when i dont use cov.erase() command everything works. But we process the acquired data everytime we stop coverage. So we need to also erase it.

[tarpas]

@nedbat the use case is pytest-testmon.

For conceptual discussion how to merge some of the goals/efforts with coverage 5.0: #774

For now. Would it be difficult to allow using coverage.Coverage as before? With as little overhead as possible when doing these operations:

    cov.erase()
    cov.start()
    test()
    cov.stop()

    data = cov.get_data()

This has to be called for each test, so ideally it wouldn't have any expensive operation (like saving a file or creating tables in sqlite.)...

[nedbat]

@tarpas I can look into making that work. Though I wonder if it would work for you to make a new Coverage object each time. I don't think that will add overhead, will it?

[tarpas]

You say the database is created at the start of the process, so I guess when creating Coverage object. Doing it once is fine, of course, but for each unit test, it feels quite a lot of unnecessary overhead. But I didn't measure..

[nedbat]

@Matlino @tarpas I've made a branch with support for this: nedbat/datafile-none. On that branch, Coverage(data_file=None) will prevent writing any data file at all. Can you try it and see if it works for you?

[nedbat]

To install the branch, use:

pip install git+https://github.com/nedbat/coveragepy@nedbat/datafile-none#egg=coverage

[tarpas]

Thanks, we'll try tomorrow in the morning.

[Matlino]

@nedbat I tested it and it works as expected.

[nedbat]

Thanks. I've merged (and deleted) the branch.
Fixed in b75575f.

[nedbat]

This was released as part of 5.0b2 today.

[Matlino]

Thank you for info.