Test failure and error in package app

[dmoney]

When I run the tests, test_score in test_models.py fails:

    ======================================================================
    FAIL: test_score (app.package.tests.test_models.VersionTests)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/app/package/tests/test_models.py", line 21, in test_score
        self.assertEqual(p.calculate_score(), p.repo_watchers)
    AssertionError: 676.9 != 967

    ----------------------------------------------------------------------
    Ran 144 tests in 1445.789s

    FAILED (failures=1)
    Destroying test database for alias 'default'...
    ERROR: 1

When I try to run just the tests for the package app, I get this failure plus an additional error for test_add_package_view in test_views.py:

    ======================================================================
    ERROR: test_add_package_view (package.tests.test_views.FunctionalPackageTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 85, in _execute
        return self.cursor.execute(sql, params)
    psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "package_package_pkey"
    DETAIL:  Key (id)=(1) already exists.


    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/app/package/tests/test_views.py", line 71, in test_add_package_view
        response = self.client.post(
      File "/usr/local/lib/python3.8/site-packages/django/test/client.py", line 756, in post
        response = super().post(path, data=data, content_type=content_type, secure=secure, **extra)
      File "/usr/local/lib/python3.8/site-packages/django/test/client.py", line 407, in post
        return self.generic('POST', path, post_data, content_type,
      File "/usr/local/lib/python3.8/site-packages/django/test/client.py", line 473, in generic
        return self.request(**r)
      File "/usr/local/lib/python3.8/site-packages/django/test/client.py", line 721, in request
        self.check_exception(response)
      File "/usr/local/lib/python3.8/site-packages/django/test/client.py", line 582, in check_exception
        raise exc_value
      File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
        response = get_response(request)
      File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
        response = wrapped_callback(request, *callback_args, **callback_kwargs)
      File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
        return view_func(request, *args, **kwargs)
      File "/app/package/views.py", line 48, in add_package
        new_package = form.save()
      File "/usr/local/lib/python3.8/site-packages/django/forms/models.py", line 466, in save
        self.instance.save()
      File "/app/package/models.py", line 346, in save
        super().save(*args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 743, in save
        self.save_base(using=using, force_insert=force_insert,
      File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 780, in save_base
        updated = self._save_table(
      File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 885, in _save_table
        results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
      File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 923, in _do_insert
        return manager._insert(
      File "/usr/local/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
        return getattr(self.get_queryset(), name)(*args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 1301, in _insert
        return query.get_compiler(using=using).execute_sql(returning_fields)
      File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1441, in execute_sql
        cursor.execute(sql, params)
      File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
        return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
      File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
        return executor(sql, params, many, context)
      File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 85, in _execute
        return self.cursor.execute(sql, params)
      File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
        raise dj_exc_value.with_traceback(traceback) from exc_value
      File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 85, in _execute
        return self.cursor.execute(sql, params)
    django.db.utils.IntegrityError: duplicate key value violates unique constraint "package_package_pkey"
    DETAIL:  Key (id)=(1) already exists.


    ======================================================================
    FAIL: test_score (app.package.tests.test_models.VersionTests)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/app/package/tests/test_models.py", line 21, in test_score
        self.assertEqual(p.calculate_score(), p.repo_watchers)
    AssertionError: 676.9 != 967

    ----------------------------------------------------------------------
    Ran 67 tests in 495.322s

    FAILED (failures=1, errors=1)
    Destroying test database for alias 'default'...
    ERROR: 1

The commands I'm running are these:

• All tests: docker-compose -f docker-compose.dev.yml run django python manage.py test

• Package app tests: docker-compose -f docker-compose.dev.yml run django python manage.py test package

Am I doing something wrong?

[dmoney]

I'll try a clean checkout in the morning and see if it still does it after I retrace my steps.

[dmoney]

Nope, still does it. Commands after cloning the repo:

    cp .env.local.example .env.local
    docker-compose -f docker-compose.dev.yml build
    docker-compose -f docker-compose.dev.yml up
    docker-compose -f docker-compose.dev.yml run django python manage.py createsuperuser --username=dustin --email=cathodion@gmail.com
    docker-compose -f docker-compose.dev.yml up
    docker-compose -f docker-compose.dev.yml run django python manage.py test

[dmoney]

This method in the Package model is what's returning something the test doesn't expect for test_score:

def calculate_score(self):
    """
    Scores a penalty of 10% of the stars for each 3 months the package is not updated;
    + a penalty of -30% of the stars if it does not support python 3.
    So an abandoned packaged for 2 years would lose 80% of its stars.
    """
    delta = relativedelta.relativedelta(now(), self.last_updated())
    delta_months = (delta.years * 12) + delta.months
    last_updated_penalty = math.modf(delta_months / 3)[1] * self.repo_watchers / 10
    last_version = self.version_set.last()
    is_python_3 = last_version and last_version.supports_python3
    # TODO: Address this better
    python_3_penalty = (
        0 if is_python_3 else min([self.repo_watchers * 30 / 100, 1000])
    )
    # penalty for docs maybe
    return self.repo_watchers - last_updated_penalty - python_3_penalty

So this scores the package based on

1. How recent it's been updated (Package.last_updated())
2. How many repo watchers it has (Package.repo_watchers)
3. Whether its latest version supports Python 3 (Package.version_set.last().supports_python3)

Since this depends on the system time and the time of last updated, maybe the test data has a date that's too old? But it looks like the test data is being inserted without an explicit date, so it should just be the current time as of when the tests start running, right?

Could be that the test data has changed and the version being pulled doesn't have supports_python3 set, so maybe it's getting that penalty? Or maybe the number of repo_watchers has changed?

I'll see if I can find anything in the commit history.

[dmoney]

@jefftriplett Is test_score inpackage/tests/test_models.py currently passing for you?

[jefftriplett]

It's failing locally for me too. I can't quite nail down what's going on because GH sees 676.9, but I see 967 locally too.

I'm working on package fixtures tonight and slowly working towards it. I keep seeing a few bugs from pypi so it's time to rewrite and get some good test cases. I should have a PR up for this tonight if the kids sleep.

[dmoney]

Good to know it's not something with my local setup.

By the way: I don't know the project very well yet, but I do have a lot of free time lately. If there's anything in particular that it would be helpful for me to focus on, feel free to let me know.

[jefftriplett]

🙏 re: free time - much appreciated.

Short term, I think we need to make tests a bit more comprehensive. The existing tests are good, but I struggle with some of the patches in place and some gaps in coverage. Ideally (like with the PyPI code), I'd like to be able to use a dump the PyPI JSON to write tests to address it, knowing what will break.

Re: test_score the only thing that sticks out to me is that this test uses now(), which will be different when run on GitHub. Maybe we can use one of the freeze time libraries and come up with a consistent result?

[dmoney]

Fix for test_score failure in PR #792.

Haven't looked into the error in test_add_package_view when only the package tests are run.

[jefftriplett]

I'd like to say this was fixed, but the real fix is to finish rewriting the tests to eventually remove data leakage and some cache issues which sometimes report a test as failing that should not.