remove outdated tests and mark the ones we need to skip for now

[fgregg]

Overview

This PR updates the tests. It adjust some tests, removes some unnecessary ones, and skips a few that will fail until we bring in other PRs.

Testing Instructions

docker-compose -f docker-compose.yml -f tests/docker-compose.yml run test

[fgregg]

we don't have this attribute on the Events model anymore. It was a complicate bit of code, that we've handled upstream by making sure minutes are attache to the right meeting in the scraper code.

[fgregg]

We can't keep this exact behavior and instead will do what we describe her: #509

[hancush]

Thanks, @fgregg! This is a bear of a task. Left a few comments inline.

[hancush]

Why define a new service instead of overriding app?

[fgregg]

it didn't occur to me to override. they seem quite different.

I would say that the intent of docker-compose -f docker-compose.yml -f tests/docker-compose.yml run test seems a lot clearer to me than docker-compose -f docker-compose.yml -f tests/docker-compose.yml run app

[hancush]

It's our current practice to override the application service to run the tests because it reduces repeat configuration code and attempts to ensure as much consistency between app and test environments as possible. (The latter will be more compelling when we start to use our containers in deployed environments, IMO, but it's there nonetheless.) I'd prefer to stick with our pattern and hear proposals for changes over in how-to, if that's all right with you?

[fgregg]

ok

[hancush]

Are you able to run the tests locally? I'm getting this exception:

(la-metro-councilmatic) call-me-hank:la-metro-councilmatic hannah$ docker-compose -f docker-compose.yml -f tests/docker-compose.yml run test
Starting lametro-postgres ... done
Starting lametro-solr     ... done
==================================================================== test session starts =====================================================================
platform linux -- Python 3.6.8, pytest-5.3.1, py-1.8.0, pluggy-0.13.1 -- /usr/local/bin/python
cachedir: .pytest_cache
Django settings: councilmatic.settings (from environment variable)
rootdir: /app, inifile: pytest.ini
plugins: django-3.7.0, mock-1.6.3
collected 26 items

tests/test_bills.py::test_bill_url Creating test database for alias 'default'...
Got an error creating the test database: database "test_lametro" already exists

Destroying old test database for alias 'default'...
ERROR

=========================================================================== ERRORS ===========================================================================
______________________________________________________________ ERROR at setup of test_bill_url _______________________________________________________________

self = <django.db.backends.utils.CursorWrapper object at 0x7f584efcdef0>, sql = 'CREATE EXTENSION IF NOT EXISTS postgis', params = None
ignored_wrapper_args = (False, {'connection': <django.contrib.gis.db.backends.postgis.base.DatabaseWrapper object at 0x7f586869a710>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7f584efcdef0>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
>               return self.cursor.execute(sql)
E               psycopg2.errors.UndefinedFile: could not open extension control file "/usr/share/postgresql/9.6/extension/postgis.control": No such file or directory

/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:82: UndefinedFile

The above exception was the direct cause of the following exception:

request = <SubRequest 'django_db_setup' for <Function test_bill_url>>, django_test_environment = None
django_db_blocker = <pytest_django.plugin._DatabaseBlocker object at 0x7f5869c05518>, django_db_use_migrations = True, django_db_keepdb = False
django_db_createdb = False, django_db_modify_db_settings = None

    @pytest.fixture(scope="session")
    def django_db_setup(
        request,
        django_test_environment,
        django_db_blocker,
        django_db_use_migrations,
        django_db_keepdb,
        django_db_createdb,
        django_db_modify_db_settings,
    ):
        """Top level fixture to ensure test databases are available"""
        from .compat import setup_databases, teardown_databases

        setup_databases_args = {}

        if not django_db_use_migrations:
            _disable_native_migrations()

        if django_db_keepdb and not django_db_createdb:
            setup_databases_args["keepdb"] = True

        with django_db_blocker.unblock():
            db_cfg = setup_databases(
                verbosity=request.config.option.verbose,
                interactive=False,
>               **setup_databases_args
            )

/usr/local/lib/python3.6/site-packages/pytest_django/fixtures.py:108:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.6/site-packages/django/test/utils.py:174: in setup_databases
    serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE', True),
/usr/local/lib/python3.6/site-packages/django/db/backends/base/creation.py:72: in create_test_db
    run_syncdb=True,
/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py:148: in call_command
    return command.execute(*args, **defaults)
/usr/local/lib/python3.6/site-packages/django/core/management/base.py:364: in execute
    output = self.handle(*args, **options)
/usr/local/lib/python3.6/site-packages/django/core/management/base.py:83: in wrapped
    res = handle_func(*args, **kwargs)
/usr/local/lib/python3.6/site-packages/django/core/management/commands/migrate.py:85: in handle
    connection.prepare_database()
/usr/local/lib/python3.6/site-packages/django/contrib/gis/db/backends/postgis/base.py:26: in prepare_database
    cursor.execute("CREATE EXTENSION IF NOT EXISTS postgis")
/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:67: in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:76: in _execute_with_wrappers
    return executor(sql, params, many, context)
/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
/usr/local/lib/python3.6/site-packages/django/db/utils.py:89: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <django.db.backends.utils.CursorWrapper object at 0x7f584efcdef0>, sql = 'CREATE EXTENSION IF NOT EXISTS postgis', params = None
ignored_wrapper_args = (False, {'connection': <django.contrib.gis.db.backends.postgis.base.DatabaseWrapper object at 0x7f586869a710>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7f584efcdef0>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
>               return self.cursor.execute(sql)
E               django.db.utils.OperationalError: could not open extension control file "/usr/share/postgresql/9.6/extension/postgis.control": No such file or directory

/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:82: OperationalError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================================================================== 1 error in 3.10s ======================================================================

[fgregg]

yes, I can run locally.

[hancush]

Got it, somehow had an old test database lurking around. Dropped it manually, resolved the problem.

[hancush]

Many thanks, @fgregg! LGTM.

[fgregg]

Thanks for the review.