[WIP] Update Models and Views for Councilmatic 2.5

[fgregg]

Overview

This PR updates the models and views for councilmatic 2.5, as well as search and management commands.

It is nearly feature complete (modulo bugs), but not completely. I want to discuss with @hancush what we want to bring into this PR versus what we should wait on to make things more reviewable.

Notes

This PR depends on changes to the scrapers and django-councilmatic. I believe they can be reviewed separately

• Upgrades to the LA Metro scraper opencivicdata/scrapers-us-municipal#299
• Adjust models and views for LA Metro Councilmatic upgrade datamade/django-councilmatic#260

Also, we are bit out of synch with LA metro master, we should discuss how we want to handle that.

Testing Instructions

Move councilmatic/settings_deployment.py.example to councilmatic/settings_deployment.py and add a GOOGLE_API_KEY setting so that the maps will work.

docker-compose up scrapers
docker-compose run --rm -e DJANGO_MANAGEPY_MIGRATE=off app python manage.py migrate
docker-compose run --rm -e DJANGO_MANAGEPY_MIGRATE=off app python manage.py compile_pdfs
docker-compose run --rm -e DJANGO_MANAGEPY_MIGRATE=off app python manage.py convert_attachment_text
docker-compose run --rm -e DJANGO_MANAGEPY_MIGRATE=off app python manage.py update_index --batch-size=100
docker-compose up

visit http://localhost:8011/

The local site should be the same as the production site with a few exceptions:

• headshots
• biographies of people
• descriptions of committees

Headshots

We have changed the way we handle head shots. Now, you upload headshots through the admin interface.

1. Create and admin user docker-compose run --rm -e DJANGO_MANAGEPY_MIGRATE=off app python manage.py createsuperuser
2. Login into the admin interface at http://localhost:8011/ and update an person with a url for their headshot

3. docker-compose run --rm -e DJANGO_MANAGEPY_MIGRATE=off app python manage.py update_headshots

Descriptions of committees and people

Currently, the settings_jurisdiction.py contains descriptions of committees and biographies of people. These get associated with the individual views by keying off of slugs. As the slugs for people and committees depend on their OCD id the slugs will be different in your local development version.

To test that the current approach works, you can edit the settings_jurisdiction.py and update the slugs to line up with your local database.

Going forward, I would recommend making these types of data attributes of the appropriate models, which will be updated and edited in an admin interface. Then we can pull the data for people and orgs as fixtures and load them as part of an initial ETL. (this can be addressed in the data migration scope item).

Issue Tracking

• Restore test event filtering
• Handle missing histories
• Factor out annotations
• Update event tag

Some tests won't pass until I get these sorted out, but I wanted to huddle with @hancush before pushing forward.

[fgregg]

this looks like a work around a data import issue, and I think it will not be necessary. I'll keep track of it though.

[fgregg]

didn't need this model.

[fgregg]

psycopg2 is properly a dependency of django-councilmatic.

[fgregg]

this stuff allows the django toolbar to work with docker.

[fgregg]

this has our data table sort by last name of the person.

[fgregg]

this causes a fatal error in django 2.1

[fgregg]

this monster has been broken up.