Repo Recommender

See Product Requirements Document on Google Docs.

The Repository Recommender system uses a Github-authenticated user's stars as machine learning features to recommend other repositories to follow, using low-rank matrix approximation.

Built With

• Languages: Python, SQL (PostGRES), JavaScript, HTML, CSS
• Frameworks: Flask, Jinja, React, Flask-SQLAlchemy
• W3.CSS
• Libraries:
  • PyGithub
  • Progress by @verigak
  • SciPy SVDS, pandas, numpy
  • d3

Key algorithms/methods

• Low-rank matrix approximation
  • Singular Value Decomposition

File structure

.
├── api_utils.py             # Helper functions interfacing with api
├── config.py                # Configuration variables
├── db_utils.py              # Helper functions interfacing with the database
├── experiments.ipynb        # Jupyter NB including SVD tests
├── model.py                 # Flask-SQLAlchemy classes for the data model
├── requirements.txt         # Defines requirements
├── rec.py                   # Recommender system functions
├── server.py                # Flask routes
├── test_db_utils.py         # Tests for db_utils.py
├── test_model.py            # Tests for model.py
├── test_rec.py              # Tests for rec.py
├── test_server.py           # Tests for server.py and front-end
├── test_utils.py            # Tests for utils.py
├── timelog.py               # Decorator for logging
├── update_pkey_seqs.py      # Script by Katie Byers to introspect DB & set autoincrementing primary keys
├── utils.py                 # Helper methods for server.py
│
├── static
│   ├── graph.js             # d3 for graph on homepage
│   ├── recs.jsx             # AJAX requests and functions to render React components
│   ├── repo.jsx             # React components for displaying repositories
│   └── style.css            # CSS
│
└── templates
    ├── base.html            # Template (includes navbar, header, & footer)
    ├── home.html            # Homepage
    ├── repo_recs.html       # Repo recommendations rendered with React components
    └── user_info.html       # Details about a user and their repositories

TODO:

• Plan features for 2.0:
  • Add AJAX to follow users
  • Write route to show stars of a user
  • Add like/dislike feature to "Like" a repo without starring
    • I.e., "see more like this" / "see less like this"
  • Write API requests instead of using PyGithub?
  • Build queue table and handlers instead of crawling recursively
  • Expand async calls to dynamically increase crawl breadth on login

Resources used:.

• SciPy Sparse Single Value Decomposition
• Matrix Factorization for Movie Recommendations in Python
• How the Facebook content placeholder works
• d3

Acknowledgements

• Katie Byers @lobsterkatie — Wrote update_pkey_seqs.py

Author

• Sarai Rosenberg @Saraislet — Software engineer and mathematician, looking for opportunities