On-the-fly ISQ datasets for UNF courses
Scrapes historical course data from UNF into a format suitable for visualization or analysis. Given a course code or professor's N#, this script will generate a CSV file from (and also cache in a local SQLite database) the following data:
- ISQ rating distributions
- Grade distributions
- Course schedules
This project requires Go. Their website provides installers. Mac and Linux users can alternatively use their respective package managers. Verify Go is properly installed by running go env GOPATH
.
# Install or update
$ go install github.com/rothso/isqool@latest
# Pull the data for Computer Science 1
$ isqool COP2220
# Pull the data for Ken Martin
$ isqool N00009873
Explore the CSV outputs using Tableau or online with RAW. For a deeper data analysis, try Python or R. The SQLite database can also be queried with SQL. Samples of the outputted datasets can be found in the sample
folder.
ISQool can also scrape and sync an entire department's course data to BigQuery, for more intense data analysis needs. Application Default Credentials for Google Cloud must be set up. Currently, this project hardcoded to write to the isqool
BigQuery dataset in the syllabank-4e5b9
project.
# Sync departmental data and save to BigQuery
$ isqool sync 6502 "Fall 2023"
# Dry run: Run without modifying the database
$ isqool sync 6502 "Fall 2023" --dry-run
# Debug mode: Output a CSV instead of writing to the database
$ isqool sync 6502 "Fall 2023" --debug