Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🛑 Stop using MyPurdue pages that require auth #57

Merged
merged 1 commit into from
Nov 12, 2023

Conversation

haydenmc
Copy link
Member

Recent changes to Purdue's authentication processes have made scraping MyPurdue pages that require authorization not feasible.

This change updates the scraping process to avoid these pages and resort to workarounds (such as manually defined data mapping tables) or omitting data entirely (such as enrollment information).

See issues #54, #55, #56 for more information on the changes to available data.

@haydenmc haydenmc merged commit 11dcdb7 into main Nov 12, 2023
1 check passed
@haydenmc haydenmc deleted the remove-auth-requirement branch November 12, 2023 08:02
haydenmc added a commit that referenced this pull request Nov 14, 2023
This change resolves remaining issues introduced by #57 and #53.

The schema has had some slight adjustments made necessary by changes to
MyPurdue as well as breaking changes to the database library detailed
[here](https://www.roji.org/postgresql-dotnet-timestamp-mapping):

- `Capacity`, `Enrolled`, `RemainingSpace`, `WaitListCapacity`,
`WaitListCount`, and `WaitListSpace` are **no longer available** on
`Section`. Recent changes to MyPurdue have made it not feasible to
synchronize these values. This is tracked in #56 for a potential future
solution.
- `StartDate`/`EndDate` on `Meeting`, `Section`, and `Term` is now
stored as
[`DateOnly`](https://learn.microsoft.com/en-us/dotnet/api/system.dateonly)
objects, and may be null
- `StartTime` on `Meeting` is stored as a
[`TimeOnly`](https://learn.microsoft.com/en-us/dotnet/api/system.timeonly)
object, and may be null
- All dates and times are local time with no offset or time zone
information.

Some offline conversion tables were introduced for pieces of information
CatalogSync no longer has access to, such as campus/building short
codes. Tracked in #54 and #55 for potential future solutions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant