Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy loading #77

Merged
merged 13 commits into from
Dec 23, 2019
Merged

Lazy loading #77

merged 13 commits into from
Dec 23, 2019

Conversation

psvenk
Copy link
Member

@psvenk psvenk commented Dec 19, 2019

Closes #56. This PR moves the schedule and PDF scrapers out of the main scrape_student function in scrape.js (which is called from the /data endpoint) and keeps separate endpoints for them, /schedule and /pdf. home.html has been updated to perform separate AJAX requests for the schedule and the PDF files (reports), with items only being downloaded as needed. In the case of the schedule, the data are downloaded as soon as the grades have been downloaded or when the user clicks on the "Schedule" tab, so that the clock in the upper left-hand corner shows the current period without the user needing to click on the "Schedule" tab to load the schedule data. In the case of the reports, the data are downloaded only when the user clicks on the "Reports" tab.

From my (somewhat limited) testing, this code causes no problems. Please let me know if any of you come across any problems with the new code.

Instead of using Tabulator to perform the AJAX request that gets the
data to populate tableData, perform the AJAX request separately and
update the table. This will make it easier to perform separate AJAX
requests as each term (Q1, Q2, Q3, Q4) is requested.
1) The schedule-scraping has been commented out of scrape_student in
scrape.js, instead exporting scrape_schedule directly.
2) A new endpoint, '/schedule', has been declared in serve.js, from which
the schedule can be retrieved as JSON (because it is no longer included
in the JSON file from /data).
3) The JavaScript in home.html now performs the schedule request separately
with a separate callback function, and clock.js no longer assumes
that tableData.schedule must exist if tableData exists.

A similar treatment will be given to other components that are scraped
until the loading has become sufficiently modular (lazy loading)
to make it feasible to retrieve data from past terms and to extend
Aspine in other ways in the future.
Same as previous commit, but with the PDF files (Reports tab).
…iting it. This fixes the problem where tableData already contains information, for example pdf_files, which are then erased when /data does 'tableData = response'. This would occur when a user immediately navigates to the 'Reports' page before /data has the chance to complete.
…hat /data returns while the user is on a different tab
@Ruborcalor
Copy link
Member

Wow awesome work! I took care of two small problems that arose from the possibility for /pdf to return before /data. I then added a loader to indicate to users that lazy loading is going on (it could be a confusing otherwise). If you agree with these changes, I think it is ready to merge.

@psvenk
Copy link
Member Author

psvenk commented Dec 19, 2019

@Ruborcalor These changes look good, thank you for catching those oversights and making fixes. I did notice that the loader was gone and intended to fix it, but for some reason I forgot to do that before creating this PR. I'll just ask @kdk1616 if he has any input before merging.

@kdk1616 kdk1616 merged commit 6b36901 into master Dec 23, 2019
@psvenk psvenk deleted the lazy_loading branch January 11, 2020 22:43
psvenk added a commit that referenced this pull request Apr 2, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 15, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 15, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 15, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 20, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 20, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 20, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 20, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 20, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 21, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 26, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 26, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 26, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request Apr 30, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
psvenk added a commit that referenced this pull request May 2, 2020
These options were added in <#62>
as a server-side import/export feature, but they no longer work as
intended since lazy loading was added in
<#77>.
In any case, this is superseded by the client-side import/export feature
and the lite version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Load page while student data continues to arrive.
3 participants