Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Facility command and task improvements #7187

Merged

Conversation

bjester
Copy link
Member

@bjester bjester commented Jun 25, 2020

Summary

  • Adds new NamespacedCacheProxy to enable efficient clearing of cache keys related to Facility dataset data
  • Clears namespaced dataset cache after deleting a facility and before syncing
  • Updates the default chunk size for facility tasks to be 200 and makes them not cancellable
  • Updates sync command to not aggregate progress tracking total
  • Updates frontend to make progress indeterminate for certain facility task stages
  • Updates morango to 0.5.1 which has deserialization improvements

Reviewer guidance

  • From the CLI:
    1. Set up another Kolibri with some real facility data
    2. On a second Kolibri instance, with fresh database, issue the kolibri manage sync command pulling from the first
    3. Verify facility data in Kolibri
    4. On the second instance, issue the kolibri manage deletefacility command deleting the facility
    5. On a second instance, issue the kolibri manage sync command pulling again to pull the same facility
    6. No warnings or errors should occur
  • From the frontend:
    1. Initiate a sync with the Kolibri bet server
    2. Verify all facility import/sync tasks show indeterminate progress when not sending or receiving data

References

learningequality/morango#84
#7127


Contributor Checklist

PR process:

  • PR has the correct target branch and milestone
  • PR has 'needs review' or 'work-in-progress' label
  • If PR is ready for review, a reviewer has been added. (Don't use 'Assignees')
  • If this is an important user-facing change, PR or related issue has a 'changelog' label
  • If this includes an internal dependency change, a link to the diff is provided

Testing:

  • Contributor has fully tested the PR manually
  • If there are any front-end changes, before/after screenshots are included
  • Critical user journeys are covered by Gherkin stories
  • Critical and brittle code paths are covered by unit tests

Reviewer Checklist

  • Automated test coverage is satisfactory
  • PR is fully functional
  • PR has been tested for accessibility regressions
  • External dependency files were updated if necessary (yarn and pip)
  • Documentation is updated
  • Contributor is in AUTHORS.md

@bjester bjester added this to the 0.14.0 milestone Jun 25, 2020
_namespace_locks = {}


class NamespacedCacheProxy(BaseCache):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rtibbles I created this proxy class to generically handle tracking cache keys and clearing them.

@bjester bjester added DEV: backend Python, databases, networking, filesystem... DEV: frontend work-in-progress Not ready for review labels Jun 25, 2020
@codecov
Copy link

codecov bot commented Jun 25, 2020

Codecov Report

Merging #7187 into release-v0.14.x will increase coverage by 0.00%.
The diff coverage is 75.47%.

Impacted Files Coverage Δ
...ri/core/auth/management/commands/deletefacility.py 0.00% <0.00%> (ø)
kolibri/core/auth/management/commands/sync.py 0.00% <0.00%> (ø)
kolibri/core/tasks/api.py 44.35% <ø> (ø)
...ets/src/views/FacilitiesPage/FacilityTaskPanel.vue 0.00% <0.00%> (ø)
kolibri/core/utils/cache.py 92.94% <94.73%> (+15.16%) ⬆️
kolibri/core/auth/management/utils.py 48.14% <100.00%> (ø)
kolibri/core/auth/models.py 91.35% <100.00%> (+0.02%) ⬆️
kolibri/core/tasks/utils.py 92.59% <100.00%> (-0.52%) ⬇️
...i/plugins/device/assets/src/views/syncTaskUtils.js 64.70% <100.00%> (+0.70%) ⬆️
kolibri/core/assets/src/core-app/client.js 10.90% <0.00%> (-1.10%) ⬇️
... and 21 more

@bjester bjester added TODO: needs review Waiting for review and removed work-in-progress Not ready for review labels Jun 25, 2020
@bjester bjester marked this pull request as ready for review June 25, 2020 17:52
@bjester bjester requested review from jonboiser and rtibbles June 25, 2020 17:52
Copy link
Member

@rtibbles rtibbles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two potential issues with the new namespaced cache class.

kolibri/core/utils/cache.py Outdated Show resolved Hide resolved
kolibri/core/utils/cache.py Outdated Show resolved Hide resolved
@bjester bjester requested a review from rtibbles June 25, 2020 18:29
@jonboiser jonboiser merged commit a35eb33 into learningequality:release-v0.14.x Jun 25, 2020
@jonboiser jonboiser removed the TODO: needs review Waiting for review label Aug 5, 2020
@bjester bjester deleted the facility-cmd-improvements branch July 15, 2021 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DEV: backend Python, databases, networking, filesystem...
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants