Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SE-3540 [LX-1653] Fix create_transcripts_xml when resource_fs is WrapFS #267

Merged
merged 4 commits into from
Nov 10, 2020

Conversation

samuelallan72
Copy link

@samuelallan72 samuelallan72 commented Nov 5, 2020

resource_fs can be set to a WrapFS instance in some cases,
notably in override_export_fs, which is used when exporting content via
the olx_rest_api.

This bug was introduced in #258,
which in turn fixed other bugs.
So this commit preserves the behaviour from that PR,
while ensuring that the original static_file_dir logic is used when
a WrapFS is passed (ie. no _sub_dir attr).

An alternate option would be to add AttributeError to an except clause,
but it feels safer to be more specific and proactive in this case.

JIRA tickets: OSPR-5109

Merge deadline: ASAP; this is breaking exporting videos, which is used by our clients.

Test instructions:

  • ensure that the behaviour from update transcript directory structure during export #258 is retained. @DawoudSheraz, you may wish to check that this PR doesn't undo any of the fixes you did there. Please also check my comments for accuracy. :) (I don't know how to test the original PR)
  • use the olx export rest api to export a video (test with a video with and without transcripts)
  • verify that the olx is successfully exported

Reviewers:

`resource_fs` can be set to a WrapFS instance in some cases,
notably in override_export_fs, which is used when exporting content via
the olx_rest_api.

This bug was introduced in openedx#258,
which in turn fixed other bugs.
So this commit preserves the behaviour from that PR,
while ensuring that the original static_file_dir logic is used when
a WrapFS is passed (ie. no _sub_dir attr).

An alternate option would be to add AttributeError to an except clause,
but it feels safer to be more specific and proactive in this case.
@openedx-webhooks
Copy link

Thanks for the pull request, @swalladge! I've created OSPR-5109 to keep track of it in JIRA, where we prioritize reviews. Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket:

  • supporting documentation
  • Open edX discussion forum threads
  • timeline information ("this must be merged by XX date", and why that is)
  • partner information ("this is a course on edx.org")
  • any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

@gabor-boros
Copy link

gabor-boros commented Nov 5, 2020

👍 🎉

  • I tested this: checked if the changes fixes the issues for our client
  • I read through the code
  • I checked for accessibility issues NA
  • Includes documentation NA
  • I made sure any change in configuration variables is reflected in the corresponding client's configuration-secure repository. NA

PS: Could you please add tests and update the branch ("This branch is out-of-date with the base branch")?

@samuelallan72
Copy link
Author

@natabene this is ready for edX review. @DawoudSheraz may wish to look at this too since it relates to #258

@natabene
Copy link

natabene commented Nov 7, 2020

@swalladge Thank you for your contribution. @DawoudSheraz Would you like to review this?

@@ -964,7 +964,7 @@ def create_transcripts_xml(video_id, video_el, resource_fs, static_dir):
video_id (str): Video id of the video.
video_el (Element): lxml Element object
static_dir (str): The Directory to store transcript file.
resource_fs (SubFS): The file system to store transcripts.
resource_fs (SubFS|WrapFS): The file system to store transcripts.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A clarification of how this can be different will be helpful for future devs.

exported_xml = exported_metadata['xml']
self.assert_xml_equal(exported_xml, expected)

# Verify that no transcript is present in the XML.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for this as the unit test docstring is already explaining this.

@unpack
def test_basic_wrapfs(self, course_id, image):
"""
Test that video export works as expected, with wrapfs as sometimes used by edx-platform.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: move , after wrapfs and mention olx_rest_api after edx-platform to be more specific

Comment on lines 1195 to 1197
fs = WrapFS(MemoryFS())
fs.makedir('course')
fs.makedir('course/static') # Video XBlock requires this directory to exists, to put srt files etc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These lines can be converted into a helper function, like _setup_wrap_fs() to avoid code duplication.

Copy link
Contributor

@DawoudSheraz DawoudSheraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good overall. Just need to address a few nits before merging.

- explain SubFS vs WrapFS better
- helper function to set up a WrapFS for tests (code deduplication)
@samuelallan72
Copy link
Author

@DawoudSheraz thanks! I addressed your comments. Ready for another pass. 😄

@DawoudSheraz
Copy link
Contributor

@swalladge PR approved. If it is not waiting for another approval on your end, I will merge it. Please let me know. Thanks for your contribution.

@samuelallan72
Copy link
Author

@DawoudSheraz thanks! We have completed our internal review process (@gabor-boros approved above), so will be great to get this merged! 😄

@DawoudSheraz DawoudSheraz merged commit ca42f35 into openedx:master Nov 10, 2020
@openedx-webhooks
Copy link

@swalladge 🎉 Your pull request was merged!

Please take a moment to answer a two question survey so we can improve your experience in the future.

@DawoudSheraz
Copy link
Contributor

PR has been merged and a release tag https://github.com/edx/edx-val/releases/tag/1.4.4 has been created.

@samuelallan72 samuelallan72 deleted the samuel/fix-video-export branch November 11, 2020 06:57
@samuelallan72
Copy link
Author

Thanks @DawoudSheraz ! Could you let us know when this has been deployed to edX staging and production please? 😄

@DawoudSheraz
Copy link
Contributor

@swalladge Hi. This has reached production after the requirements update in https://github.com/edx/edx-platform/pull/25554. Thank You

@samuelallan72
Copy link
Author

@DawoudSheraz thanks, that's good to know. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged open-source-contribution PR author is not from Axim or 2U
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants