i18n tooling using Transifex

[OmarIthawi]

Note: @oppia don't have the time to review the upstream pull request (oppia#2), hence they can't get it merged. Opening it here as an intermediate solution. I've also switch to Transifex.

---

We'd like this XBlock to be translated to Japanese and other languages, I've also supported JavaScript translations via
django-statici18n.

Before you continue, take a look at edx/i18n-tools and edx/cookiecutter-django-app.

Summary of the Changes

• Added a Makefile to help manage the translations. The file is copied and modified from: edx/cookiecutter-django-app
• Created a locale link to oppia/translations to allow the i18n_tools find the po/mo files
• Added oppia/translations/config.yaml which works as the configuration file of the i18n_tools command
• Added requirements-dev.txt to pin development packages. Used for now only for translations.

TODO

• Support JavaScript translations as textjs.po
• Document how to update the strings properly from Transifex
• Manually test that the the dummy Esperanto translation is
• After merge: Inform @oppia to update their README.md to point to this XBlock

Cheese

Studio

LMS

[OmarIthawi]

@melvinsoft do you mind taking a look here?

[melvinsoft]

@OmarIthawi Thanks for creating the PR. The code looks amazing, but following your advice I've been trying to run the XBlock in my devstack.

I installed your feature branch and the first error I have is the following:

Traceback (most recent call last):
  File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/views/preview.py", line 304, in get_preview_fragment
    fragment = module.render(preview_view, context)
  File "/edx/app/edxapp/venvs/edxapp/src/xblock/xblock/core.py", line 193, in render
    return self.runtime.render(self, view, context)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1889, in render
    return self.__getattr__('render')(block, view_name, context)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1297, in render
    return super(MetricsMixin, self).render(block, view_name, context=context)
  File "/edx/app/edxapp/venvs/edxapp/src/xblock/xblock/runtime.py", line 806, in render
    frag = view_fn(context)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/vertical_block.py", line 97, in author_view
    self.render_children(context, fragment, can_reorder=True, can_add=True)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/studio_editable.py", line 26, in render_children
    rendered_child = child.render(StudioEditableModule.get_preview_view_name(child), context)
  File "/edx/app/edxapp/venvs/edxapp/src/xblock/xblock/core.py", line 193, in render
    return self.runtime.render(self, view, context)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1889, in render
    return self.__getattr__('render')(block, view_name, context)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1297, in render
    return super(MetricsMixin, self).render(block, view_name, context=context)
  File "/edx/app/edxapp/venvs/edxapp/src/xblock/xblock/runtime.py", line 806, in render
    frag = view_fn(context)
  File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/oppia/oppia.py", line 85, in student_view
    'oppiaid': self.oppiaid,
  File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/oppia/oppia.py", line 67, in render_template
    i18n_service=self.runtime.service(self, "i18n"),
TypeError: render_django_template() got an unexpected keyword argument 'i18n_service'

I think you're working on a newer version of XBlock and XBlock-utils.

I tried upgrading the XBlock and XBlock-utils versions manually and I got a new error:

Traceback (most recent call last):
  File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/views/preview.py", line 304, in get_preview_fragment
    fragment = module.render(preview_view, context)
  File "/edx/app/edxapp/venvs/edxapp/src/xblock/xblock/core.py", line 202, in render
    return self.runtime.render(self, view, context)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1889, in render
    return self.__getattr__('render')(block, view_name, context)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1297, in render
    return super(MetricsMixin, self).render(block, view_name, context=context)
  File "/edx/app/edxapp/venvs/edxapp/src/xblock/xblock/runtime.py", line 810, in render
    frag = view_fn(context)
  File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/oppia/oppia.py", line 85, in student_view
    'oppiaid': self.oppiaid,
  File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/oppia/oppia.py", line 67, in render_template
    i18n_service=self.runtime.service(self, "i18n"),
  File "/edx/app/edxapp/venvs/edxapp/src/xblock-utils/xblockutils/resources.py", line 77, in render_django_template
    template_str = self.load_unicode(template_path)
  File "/edx/app/edxapp/venvs/edxapp/src/xblock-utils/xblockutils/resources.py", line 47, in load_unicode
    resource_content = pkg_resources.resource_string(self.module_name, resource_path)
  File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1215, in resource_string
    self, resource_name
  File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1457, in get_resource_string
    return self._get(self._fn(self.module_path, resource_name))
  File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1577, in _get
    with open(path, 'rb') as stream:
IOError: [Errno 2] No such file or directory: '/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/oppia/templates/oppia.html'

Can you point me to some instructions to install it? Or if we need to upgrade the dependencies version (we're in still on Ficus) we will need to plan that, to make sure others XBlocks still work.

Thanks!

[OmarIthawi]

@melvinsoft You'd need the same version of XBlock in Ficus:

git+https://github.com/edx/xblock-utils.git@v1.0.3#egg=xblock-utils==1.0.3

and this specific XBlock utils:

• Update XBlock Utils to 1.1.1 for better i18n support in XBlock edx-platform#265

Please also make sure you're using the same branch of this PR omar/tx-i18n.

I've just tried it and it works fine, please try again.

[melvinsoft]

@OmarIthawi Thank for pointing me to the right versions.

I think I've found the problem, the templates folder didn't exist in the installation. I added the templates and translations folders int he package data and re installed, and started to work.

package_data=package_data("oppia", ["static", "templates", "public", "translations"]),

But then I found another problem that I commented in the code inline, when I change the language I get this error:

[Errno 2] No such file or directory: '/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/oppia/public/js/translations/en/textjs.js'

Pretty similar to the previous one. Looks like the public folder doesn't exists in the XBlock, and isn't in the package_data either.

Please advise.

[melvinsoft]

We don't have the public folder in the XBlock, should it be created on the fly in the installation? or this default should point to static?

[OmarIthawi]

Thanks @melvinsoft, I'll take a look and let you know.

[OmarIthawi]

@melvinsoft Please take another look. I've fixed the data and public issue.

[melvinsoft]

@OmarIthawi Thanks for addressing the change! It's working now. I'm testing the Makefile commands and I've found a couple of issues. All the other commands work like a charm.

pull_translations

If you run make pull_translations without a transifex file, it's just return this exception:

Pulling conf/locale/config.yaml:locales from Transifex...
Exception: No user credentials found for host https://www.transifex.com. Edit ~/.transifexrc and add the appropriate info in there.
Traceback (most recent call last):
  ...
make: *** [pull_translations] Error 1

Do you think we should add some check in the Make command and return a proper error message and some instructions if the file isn't there, instead of returning the exception directly?

make test

This command is throwing an error, here is the stack trace I'm getting.

find . -name '__pycache__' -exec rm -rf {} +
find . -name '*.pyc' -exec rm -f {} +
find . -name '*.pyo' -exec rm -f {} +
find . -name '*~' -exec rm -f {} +
find . -name '*.pot' -exec rm -f {} +
rm -f locale/en/LC_MESSAGES/*
find oppia/translations -name djangojs.mo -exec rm -f {} +
find oppia/translations -name djangojs.po -exec rm -f {} +
find oppia/translations -name textjs.mo -exec rm -f {} +
rm -fr build/
rm -fr dist/
rm -fr *.egg-info
python run_tests.py
python: can't open file 'run_tests.py': [Errno 2] No such file or directory
make: *** [test] Error 2

make quality

echo 'TODO: Refactor quality checks from .travis.yml into this Makefile'
TODO: Refactor quality checks from .travis.yml into this Makefile
exit 1
make: *** [quality] Error 1

[OmarIthawi]

@melvinsoft I've removed quality and test Make rules.

Regarding the pull_translations, please take a this section of the README.

[melvinsoft]

@OmarIthawi Thanks for addressing all the changes. Great work here!

[OmarIthawi]

Thanks @melvinsoft! I'll merge it.

[OmarIthawi]

@seanlip FYI.