feat: implement toggles to use experimental images

[kdmccormick]

Description

Previously, all Open edX services and some
supporting services were hard-coded to use docker images
from the edxops DockerHub org. This commit parameterizes those
images under Makefile variables. For example, registrar
now specifies ${_REGISTRAR_IMAGE} instead of
edxops/registrar. By default, all the image variables
are still set to the same old images from
the edxops DockerHub org.

Furthermore, two toggles were introduced:

• USE_EXPERIMENTAL_EDX_PLATFORM_IMAGES
• USE_EXPERIMENTAL_CREDENTIALS_IMAGE
  allowing users to test out the new, optimized
  openedx/ images for LMS, CMS, and Credentials.
  Provisioning tests will be run for these three
  experimental images as part of PR builds.

Our intent is to test and eventually switch
all services over to the new openedx images,
which are built using more idiomatic docker
practices and should be smaller and more resource
efficient.

Co-Authored-By: @cmuniz

Test instructions

Ensure: by default, the old edxops images are still used

On master: Starting with a provisioned devstack, bring containers down.

Switch to this branch. Confirm that all backend services, as well as the supporting services affected by this PR, can be brought up.

$ make dev.up.studio+credentials+ecommerce+edx_notes_api+registrar+xqueue
$ make dev.up.lms+discovery+devpi+forum  # Should be a no-op, as these are deps of studio et al.
$ make dev.up.analyticsapi+insights
$ make dev.up.chrome+firefox

Ensure that there now exist services running thirteen distinct edxops/... images.

$ docker ps --filter name="edx.devstack." --format "table {{.Names}}\t{{.Image}}\t{{.Status}}
NAMES                           IMAGE                         STATUS
edx.devstack.firefox            edxops/firefox:latest         Up 23 minutes
edx.devstack.chrome             edxops/chrome:latest          Up 23 minutes
edx.devstack.registrar          edxops/registrar:latest       Up 24 minutes
edx.devstack.xqueue             edxops/xqueue:latest          Up 24 minutes
edx.devstack.redis              redis:2.8                     Up 24 minutes
edx.devstack.insights           edxops/insights:latest        Up 25 minutes
edx.devstack.analyticsapi       edxops/analytics_api:latest   Up 25 minutes
edx.devstack.edxnotesapi        edxops/notes:latest           Up 24 minutes
edx.devstack.credentials        edxops/credentials:latest     Up 24 minutes
edx.devstack.ecommerce          edxops/ecommerce:latest       Up 24 minutes
edx.devstack.studio             edxops/edxapp:latest          Up 24 minutes
edx.devstack.lms                edxops/edxapp:latest          Up 25 minutes
edx.devstack.discovery          edxops/discovery:latest       Up 25 minutes
edx.devstack.forum              edxops/forum:latest           Up 25 minutes
edx.devstack.elasticsearch710   elasticsearch:7.10.1          Up 25 minutes
edx.devstack.devpi              edxops/devpi:latest           Up 25 minutes
edx.devstack.memcached          memcached:1.5.10-alpine       Up 25 minutes
edx.devstack.mongo              mongo:4.2.14                  Up 25 minutes
edx.devstack.mysql57            mysql:5.7                     Up 25 minutes

And ensure that they can be brought down:

$ make dev.down
$ docker ps --filter name="edx.devstack." --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
NAMES     IMAGE     STATUS

Ensure: LMS, Studio and Credentials can be toggled to use the experimental openedx images

Bring containers down (make dev.down). Then, create or edit options.local.mk in the root of your devstack repo. Add:

USE_EXPERIMENTAL_EDX_PLATFORM_IMAGES = 1
USE_EXPERIMENTAL_CREDENTIAL_IMAGE = 1

Bring up LMS, Studio and Credentials:

make dev.up.lms+studio+credentials

and ensure the openedx/ images (or kdmccormick96/ images, while this PR is still a draft) are used:

$ docker ps --filter name="edx.devstack." --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
NAMES                            IMAGE                             STATUS
edx.devstack2.studio             kdmccormick96/studio-dev:latest   Up 20 seconds
edx.devstack2.lms                kdmccormick96/lms-dev:latest      Up 21 seconds
edx.devstack2.discovery          edxops/discovery:latest           Up 21 seconds
edx.devstack2.forum              edxops/forum:latest               Up 21 seconds
edx.devstack2.mysql57            mysql:5.7                         Up 21 seconds
edx.devstack2.devpi              edxops/devpi:latest               Up 22 seconds
edx.devstack2.memcached          memcached:1.5.10-alpine           Up 22 seconds
edx.devstack2.mongo              mongo:4.2.14                      Up 22 seconds
edx.devstack2.elasticsearch710   elasticsearch:7.10.1              Up 22 seconds
~/openedx/devstack 🍀 

Finally, log into LMS and play around. For example, take the demo course, edit the course in Studio, publish it, export and re-import it, upload an image, view credentials, visit all three admin panels, etc.

Supporting Information

Begins addressing "Move off of edxops DockerHub org images".

Other information

Blocked by https://github.com/edx/devstack/pull/874

Tentative rollout plan

• Merge the credentials PR and let a new image be pushed to DockerHub.
• Merge the edx-platform PR and let new images be pushed to DockerHub.
• Merge this PR.
• Make a forum post informing folks that USE_EXPERIMENTAL_EDX_PLATFORM_IMAGES and USE_EXPERIMENTAL_CREDENTIALS_IMAGE are available for testing. Inform edX's Arch-BOM (~ developer experience) team so that they can circulate the information within edX
• Wait and collect feedback. Make bug fixes as necessary.
• Make the openedx/lms-dev, openedx/studio-dev, and openedx/credentials-dev images the default.
• Repeat the same process with the other services, probably in batches, since getting edx-platform working will probably expose most of the kinks of the new images early, allowing us to roll out more aggressively in the later stages.
• For the services that don't have experimental images: Write Dockerfiles for new images and GitHub actions to push them to the openedx DockerHub org. For third party services, try to simply switch over to the official image. If needed, repeat the same process of trying them out behind toggles and eventually making them the default.
• Once no more edxops images are used, the edX is safe to stop pushing images to the edxops DockerHub org.
• The old images in edxops should remain available for one Open edX release cycle. After that, edX is safe to decommission the edxops DockerHub org entirely if they want to.

Why is the plan "tentative?"

Because it reflects only one of the three possible approaches to the greater edxops issue. Depending on how things evolve, we may switch all images over, switch only some of them over, or abandon them entirely in favor of Tutor's images. Regardless, I think this change is worthwhile, because it gives us an opportunity to see how Ansible and non-Ansible images compare in the context of devstack.

It is part of the linked issue's acceptance criteria to clean all of this up in the event that the experimental images and/or devstack are abandoned.

[kdmccormick]

@Carlos-Muniz

[syedimranhassan]

@kdmccormick The GH action added in this PR is showing up in other PR too and not triggering because it's not merged yet.
Here is my PR 871 and this GH action not allowing me to merge my PR.

[kdmccormick]

My bad @syedimranhassan -- I just marked that check as non-required for now.

[kdmccormick]

In light of the kickoff of work to replace devstack with tutor, I don't feel that it's worth our while any more to review this and roll it out. We can always reopen this later if it becomes useful again.