Skip to content

Commit

Permalink
Dockerize (#1365)
Browse files Browse the repository at this point in the history
Provide a docker development and testing environment

* CI (Travis) now runs tests via the same docker environment that is available to developers.
* A simple Makefile has been added to make getting started easier.
    * `make dev` will standup the development environment.
    * `make test` will standup the development environment and also run the test suite.
    * `make clean` will remove the development environment.
  • Loading branch information
andymccurdy authored Jul 20, 2020
2 parents 142a44a + 1f988d8 commit 75a2cfe
Show file tree
Hide file tree
Showing 45 changed files with 636 additions and 408 deletions.
2 changes: 2 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[run]
source = redis
3 changes: 3 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
**/__pycache__
**/*.pyc
.tox
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,6 @@ vagrant/.vagrant
.eggs
.idea
.coverage
env
venv
coverage.xml
52 changes: 12 additions & 40 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,42 +1,14 @@
language: python
cache: pip
matrix:
include:
- env: TOXENV=flake8
- python: 2.7
env: TOXENV=py27-plain
- python: 2.7
env: TOXENV=py27-hiredis
- python: 3.5
env: TOXENV=py35-plain
- python: 3.5
env: TOXENV=py35-hiredis
- python: 3.6
env: TOXENV=py36-plain
- python: 3.6
env: TOXENV=py36-hiredis
- python: 3.7
env: TOXENV=py37-plain
- python: 3.7
env: TOXENV=py37-hiredis
- python: 3.8
env: TOXENV=py38-plain
- python: 3.8
env: TOXENV=py38-hiredis
- python: pypy
env: TOXENV=pypy-plain
- python: pypy
env: TOXENV=pypy-hiredis
- python: pypy3
env: TOXENV=pypy3-plain
- python: pypy3
env: TOXENV=pypy3-hiredis
env:
- DOCKER_COMPOSE_VERSION=1.26.2

before_install:
- wget https://github.com/antirez/redis/archive/6.0.5.tar.gz && mkdir redis_install && tar -xvzf 6.0.5.tar.gz -C redis_install && cd redis_install/redis-6.0.5 && make && src/redis-server --daemonize yes && cd ../..
- redis-cli info
install:
- pip install codecov tox
- sudo rm /usr/local/bin/docker-compose
- curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin

services:
- docker

script:
- tox
after_success:
- "if [[ $TOXENV != 'flake8' ]]; then codecov; fi"
- make test
3 changes: 3 additions & 0 deletions CHANGES
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
* (in development)
* Provide a development and testing environment via docker. Thanks
@abrookins. #1365
* 3.5.3 (June 1, 2020)
* Restore try/except clauses to __del__ methods. These will be removed
in 4.0 when more explicit resource management if enforced. #1339
Expand Down
128 changes: 128 additions & 0 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
Contributing
============

Introduction
------------

First off, thank you for considering contributing to redis-py. We value community contributions!

Contributions We Need
----------------------

You may already know what you want to contribute -- a fix for a bug you encountered, or a new feature your team wants to use.

If you don't know what to contribute, keep an open mind! Improving documentation, bug triaging, and writing tutorials are all examples of helpful contributions that mean less work for you.

Your First Contribution
-----------------------
Unsure where to begin contributing? You can start by looking through `help-wanted issues <https://github.com/andymccurdy/redis-py/issues?q=is%3Aopen+is%3Aissue+label%3ahelp-wanted>`_.

Never contributed to open source before? Here are a couple of friendly tutorials:

- http://makeapullrequest.com/
- http://www.firsttimersonly.com/

Getting Started
---------------

Here's how to get started with your code contribution:

1. Create your own fork of redis-py
2. Do the changes in your fork
3. If you need a development environment, run ``make dev``
4. While developing, make sure the tests pass by running ``make test``
5. If you like the change and think the project could use it, send a pull request

The Development Environment
---------------------------

Running ``make dev`` will create a Docker-based development environment that starts the following containers:

* A master Redis node
* A slave Redis node
* Three sentinel Redis nodes
* A test container

The slave is a replica of the master node, using the `leader-follower replication <https://redis.io/topics/replication>`_ feature.

The sentinels monitor the master node in a `sentinel high-availability configuration <https://redis.io/topics/sentinel>`_.

Meanwhile, the `test` container hosts the code from your checkout of ``redis-py`` and allows running tests against many Python versions.

Docker Tips
^^^^^^^^^^^

Following are a few tips that can help you work with the Docker-based development environment.

To get a bash shell inside of a container:

``$ docker-compose run <service> /bin/bash``

**Note**: The term "service" refers to the "services" defined in the ``docker-compose.yml`` file: "master", "slave", "sentinel_1", "sentinel_2", "sentinel_3", "test".

Containers run a minimal Debian image that probably lacks tools you want to use. To install packages, first get a bash session (see previous tip) and then run:

``$ apt update && apt install <package>``

You can see the combined logging output of all containers like this:

``$ docker-compose logs``

The command `make test` runs all tests in all tested Python environments. To run the tests in a single environment, like Python 3.6, use a command like this:

``$ docker-compose run test tox -e py36 -- --redis-url=redis://master:6379/9``

Here, the flag ``-e py36`` runs tests against the Python 3.6 tox environment. And note from the example that whenever you run tests like this, instead of using `make test`, you need to pass ``-- --redis-url=redis://master:6379/9``. This points the tests at the "master" container.

Our test suite uses ``pytest``. You can run a specific test suite against a specific Python version like this:

``$ docker-compose run test tox -e py36 -- --redis-url=redis://master:6379/9 tests/test_commands.py``

Troubleshooting
^^^^^^^^^^^^^^^
If you get any errors when running ``make dev`` or ``make test``, make sure that you
are using supported versions of Docker and docker-compose.

The included Dockerfiles and docker-compose.yml file work with the following
versions of Docker and docker-compose:

* Docker 19.03.12
* docker-compose 1.26.2

How to Report a Bug
-------------------

Security Vulnerabilities
^^^^^^^^^^^^^^^^^^^^^^^^

**NOTE**: If you find a security vulnerability, do NOT open an issue. Email Andy McCurdy (sedrik@gmail.com) instead.

In order to determine whether you are dealing with a security issue, ask yourself these two questions:

* Can I access something that's not mine, or something I shouldn't have access to?
* Can I disable something for other people?

If the answer to either of those two questions are "yes", then you're probably dealing with a security issue. Note that even if you answer "no" to both questions, you may still be dealing with a security issue, so if you're unsure, just email Andy at sedrik@gmail.com.

Everything Else
^^^^^^^^^^^^^^^

When filing an issue, make sure to answer these five questions:

1. What version of redis-py are you using?
2. What version of redis are you using?
3. What did you do?
4. What did you expect to see?
5. What did you see instead?

How to Suggest a Feature or Enhancement
---------------------------------------

If you'd like to contribute a new feature, make sure you check our issue list to see if someone has already proposed it. Work may already be under way on the feature you want -- or we may have rejected a feature like it already.

If you don't see anything, open a new issue that describes the feature you would like and how it should work.

Code Review Process
-------------------

The core team looks at Pull Requests on a regular basis. We will give feedback as as soon as possible. After feedback, we expect a response within two weeks. After that time, we may close your PR if it isn't showing any activity.
7 changes: 7 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
FROM fkrull/multi-python:latest

RUN apt update && apt install -y pypy pypy-dev pypy3-dev

WORKDIR /redis-py

COPY . /redis-py
14 changes: 14 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
.PHONY: base clean dev test

base:
docker build -t redis-py-base docker/base

dev: base
docker-compose up -d --build

test: dev
docker-compose run --rm test /redis-py/docker-entry.sh

clean:
docker-compose stop
docker-compose rm -f
6 changes: 6 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,12 @@ or from source:
$ python setup.py install
Contributing
------------

Want to contribute a feature, bug report, or report an issue? Check out our `guide to
contributing <https://github.com/andymccurdy/redis-py/blob/master/CONTRIBUTING.rst>`_.


Getting Started
---------------
Expand Down
1 change: 0 additions & 1 deletion build_tools/.bash_profile

This file was deleted.

4 changes: 0 additions & 4 deletions build_tools/bootstrap.sh

This file was deleted.

29 changes: 0 additions & 29 deletions build_tools/build_redis.sh

This file was deleted.

37 changes: 0 additions & 37 deletions build_tools/install_redis.sh

This file was deleted.

37 changes: 0 additions & 37 deletions build_tools/install_sentinel.sh

This file was deleted.

8 changes: 0 additions & 8 deletions build_tools/redis-configs/001-master

This file was deleted.

10 changes: 0 additions & 10 deletions build_tools/redis-configs/002-slave

This file was deleted.

Loading

0 comments on commit 75a2cfe

Please sign in to comment.