Background jobs functionality for CKAN 2.6 and earlier.
This is a backport of the background jobs functionality (based on RQ) that was introduced in CKAN 2.7. With this extension, you can make use of RQ based background tasks, on earlier versions of CKAN.
It is based on the code by @torfsen, mainly here: ckan/ckan#3165
TODO:
- check Redis is available on startup
- setting ckan.redis.url from environment variable CKAN_REDIS_URL
You won't be able to use the normal plugin.toolkit.enqueue_job function. Use this import which contains a fallback:
try: enqueue_job = p.toolkit.enqueue_job except AttributeError: from ckanext.rq.jobs import enqueue as enqueue_job
For CKAN versions 2.3 to 2.6.x. (Must not be used with CKAN 2.7.0 or later)
You need Redis installed:
sudo apt-get install redis-server
To install ckanext-rq:
Activate your CKAN virtual environment, for example:
. /usr/lib/ckan/default/bin/activate
Install the ckanext-rq Python package into your virtual environment:
Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
sudo service apache2 reload
Start the worker:
paster --plugin=ckanext-rq jobs worker --config=/etc/ckan/default/development.ini
To run the worker in a robust way, install and configure Supervisor: http://docs.ckan.org/en/latest/maintaining/background-tasks.html#using-supervisor
# URL to your Redis instance, including the database to be used. ckan.redis.url = redis://localhost:6379/0
To install ckanext-rq for development, activate your CKAN virtualenv and do:
git clone https://github.com/ckan/ckanext-rq.git cd ckanext-rq python setup.py develop pip install -r requirements.txt pip install -r dev-requirements.txt
To run the tests, do:
nosetests --nologcapture --with-pylons=test.ini
To run the tests and produce a coverage report, first make sure you have
coverage installed in your virtualenv (pip install coverage
) then run:
nosetests --nologcapture --with-pylons=test.ini --with-coverage --cover-package=ckanext.rq --cover-inclusive --cover-erase --cover-tests
ckanext-rq is availabe on PyPI as https://pypi.python.org/pypi/ckanext-rq. To publish a new version to PyPI follow these steps:
Update the version number in the
setup.py
file. See PEP 440 for how to choose version numbers.Create a source distribution of the new version:
python setup.py sdist
Upload the source distribution to PyPI:
python setup.py sdist upload
Tag the new release of the project on GitHub with the version number from the
setup.py
file. For example if the version number insetup.py
is 0.0.2 then do:git tag 0.0.2 git push --tags