CKAN extension that allows users to subscribe to dataset/organization/group updates WITHOUT requiring them to login.
This feature is complementary to CKAN's existing "Follow" feature, which allows logged in users to subscribe to get update emails. Log-in can be a barrier to casual interest in say a handful of datasets. Generating and storing a password is a burden on the user, and for casual use just using temporary email links, as in this extension, is more appropriate.
More screenshots: https://github.com/bellisk/ckanext-subscribe/tree/master/doc
Compatibility with core CKAN versions:
CKAN version | Compatibility |
---|---|
2.6 and earlier | no |
2.7 | yes |
2.8 | yes |
2.9 | not yet |
To install ckanext-subscribe:
Activate your CKAN virtual environment, for example:
. /usr/lib/ckan/default/bin/activate
Install the ckanext-subscribe Python package into your virtual environment:
pip install ckanext-subscribe
Add
subscribe
to theckan.plugins
setting in your CKAN configuration file (by default the config file is located at/etc/ckan/default/production.ini
).Make sure that
ckan.site_url
is set correctly in the[app:main]
section of your CKAN configuration file. This is used to generate links in the bodies of the notification emails. For example:ckan.site_url = https://example.com
Make sure that
smtp.mail_from
is set correctly in the[app:main]
section of your CKAN configuration file. This is the email address that CKAN's email notifications will appear to come from. For example:smtp.mail_from = info@example.com
This is combined with your
ckan.site_title
to form theFrom:
header of the email that are sent, for example:From: Sunnyville Open Data <info@example.com>
If you would like to use an alternate reply address, such as a "no-reply" address, set
smtp.reply_to
in the[app:main]
section of your CKAN configuration file. For example:smtp.reply_to = noreply@example.com
If you do not have an SMTP server running locally on the machine that hosts your CKAN instance, you can change the
email-settings
to send email via an external SMTP server. For example, these settings in the[app:main]
section of your configuration file will send emails using a gmail account (not recommended for production websites!):smtp.server = smtp.gmail.com:587 smtp.starttls = True smtp.user = your_username@gmail.com smtp.password = your_gmail_password smtp.mail_from = your_username@gmail.com
Initialize the subscribe tables in the database:
paster --plugin=ckanext-subscribe subscribe initdb
Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
sudo service apache2 reload
You need to run the 'send-any-notifications' command regularly. You can see it running on the command-line:
paster --plugin=ckanext-subscribe subscribe send-any-notifications -c /etc/ckan/default/production.ini
However instead you'll probably want a cron job setup to run it every minute or so. We're going to edit the cron table. On a development machine, just do this for your user:
crontab -e
Or a production machine use the 'ckan' user, instead of checking for notifications on the command-line, create CRON job. To do so, edit the cron table with the following command (it may ask you to choose an editor):
sudo crontab -e -u ckan
Paste this line into your crontab, again replacing the paths to paster and the ini file with yours:
# m h dom mon dow command * * * * * /usr/lib/ckan/default/bin/paster --plugin=ckanext-subscribe subscribe send-any-notifications --config=/etc/ckan/default/production.ini
This particular example will check for notifications every minute.
Also in this cron you will likely see it also running a paster command for /api/action/send_email_notifications. This is similar but separate functionality, that core CKAN uses to send emails to users that have created user accounts e.g. for the 'follower' functionality. There's more about this here: https://docs.ckan.org/en/2.8/maintaining/email-notifications.html
# Email notifications older than this time period will not be sent. # So, after a pause in the sending of emails, when it restarts, it will not # notify about activity which is: # * older than this period, for immediate subscriptions # * older than this period + 1 day, for daily subscriptions # * older than this period + 1 week, for weekly subscriptions # Accepted formats: ‘2 days’, ‘14 days’, ‘4:35:00’ (hours, minutes, seconds), # ‘7 days, 3:23:34’, etc. # See also: https://docs.ckan.org/en/2.8/maintaining/configuration.html#ckan-email-notifications-since # (optional, default: ‘2 days’) ckan.email_notifications_since = 24:00:00 # The time that daily and weekly notification subscriptions are sent (UTC # timezone) ckanext.subscribe.daily_and_weekly_notification_time = 09:00 # The day of the week that weekly notification subscriptions are sent ckanext.subscribe.weekly_notification_day = friday *** reCAPTCHA implementation *** Applying reCAPTCHA helps enhance the security of the dataset subscription form by preventing automated bots from submitting them. To integrate reCAPTCHA, you must first register your domain at https://www.google.com/recaptcha and obtain private and public keys. These keys, along with the reCAPTCHA API URL, need to be configured in your setup:: ckanext.subscribe.recaptcha.privatekey = [Your_Private_Key] ckanext.subscribe.recaptcha.publickey = [Your_Public_Key] ckanext.subscribe.recaptcha.api_url = [Your_API_URL], e.g., https://www.google.com/recaptcha/api/siteverify Additionally, ensure that the `apply_recaptcha` configuration is set to true in order to enable reCAPTCHA:: ckanext.subscribe.apply_recaptcha = false
Notification emails not being sent
Check your cron schedule is working:
tail -f /var/log/syslog | grep subscribe
You should see messages every minute:
Jan 10 15:24:01 ip-172-30-3-71 CRON[29231]: (ubuntu) CMD (/usr/lib/ckan/default/bin/paster --plugin=ckanext-subscribe subscribe run --config=/etc/ckan/default/production.ini)
Create a test activity for a dataset/group/org you are subscribed to:
paster --plugin=ckanext-subscribe subscribe create-test-activity mydataset --config=/etc/ckan/default/production.ini
The log of the cron-activated paster command itself is not currently stored anywhere, so it's best to test it on the commandline:
paster --plugin=ckanext-subscribe subscribe send-any-notifications --config=/etc/ckan/default/production.ini
You should see emails being sent to subscribers of that dataset:
2020-01-06 16:30:40,591 DEBUG [ckanext.subscribe.notification] send_any_immediate_notifications 2020-01-06 16:30:40,628 DEBUG [ckanext.subscribe.notification] sending 1 emails (immediate frequency) 2020-01-06 16:30:42,116 INFO [ckanext.subscribe.mailer] Sent email to david.read@hackneyworkshop.com
Clean up all test activity afterwards:
paster --plugin=ckanext-subscribe subscribe delete-test-activity --config=/etc/ckan/default/production.ini
NameError: global name 'Subscription' is not defined
You need to initialize the subscribe tables in the database. See 'Installation' section above to do this.
KeyError: "Action 'subscribe_signup' not found"
You need to enable the subscribe plugin in your CKAN config. See 'Installation' section above to do this.
ProgrammingError: (ProgrammingError) relation "subscription" does not exist
You're running the tests with --reset-db and this extension doesn't work with that. Instead, if you need to wipe the tables before running tests, do it this way:
sudo -u postgres psql ckan_test -c 'drop table if exists subscription; drop table if exists subscribe_login_code; drop table if exists subscribe;'
or simply:
sudo -u postgres dropdb ckan_test sudo -u postgres createdb -O ckan_default ckan_test -E utf-8 paster --plugin=ckan db init -c ../ckan/test-core.ini
To install ckanext-subscribe for development, activate your CKAN virtualenv and do:
git clone https://github.com/bellisk/ckanext-subscribe.git cd ckanext-subscribe python setup.py develop pip install -r dev-requirements.txt
Now continue Installation steps from step 3
To run the tests, do:
nosetests --nologcapture --with-pylons=test.ini
To run the tests and produce a coverage report, first make sure you have
coverage installed in your virtualenv (pip install coverage
) then run:
nosetests --nologcapture --with-pylons=test.ini --with-coverage --cover-package=ckanext.subscribe --cover-inclusive --cover-erase --cover-tests
ckanext-subscribe should be available on PyPI as https://pypi.org/project/ckanext-subscribe. To publish a new version to PyPI follow these steps:
Update the version number in the
setup.py
file. See PEP 440 for how to choose version numbers.Update the CHANGELOG.md
Make sure you have the latest version of necessary packages:
pip install --upgrade setuptools wheel twine
Create a source and binary distributions of the new version:
python setup.py sdist bdist_wheel && twine check dist/*
Fix any errors you get.
Upload the source distribution to PyPI:
twine upload dist/*
Commit any outstanding changes:
git commit -a git push
Tag the new release of the project on GitHub with the version number from the
setup.py
file. For example if the version number insetup.py
is 0.0.1 then do:git tag 0.0.1 git push --tags