Human Lambdas is an open source Human in the Loop platform for anyone to run their own private Mechanical Turk.
It is designed to provide an all-in-one solution to build, operate and monitor Human in the Loop processes. Its highly flexible, composable and user-friendly UI is designed to enable anyone to easily support all sorts of bespoke manual tasks. Some examples are: data annotation, data cleaning, quality assurance, data extraction, sales research, content moderation, customer operations...
We are going to run Human Lambdas on a developer device.
Please ensure you have Docker installed.
mkdir .human_lambdas # create data directory
docker run \
-p 8000:8000 \
-it \
--rm \
-v $(pwd)/.human_lambdas:/.human_lambdas \
hlambdas/human-lambdas:v1.0
A shell should eventually start inside the container.
Now initialize a sqlite database in your .human_lambdas dir
human-lambdas initdb
Start the server
human-lambdas up
Human Lambdas is now running against a Sqlite database stored in your working directory.
Check out the server at http://localhost:8000 .
Note: It is best to use incognito or clear cookies if you have used it before without setting a Django SECRET_KEY
Next Try defining your first queue by following this guide.
Human Lambdas requires a database for storing user and task data. When you deploy an instance for your team it is best to use a production-grade database such as Postgres.
You can use a managed Postgres product such as Amazon RDS, or use a Docker container as follows.
When starting your container, pass in authentication details in environment variables.
export POSTGRES_USER=hlambda
export POSTGRES_DB=hlambda
export POSTGRES_PASSWORD=some_password
docker run \
-p 5432:5432 \
-e "POSTGRES_USER=$POSTGRES_USER" \
-e "POSTGRES_PASSWORD=$POSTGRES_PASSWORD" \
-e "POSTGRES_DB=$POSTGRES_DB" \
postgres:10
Then initialise your database and start the Human Lambdas server in another container:
export POSTGRES_HOST="host.docker.internal" # Note: This assumes you are trying this out on OS X
export POSTGRES_PORT=5432
export POSTGRES_USER=hlambda
export POSTGRES_DB=hlambda
export POSTGRES_PASSWORD=some_password
docker run \
-it \
--rm \
-e "POSTGRES_HOST=$POSTGRES_HOST" \
-e "POSTGRES_PORT=$POSTGRES_PORT" \
-e "POSTGRES_USER=$POSTGRES_USER" \
-e "POSTGRES_PASSWORD=$POSTGRES_PASSWORD" \
-e "POSTGRES_DB=$POSTGRES_DB" \
--entrypoint=bash \
-p 8000:8000 \
-v $(pwd)/.human_lambdas:/.human_lambdas \
hlambdas/human-lambdas:v1.0 \
-c 'hl initdb && hl up'
Human Lambdas is now available on localhost:8000.
This approach lets you use Human Lambdas in stateless environments such as Google Cloud Run as long as
- You do not run
initdb
on startup, as it is not thread-safe - You set a single
SECRET_KEY
environment variable so that all Django Invite/Session tokens work
Requires Python 3.
pip install human-lambdas