doc.go

/*
Package workerpool queues work to a limited number of goroutines.

The purpose of the worker pool is to limit the concurrency of tasks executed by
the workers. This is useful when performing tasks that require sufficient
resources (CPU, memory, etc.), and running too many tasks at the same time
would exhaust resources.

Non-blocking task submission

A task is a function submitted to the worker pool for execution. Submitting
tasks to this worker pool will not block, regardless of the number of tasks.
Incoming tasks are immediately dispatched to an available worker. If no worker
is immediately available, or there are already tasks waiting for an available
worker, then the task is put on a waiting queue to wait for an available
worker.

The intent of the worker pool is to limit the concurrency of task execution,
not limit the number of tasks queued to be executed. Therefore, this unbounded
input of tasks is acceptable as the tasks cannot be discarded. If the number of
inbound tasks is too many to even queue for pending processing, then the
solution is outside the scope of workerpool. It should be solved by
distributing load over multiple systems, and/or storing input for pending
processing in intermediate storage such as a database, file system, distributed
message queue, etc.

Dispatcher

This worker pool uses a single dispatcher goroutine to read tasks from the
input task queue and dispatch them to worker goroutines. This allows for a
small input channel, and lets the dispatcher queue as many tasks as are
submitted when there are no available workers. Additionally, the dispatcher can
adjust the number of workers as appropriate for the work load, without having
to utilize locked counters and checks incurred on task submission.

When no tasks have been submitted for a period of time, a worker is removed by
the dispatcher. This is done until there are no more workers to remove. The
minimum number of workers is always zero, because the time to start new workers
is insignificant.

Usage note

It is advisable to use different worker pools for tasks that are bound by
different resources, or that have different resource use patterns. For example,
tasks that use X Mb of memory may need different concurrency limits than tasks
that use Y Mb of memory.

Waiting queue vs goroutines

When there are no available workers to handle incoming tasks, the tasks are put
on a waiting queue, in this implementation. In implementations mentioned in the
credits below, these tasks were passed to goroutines. Using a queue is faster
and has less memory overhead than creating a separate goroutine for each
waiting task, allowing a much higher number of waiting tasks. Also, using a
waiting queue ensures that tasks are given to workers in the order the tasks
were received.

Credits

This implementation builds on ideas from the following:

http://marcio.io/2015/07/handling-1-million-requests-per-minute-with-golang
http://nesv.github.io/golang/2014/02/25/worker-queues-in-go.html

*/
package workerpool