Council is an open-source platform for building applications with Large Language Models (LLMs) - built in Python.
Council provides a unified interface for working with different LLM providers like OpenAI, Anthropic, Google and Ollama. The framework makes it easy to switch between providers while maintaining consistent interfaces and monitoring capabilities.
Council aims to provide enterprise-grade quality control and monitoring for LLM applications (contributions are welcome).
- 🧐 Unified LLM Interface: Consistent API across different LLM providers with built-in error handling and retries
- 🔄 Provider Flexibility: Easy switching between LLM providers like OpenAI, Anthropic, Google Gemini, and local models via Groq and Ollama
- 📊 Usage Monitoring: Built-in consumption tracking and monitoring capabilities
- 🛠️ Configuration Management: Flexible configuration system for LLM parameters like temperature, max tokens etc.
- 🔒 Error Handling: Robust error handling and retry mechanisms for production use
The core of Council is the LLM interface which provides a unified way to interact with different language model providers. This includes:
- Flexible configuration options
- Consistent message formatting across providers
- Built-in retry mechanisms
- Usage tracking and monitoring
Council provides a robust configuration system that allows you to:
- Set provider-specific parameters
- Configure retry behavior
- Control model parameters like temperature, max tokens etc.
- Manage API credentials
Built-in monitoring capabilities help track:
- Token usage and costs
- Number of API calls
- Response times
Install Council in one of multiple ways:
- (Recommended) Install with pip via Pypi:
pip install council-ai - Install with pip from git ref:
pip install git+https://github.com/chain-ml/council.git@<branch_name>- More documentation here: https://pip.pypa.io/en/stable/topics/vcs-support/#git
- Install with pip from local copy:
- Clone this repository
- Navigate to local project root and install via
pip install -e git+https://github.com/chain-ml/council.git@<branch_name>.
Uninstall with: pip uninstall council-ai
Set up your required API keys in a .env file (e.g. OpenAI). Refer to .env.example as an example.
Use make lint to verify your code.
Use black . to automatically reformat files.
A detailed documentation of Council can be found at council.dev.
Please submit a GitHub issue should you need any help or reach out to the team via Discord.
Council is a project under active development. We welcome all contributions, pull requests, feature requests or reported issues.