Infino is an open-source observability platform that stores both metrics and application logs together.
Key features of infino include:
- Metrics Tracking: Capture time taken by LLM model to handle request, errors, number of tokens, and costing indication for the particular LLM.
- Data Tracking: Log and store prompt, request, and response data for each LangChain interaction.
- Graph Visualization: Generate basic graphs over time, depicting metrics such as request duration, error occurrences, token count, and cost.
First, you'll need to install the infinopy Python package as follows:
pip install infinopyIf you already have an Infino Server running, then you're good to go; but if you don't, follow the next steps to start it:
- Make sure you have Docker installed
- Run the following in your terminal:
docker run --rm --detach --name infino-example -p 3000:3000 infinohq/infino:latest
See a usage example of InfinoCallbackHandler.
from langchain.callbacks import InfinoCallbackHandler