Skip to content

langchain-ai/langchain-benchmarks

Folders and files

NameName
Last commit message
Last commit date

Latest commit

a0ea197 Β· Dec 20, 2023
Dec 20, 2023
Dec 13, 2023
Dec 19, 2023
Dec 19, 2023
Nov 21, 2023
Dec 14, 2023
Nov 21, 2023
Nov 22, 2023
Dec 20, 2023
Dec 7, 2023
Dec 14, 2023
Dec 14, 2023

Repository files navigation

πŸ¦œπŸ’― LangChain Benchmarks

Release Notes CI License: MIT Twitter Open Issues

πŸ“– Documentation

A package to help benchmark various LLM related tasks.

The benchmarks are organized by end-to-end use cases, and utilize LangSmith heavily.

We have several goals in open sourcing this:

  • Showing how we collect our benchmark datasets for each task
  • Showing what the benchmark datasets we use for each task is
  • Showing how we evaluate each task
  • Encouraging others to benchmark their solutions on these tasks (we are always looking for better ways of doing things!)

Installation

To install the packages, run the following command:

pip install -U langchain-benchmarks

All the benchmarks come with an associated benchmark dataset stored in LangSmith. To take advantage of the eval and debugging experience, sign up, and set your API key in your environment:

export LANGCHAIN_API_KEY=ls-...

Repo Structure

The package is located within langchain_benchmarks. Check out the docs for information on how to get starte.

The other directories are legacy and may be moved in the future.

Archived

Below are archived benchmarks that require cloning this repo to run.

Related