Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
Update types & omit template
  • Loading branch information
hinthornw committed Sep 19, 2024
2 parents c2d1578 + 3f407cd commit 3477826
Show file tree
Hide file tree
Showing 14 changed files with 445 additions and 249 deletions.
4 changes: 2 additions & 2 deletions .eslintrc.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ module.exports = {
ignorePatterns: [
".eslintrc.cjs",
"scripts",
"src/utils/lodash/*",
"node_modules",
"dist",
"dist-cjs",
Expand All @@ -23,10 +22,11 @@ module.exports = {
"*.d.ts",
],
rules: {
"no-process-env": 2,
"no-process-env": 0,
"no-instanceof/no-instanceof": 2,
"@typescript-eslint/explicit-module-boundary-types": 0,
"@typescript-eslint/no-empty-function": 0,
"@typescript-eslint/no-non-null-assertion": 0,
"@typescript-eslint/no-shadow": 0,
"@typescript-eslint/no-empty-interface": 0,
"@typescript-eslint/no-use-before-define": ["error", "nofunc"],
Expand Down
84 changes: 68 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,23 @@
[![Integration Tests](https://github.com/langchain-ai/data-enrichment-js/actions/workflows/integration-tests.yml/badge.svg)](https://github.com/langchain-ai/data-enrichment-js/actions/workflows/integration-tests.yml)
[![Open in - LangGraph Studio](https://img.shields.io/badge/Open_in-LangGraph_Studio-00324d.svg?logo=data:image/svg%2bxml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI4NS4zMzMiIGhlaWdodD0iODUuMzMzIiB2ZXJzaW9uPSIxLjAiIHZpZXdCb3g9IjAgMCA2NCA2NCI+PHBhdGggZD0iTTEzIDcuOGMtNi4zIDMuMS03LjEgNi4zLTYuOCAyNS43LjQgMjQuNi4zIDI0LjUgMjUuOSAyNC41QzU3LjUgNTggNTggNTcuNSA1OCAzMi4zIDU4IDcuMyA1Ni43IDYgMzIgNmMtMTIuOCAwLTE2LjEuMy0xOSAxLjhtMzcuNiAxNi42YzIuOCAyLjggMy40IDQuMiAzLjQgNy42cy0uNiA0LjgtMy40IDcuNkw0Ny4yIDQzSDE2LjhsLTMuNC0zLjRjLTQuOC00LjgtNC44LTEwLjQgMC0xNS4ybDMuNC0zLjRoMzAuNHoiLz48cGF0aCBkPSJNMTguOSAyNS42Yy0xLjEgMS4zLTEgMS43LjQgMi41LjkuNiAxLjcgMS44IDEuNyAyLjcgMCAxIC43IDIuOCAxLjYgNC4xIDEuNCAxLjkgMS40IDIuNS4zIDMuMi0xIC42LS42LjkgMS40LjkgMS41IDAgMi43LS41IDIuNy0xIDAtLjYgMS4xLS44IDIuNi0uNGwyLjYuNy0xLjgtMi45Yy01LjktOS4zLTkuNC0xMi4zLTExLjUtOS44TTM5IDI2YzAgMS4xLS45IDIuNS0yIDMuMi0yLjQgMS41LTIuNiAzLjQtLjUgNC4yLjguMyAyIDEuNyAyLjUgMy4xLjYgMS41IDEuNCAyLjMgMiAyIDEuNS0uOSAxLjItMy41LS40LTMuNS0yLjEgMC0yLjgtMi44LS44LTMuMyAxLjYtLjQgMS42LS41IDAtLjYtMS4xLS4xLTEuNS0uNi0xLjItMS42LjctMS43IDMuMy0yLjEgMy41LS41LjEuNS4yIDEuNi4zIDIuMiAwIC43LjkgMS40IDEuOSAxLjYgMi4xLjQgMi4zLTIuMy4yLTMuMi0uOC0uMy0yLTEuNy0yLjUtMy4xLTEuMS0zLTMtMy4zLTMtLjUiLz48L3N2Zz4=)](https://langgraph-studio.vercel.app/templates/open?githubUrl=https://github.com/langchain-ai/data-enrichment-js)

This is a starter project to help you get started with developing a data enrichment agent using [LangGraph.js](https://github.com/langchain-ai/langgraphjs) in [LangGraph Studio](https://github.com/langchain-ai/langgraph-studio).
Producing structured results (e.g., to populate a database or spreadsheet) from open-ended research (e.g., web research) is a common use case that LLM-powered agents are well-suited to handle. Here, we provide a general template for this kind of "data enrichment agent" agent using [LangGraph](https://github.com/langchain-ai/langgraph) in [LangGraph Studio](https://github.com/langchain-ai/langgraph-studio). It contains an example graph exported from `src/enrichment_agent/graph.ts` that implements a research assistant capable of automatically gathering information on various topics from the web and structuring the results into a user-defined JSON format.

![](/static/studio.png)
![Overview of agent](./static/overview.png)

It contains an example graph exported from `src/enrichment_agent/graph.ts` that implements a research assistant capable of automatically gathering information on various topics from the web.
![](/static/studio.png)

## What it does
# What it does

The enrichment agent:
The enrichment agent defined in `src/enrichment_agent/graph.ts` performs the following steps:

1. Takes a research **topic** and requested **extractionSchema** as input
1. Takes a research **topic** and requested **extractionSchema** as input.
2. Searches the web for relevant information
3. Reads and extracts key details from websites
4. Organizes the findings into the requested structured format
5. Validates the gathered information for completeness and accuracy

By default, it's set up to gather information based on the user-provided schema passed through the `extractionSchema` key in the state.
![Graph view in LangGraph studio UI](./static/studio.png)

## Getting Started

Expand All @@ -43,11 +43,11 @@ Setup instruction auto-generated by `langgraph template lock`. DO NOT EDIT MANUA
-->

<details>
<summary>Setup for `modelName`</summary>
<summary>Setup for `model`</summary>
The `llm` configuration defaults are shown below:

```yaml
modelName: anthropic/claude-3-5-sonnet-20240620
model: anthropic/claude-3-5-sonnet-20240620
```
Follow the instructions below to get set up, or pick one of the additional options.
Expand Down Expand Up @@ -91,27 +91,79 @@ OPENAI_API_KEY=your-api-key
End setup instructions
-->
3. Customize whatever you'd like in the code.
4. Open the folder LangGraph Studio!
3. Consider a research topic and desired extraction schema.
As an example, here is a research topic we can consider.
```
"Top 5 chip providers for LLM Training"
```
And here is a desired extraction schema.
```json
"extractionSchema": {
"type": "object",
"properties": {
"companies": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Company name"
},
"technologies": {
"type": "string",
"description": "Brief summary of key technologies used by the company"
},
"market_share": {
"type": "string",
"description": "Overview of market share for this company"
},
"future_outlook": {
"type": "string",
"description": "Brief summary of future prospects and developments in the field for this company"
},
"key_powers": {
"type": "string",
"description": "Which of the 7 Powers (Scale Economies, Network Economies, Counter Positioning, Switching Costs, Branding, Cornered Resource, Process Power) best describe this company's competitive advantage"
}
},
"required": ["name", "technologies", "market_share", "future_outlook"]
},
"description": "List of companies"
}
},
"required": ["companies"]
}
```

4. Open the folder LangGraph Studio, and input `topic` and `extractionSchema`.

## How to customize

1. **Customize research targets**: Provide a custom `extractionSchema` when calling the graph to gather different types of information.
1. **Customize research targets**: Provide a custom JSON `extractionSchema` when calling the graph to gather different types of information.
2. **Select a different model**: We default to anthropic (claude-3-5-sonnet-20240620). You can select a compatible chat model using `provider/model-name` via configuration. Example: `openai/gpt-4o-mini`.
3. **Customize the prompt**: We provide a default prompt in [src/enrichment_agent/prompts.ts](./src/enrichment_agent/prompts.ts). You can easily update this via configuration in the studio.
3. **Customize the prompt**: We provide a default prompt in [src/enrichment_agent/prompts.ts](./src/enrichment_agent/prompts.ts). You can easily update this via configuration.

For quick prototyping, these configurations can be set in the studio UI.

![Config In Studio](./static/config.png)

You can also quickly extend this template by:

- Adding new tools and API connections in [src/enrichment_agent/tools.ts](./src/enrichment_agent/tools.ts). These are just any TypeScript functions.
- Adding additional steps in [src/enrichment_agent/graph.ts](./src/enrichment_agent/graph.ts). Concerned about hallucination? Add a fact-checking step!
- Adding additional steps in [src/enrichment_agent/graph.ts](./src/enrichment_agent/graph.ts).

## Development

While iterating on your graph, you can edit past state and rerun your app from past states to debug specific nodes. Local changes will be automatically applied via hot reload. Try adding an interrupt before the agent calls tools, updating the default system message in [src/enrichment_agent/utils.ts](./src/enrichment_agent/utils.ts) to take on a persona, or adding additional nodes and edges!

Follow up requests will be appended to the same thread. You can create an entirely new thread, clearing previous history, using the `+` button in the top right.

You can find the latest (under construction) docs on [LangGraph.js](https://langchain-ai.github.io/langgraphjs/) here, including examples and other references. Using those guides can help you pick the right patterns to adapt here for your use case.
You can find the latest (under construction) docs on [LangGraph.JS](https://langchain-ai.github.io/langgraphjs/) here, including examples and other references. Using those guides can help you pick the right patterns to adapt here for your use case.

LangGraph Studio also integrates with [LangSmith](https://smith.langchain.com/) for more in-depth tracing and collaboration with teammates.

Expand All @@ -124,7 +176,7 @@ Configuration auto-generated by `langgraph template lock`. DO NOT EDIT MANUALLY.
"agent": {
"type": "object",
"properties": {
"modelName": {
"model": {
"type": "string",
"default": "anthropic/claude-3-5-sonnet-20240620",
"description": "The name of the language model to use for the agent. Should be in the form: provider/model-name.",
Expand Down
3 changes: 3 additions & 0 deletions jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,7 @@ export default {
],
},
extensionsToTreatAsEsm: [".ts"],
setupFiles: ["dotenv/config"],
passWithNoTests: true,
testTimeout: 20_000,
};
16 changes: 9 additions & 7 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
{
"name": "data-enrichment",
"version": "0.0.1",
"description": "A LangGraph.JS template that populates ",
"description": "A starter template for building a research agent that uses a web search tool to populate a user-provided schema.",
"main": "src/enrichment_agent/graph.ts",
"author": "William Fu-Hinthorn",
"license": "MIT",
"private": true,
"type": "module",
"packageManager": "yarn@1.22.22",
"scripts": {
"build": "tsc",
"clean": "rm -rf dist",
Expand All @@ -20,23 +21,24 @@
"test:all": "yarn test && yarn test:int && yarn lint:langgraph"
},
"dependencies": {
"@langchain/anthropic": "^0.3.0",
"@langchain/community": "^0.3.0",
"@langchain/core": "^0.3.1",
"@langchain/langgraph": "^0.2.3",
"@langchain/anthropic": "^0.3.1",
"@langchain/community": "^0.3.1",
"@langchain/core": "^0.3.2",
"@langchain/langgraph": "^0.2.8",
"langchain": "^0.3.2",
"langsmith": "^0.1.55",
"langsmith": "^0.1.59",
"ts-node": "^10.9.2",
"zod": "^3.23.8"
},
"devDependencies": {
"@eslint/eslintrc": "^3.1.0",
"@eslint/js": "^9.9.1",
"@langchain/openai": "^0.2.7",
"@jest/globals": "^29.7.0",
"@tsconfig/recommended": "^1.0.7",
"@types/jest": "^29.5.0",
"@typescript-eslint/eslint-plugin": "^5.59.8",
"@typescript-eslint/parser": "^5.59.8",
"dotenv": "^16.4.5",
"eslint": "^8.41.0",
"eslint-config-prettier": "^8.8.0",
"eslint-plugin-import": "^2.27.5",
Expand Down
46 changes: 29 additions & 17 deletions src/enrichment_agent/configuration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,49 +3,61 @@
*/

import { RunnableConfig } from "@langchain/core/runnables";
import { Annotation } from "@langchain/langgraph";
import { MAIN_PROMPT } from "./prompts.js";

export interface Configuration {
/**
* The complete configuration for the agent.
*/
export const ConfigurationAnnotation = Annotation.Root({
/**
* The name of the language model to use for the agent.
*
* Should be in the form: provider/model-name.
*/
modelName: string;
model: Annotation<string>,

/**
* The main prompt template to use for the agent's interactions.
*
* Expects two template literals: ${info} and ${topic}.
*/
prompt: string;
prompt: Annotation<string>,

/**
* The maximum number of search results to return for each search query.
*/
maxSearchResults: number;
maxSearchResults: Annotation<number>,

/**
* The maximum number of times the Info tool can be called during a single interaction.
*/
maxInfoToolCalls: number;
maxInfoToolCalls: Annotation<number>,

/**
* The maximum number of interaction loops allowed before the agent terminates.
*/
maxLoops: number;
}
maxLoops: Annotation<number>,
});

/**
* Create a typeof ConfigurationAnnotation.State instance from a RunnableConfig object.
*
* @param config - The configuration object to use.
* @returns An instance of typeof ConfigurationAnnotation.State with the specified configuration.
*/
export function ensureConfiguration(
config?: RunnableConfig,
): typeof ConfigurationAnnotation.State {
const configurable = (config?.configurable || {}) as Partial<
typeof ConfigurationAnnotation.State
>;

export function ensureConfiguration(config?: RunnableConfig): Configuration {
/**
* Ensure the defaults are populated.
*/
const configurable = config?.configurable ?? {};
return {
modelName: configurable.modelName ?? "anthropic/claude-3-5-sonnet-20240620",
prompt: configurable.prompt ?? MAIN_PROMPT,
maxSearchResults: configurable.maxSearchResults ?? 10,
maxInfoToolCalls: configurable.maxInfoToolCalls ?? 3,
maxLoops: configurable.maxLoops ?? 6,
model: configurable.model || "anthropic/claude-3-5-sonnet-20240620",
prompt: configurable.prompt || MAIN_PROMPT,
maxSearchResults: configurable.maxSearchResults || 10,
maxInfoToolCalls: configurable.maxInfoToolCalls || 3,
maxLoops: configurable.maxLoops || 6,
};
}
Loading

0 comments on commit 3477826

Please sign in to comment.