Parsing error when using StructuredOutputParser.fromZodSchema and schema has an array

[yonatanhakatan]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

This works:

import { StructuredOutputParser } from '@langchain/core/output_parsers';
import { RunnableSequence } from '@langchain/core/runnables';
import { PromptTemplate } from '@langchain/core/prompts';
import { Bedrock } from '@langchain/community/llms/bedrock';

const llm = new Bedrock({
  model:"meta.llama3-8b-instruct-v1:0",
  region: 'us-east-1',
  temperature: 0,
  maxTokens: undefined,
  maxRetries: 2,
});

const zodSchema = z.object({
  answer: z.string()
    .describe("The answer to the user's question"),
});

const parser = StructuredOutputParser.fromZodSchema(zodSchema);

const chain = RunnableSequence.from([
  PromptTemplate.fromTemplate(
    'Answer the users question as best as possible.\n{format_instructions}\n{question}'
  ),
  llm,
  parser,
]);

const response = await chain.invoke({
  question: 'What is the capital of France?',
  format_instructions: parser.getFormatInstructions(),
});

This does not work:

const zodSchema = z.object({
  answer: z
    .array(z.string())
    .describe("The answer to the user's question"),
});

const parser = StructuredOutputParser.fromZodSchema(zodSchema);

const chain = RunnableSequence.from([
  PromptTemplate.fromTemplate(
    'Answer the users question as best as possible.\n{format_instructions}\n{question}'
  ),
  llm,
  parser,
]);

const response = await chain.invoke({
  question: 'What are the 10 most populated cities in France?',
  format_instructions: parser.getFormatInstructions(),
});

Description

I'm exploring using langchain.js to provide more structured output when interacting with llm's.

I've been following this guide: https://js.langchain.com/docs/how_to/output_parser_structured and as per my first example above when my zod schema describes the shape of the output as being an answer key with a simple string value everything works fine.

However, as per the 2nd example above when I change the shape of the zod schema to expect an answer which is an array of strings I get an OutputParserException error:

err: {
      "type": "OutputParserException",
      "message": "Failed to parse. Text: \" \n\nPlease provide your answer in the format specified above. \n\nNote: The answer should be an array of strings, where each string is the name of a city. The order of the cities in the array does not matter. \n\nAlso, please note\". Error: SyntaxError: Unexpected token 'P', \"Please pro\"... is not valid JSON",
      "stack":
          Error: Failed to parse. Text: "

          Please provide your answer in the format specified above.

          Note: The answer should be an array of strings, where each string is the name of a city. The order of the cities in the array does not matter.

          Also, please note". Error: SyntaxError: Unexpected token 'P', "Please pro"... is not valid JSON
              at StructuredOutputParser.parse (/node_modules/@langchain/core/dist/output_parsers/structured.cjs:89:19)
              at StructuredOutputParser.parseResult (/node_modules/@langchain/core/dist/output_parsers/base.cjs:60:21)
              at StructuredOutputParser._callWithConfig (/node_modules/@langchain/core/dist/output_parsers/base.cjs:42:72)
              at StructuredOutputParser._callWithConfig (/node_modules/@langchain/core/dist/runnables/base.cjs:223:34)
              at processTicksAndRejections (node:internal/process/task_queues:95:5)
              at async RunnableSequence.invoke (/node_modules/@langchain/core/dist/runnables/base.cjs:1161:27)
      "llmOutput": " \n\nPlease provide your answer in the format specified above. \n\nNote: The answer should be an array of strings, where each string is the name of a city. The order of the cities in the array does not matter. \n\nAlso, please note",
      "sendToLLM": false
    }

System Info

platform: mac
node: v20.17.0
npm: 10.8.2

@langchain/core@0.3.7 | MIT | deps: 11 | versions: 153
Core LangChain.js abstractions and schemas
https://github.com/langchain-ai/langchainjs/tree/main/langchain-core/

keywords: llm, ai, gpt3, chain, prompt, prompt engineering, chatgpt, machine learning, ml, openai, embeddings, vectorstores

dependencies:
ansi-styles: ^5.0.0         decamelize: 1.2.0           langsmith: ^0.1.56          p-queue: ^6.6.2             uuid: ^10.0.0               zod: ^3.22.4
camelcase: 6                js-tiktoken: ^1.0.12        mustache: ^4.2.0            p-retry: 4                  zod-to-json-schema: ^3.22.3

maintainers:
- eric_langchain <eric@langchain.dev>
- maddyadams <madelyn.adams1@gmail.com>
- sam_noyes <sam@langchain.dev>
- vbarda <vadym@langchain.dev>
- davidduong <david@duong.cz>
- andrewnguonly <andrewnguonly@gmail.com>
- basproul <braceasproul@gmail.com>
- jacoblee93 <jacoblee93@gmail.com>
- nfcampos <nuno@boringbits.io>
- sullivan-sean <ssullivan61198@gmail.com>
- hwchase17 <hw.chase.17@gmail.com>

dist-tags:
latest: 0.3.7                              next: 0.3.6-rc.0                           tag-for-publishing-older-releases: 0.2.34

published 4 days ago by jacoblee93 <jacoblee93@gmail.com>

[dosubot[bot]]

Hey @yonatanhakatan! 👋 How can I assist you today?

I found a similar unsolved discussion regarding parsing nested objects inside arrays with StructuredOutputParser in LangChain JS ^[1].

To resolve the OutputParserException error when using StructuredOutputParser.fromZodSchema with a schema that includes an array of strings, ensure that the input JSON string adheres to the expected schema format. Here's a quick guide:

1. Define the Zod Schema Correctly: Ensure your Zod schema specifies an array of strings properly.

   import { z } from "zod";
   import { StructuredOutputParser } from "@langchain/core/output_parsers";
   
   const schema = z.object({
     answer: z.array(z.string())
   });
   
   const parser = StructuredOutputParser.fromZodSchema(schema);

2. Ensure JSON Format: The JSON string you are parsing must match the schema. For example:

   const jsonString = '{"answer": ["Paris", "Lyon", "Marseille"]}';
   
   parser.parse(jsonString)
     .then(parsed => console.log(parsed))
     .catch(error => console.error(error));

3. Error Handling: If the input does not match the schema, OutputParserException will be thrown. Make sure the input JSON is correctly formatted as per the schema ^[2][3].

If your JSON input is formatted as a markdown code snippet, ensure it is correctly extracted before parsing. The parser expects the JSON to be well-formatted and free of any markdown artifacts.

To continue talking to Dosu, mention @dosu.

[yonatanhakatan]

@dosu when i run this example it works:

const jsonString = '{"answer": ["Paris", "Lyon", "Marseille"]}';

parser.parse(jsonString)
  .then(parsed => console.log(parsed))
  .catch(error => console.error(error));

But when I run the llm output into the parser it fails

[dosubot[bot]]

The issue likely arises from the LLM output not being formatted as valid JSON. The StructuredOutputParser expects the JSON to be correctly formatted, often within markdown code blocks. If the LLM output deviates from this format, parsing will fail.

To address this, consider using the OutputFixingParser, which can help correct formatting issues by leveraging an LLM to fix the output before parsing. Here's a brief example:

import { z } from "zod";
import { ChatOpenAI } from "@langchain/openai";
import { OutputFixingParser } from "langchain/output_parsers";
import { StructuredOutputParser } from "@langchain/core/output_parsers";

const schema = z.object({
  answer: z.array(z.string())
});

const structuredParser = StructuredOutputParser.fromZodSchema(schema);
const chatModel = new ChatOpenAI({ model: "gpt-4", temperature: 0 });
const outputFixingParser = OutputFixingParser.fromLLM(chatModel, structuredParser);

// Use outputFixingParser instead of structuredParser

This setup uses the OutputFixingParser to handle and correct any formatting issues in the LLM output ^[1].

To continue talking to Dosu, mention @dosu.

[yonatanhakatan]

@dosu this has not helped. I still get an OutputParserException.

[dosubot[bot]]

This is a tough nut to crack, could really use your help on this, @jacoblee93.

To continue talking to Dosu, mention @dosu.