feat(conversation): update directive input for tool definition for model list queries

[atierian]

Related PR

• feat(ai): support model generated and custom op queries with a.ai.dataTool amplify-data#390

Problem

1. Customers wanting to use generated model queries as tools for conversation routes must know the name of the generated model query in advance.

type Todo @model @auth(rules: [{ allow: owner }]) {
  content: String
  isDone: Boolean
}

type Mutation {
  chat(
    conversationId: ID!,
    content: [ContentBlockInput],
    aiContext: AWSJSON,
    toolConfiguration: ToolConfigurationInput
  ): ConversationMessage
  @conversation(
    aiModel: "anthropic.claude-3-haiku-20240307-v1:0",
    systemPrompt: "You are a helpful chatbot. Answer questions to the best of your ability.",
    tools: [
      {
        name: "listTodos",  # <--- must match query name
        description: "lists todos",
      },
    ],
    auth: { strategy: owner, provider: userPools },
  )
}

2. The name of a tool must be the name of the GraphQL query. This is an arbitrary requirement we currently enforce through the directive API that isn't necessary.

Description of changes

Updates the ToolMap input in the @conversation directive definition

  input ToolMap {
-    name: String
-    description: String
+    name: String!
+    description: String!
+    queryName: String
+    modelName: String
+    modelOperation: ConversationToolModelOperation
  }

+  enum ConversationToolModelOperation {
+    list
+  }

As referenced in the inline documentation, this is essentially a fake union type.

New DX

# model generated list query
@conversation(
  aiModel: "anthropic.claude-3-haiku-20240307-v1:0",
  systemPrompt: "You are a helpful chatbot. Answer questions to the best of your ability.",
  tools: [
    {
      name: "list_customers",
      description: "Provides data about the customer sending a message",
      modelName: "Customer",
      modelOperation: list,
    },
  ],
  auth: { strategy: owner, provider: userPools },
)

# custom query
@conversation(
  aiModel: "anthropic.claude-3-haiku-20240307-v1:0",
  systemPrompt: "You are a helpful chatbot. Answer questions to the best of your ability.",
  tools: [
    { name: "thermometer", description: "does a thing", queryName: "getTemperature" },
    { name: "calculator", description: "does a different thing", queryName: "plus" }
  ],
  auth: { strategy: owner, provider: userPools },
)

Why only support list queries?

• We only support queries (non-mutating) tools currently. This is an intentional constraint to prevent unexpected destructive actions through the non-deterministic nature of LLMs.
• We only support list queries because get queries require the LLM to have the primary key of the relevant model. It's possible to make this work today by including the primary key in the aiContext of a message, however this approach hasn't proven dependable or an acceptable DX. We're planning on exploring alternative ways to support get queries; until then, we're constraining model generated query tools to list.

Why not just have data-schema generate the queryName rather than changing the directive API?

This was the initial route I planned, but it's the wrong one as it would force data-schema to take a dependency on the implementation details of the model transformer.

What about custom named model generated list queries?

Supported 😄

CDK / CloudFormation Parameters Changed

N/A

Issue #, if available

N/A

Description of how you validated changes

Checklist

• PR description included
• yarn test passes
• E2E test run linked
• Tests are changed or added
• Relevant documentation is changed or added (and PR referenced)
• New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies
• Any CDK or CloudFormation parameter changes are called out explicitly

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[palpatim]

Does the validation allow for mixed-mode tool maps? In other words, the map contains both a queryName style and a modelOperation style? (I can’t think of why this would be necessary, but if it’s supported, we should test it)

Approved assuming the answer is "no", or that we'll add tests for mixed-mode support later.

Also approved as API BR since this is a change to the conversation directive API.

[atierian]

Does the validation allow for mixed-mode tool maps? In other words, the map contains both a queryName style and a modelOperation style? (I can’t think of why this would be necessary, but if it’s supported, we should test it)

A single tool definition cannot have both queryName (custom operation) and modelName / modelOperation (model generated query). This is validated here and we have a test case here

But a single @conversation can have multiple tools, which can be mixed. I thought I added a test case for that, but looking again, I only added one in data-schema, which doesn't cover everything we'd get with a test here.

Approved assuming the answer is "no", or that we'll add tests for mixed-mode support later.

If I need to make any other changes to this branch before merging, I'll include a test case for mixed tool definitions. Otherwise, I'll include it in a follow up.

[atierian]

The failing test is verify_amplify_backend_compatability because this branch isn't up to date with main. It's not related to these changes.