thiggle

API Docs | Website

Structured LLM APIs that always return deterministic results.

• LLM Model API Gateway -- Call over 20+ LLMs with a single API
• LLM Typed Completion API -- Generate LLM output in JSON that matches a TypeScript schema
• LLM Regex Completion API -- Generate LLM output that always matches a regex pattern
• LLM Context-Free Grammar Completion API -- Generate LLM output that always matches a context-free grammar
• LLM Categorization API -- A simple structured API to run categorization tasks with an LLM

Model API Gateway

Call over 20+ LLM models with a single API from Llama 2, GPT-4, Cohere, and more.

• Single API to inference over 20+ LLMs. Llama 2, GPT-4, Cohere, StableLM and more.
• Same price as direct inference. No upcharge.
• Drop-in compatible with OpenAI clients. Just change one line of code.
• One API key. Never give up your OpenAI credentials.

If you’re already using the OpenAI Python APIs, you can switch

openai.api_base = 'https://api.thiggle.com/v1/'

Otherwise, you can use the client libraries provided here, or a simple cURL command.

curl -X POST "https://api.thiggle.com/v1/completion" \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $THIGGLE_API_KEY" \
     -d '{
         "prompt": "What animal barks?",
         "max_new_tokens": 10,
         "model": ["llama-2-70b-chat", "gpt-4"]
     }'

Models Supported

• llama-2-70b-chat
• llama-2-70b
• llama-2-13b-chat
• llama-2-7b
• llama-2-7b-chat
• gpt-4
• gpt-4-0613
• gpt-4-0314
• gpt-4-32k
• gpt-4-32k-0613
• gpt-4-32k-0314
• gpt-3.5-turbo
• gpt-3.5-turbo-0613
• gpt-3.5-turbo-0301
• gpt-3.5-turbo-16k
• gpt-3.5-turbo-16k-0613
• cohere-command
• cohere-command-light
• cohere-command-nightly
• text-davinci-00{3,2,1}

Regex Completion API

• Regex Constraint: Generate LLM output that always matches a regex pattern.
• Deterministic: Never returns unexpected or unparsable results.

Given a prompt and a regex pattern, produces a constrained LLM text generation. Useful for generating specific semantic structures, typed primitives, or templates. The output is always deterministic and will always match the regex pattern provided.

Typed Completion API

• TypeScript Constraint: Generate LLM output in JSON that matches a TypeScript schema.

Given a set of TypeScript definitions and a target type, generate LLM output that is valid JSON in shape of that type. Useful for generating specific semantic structures, typed primitives, or templates.

Categorization API

A simple structured API to run categorization tasks with an LLM.

• Zero Parsing: Always returns structured JSON with only your categories
• 0,1,or N Labels: Return exactly one class, or allow multiple classes, or allow uncategorized results
• Deterministic: Never returns unexpected or unparsable results.

Examples

• Building block for building higher-level AI agents
• Answering multiple choice questions
• Labeling training data
• Sentiment analysis

Quickstart

Get an API key at thiggle.com/account. Set it as an environment variable THIGGLE_API_KEY. Call the API directly over HTTPS or use one of the client libraries.

• cURL
• Python
• Go
• TypeScript

cURL

curl -X POST "https://api.thiggle.com/v1/categorize" \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $THIGGLE_API_KEY" \
     -d '{
         "prompt": "What animal barks?",
         "categories": ["Dog", "Cat", "Bird", "Fish"]
     }'

Python

pip install thiggle

import thiggle as tg

api = tg.API(os.getenv("THIGGLE_API_KEY"))
response = api.categorize("What animal barks?", ["dog", "cat", "bird", "fish"])
print(response)

TypeScript

npm install @thiggle/client

import Thiggle from "@thiggle/client";

const api = new Thiggle(process.env.THIGGLE_API_KEY);
const response = await api.categorize("What animal barks?", [
  "dog",
  "cat",
  "bird",
  "fish",
]);
console.log(response);

Go

go get github.com/thiggle/api/client-go

package main

import (
    "fmt"
    "os"

    "github.com/thiggle/api"
)

func main() {
    client := api.NewClient(os.Getenv("THIGGLE_API_KEY"))
    response, err := client.Categorize("What animal barks?", []string{"dog", "cat", "bird", "fish"})
    if err != nil {
        panic(err)
    }
    fmt.Println(response)
}

Context-Free Grammar Completion API

Given a prompt and a context-free grammar, produces a constrained LLM text generation. Useful for generating specific semantic structures, typed primitives, or templates. The output is always deterministic and will always match the context-free grammar provided.

Grammars must be LALR grammars in Lark format. See the lark-parser.

See docs.thiggle.com for more information and examples.

API Keys

To get started, you'll need an API key. You can get one by signing up for an account at https://thiggle.com. Once you create an account, you generate API keys on your account page. Set the THIGGLE_API_KEY environment variable to your API key.

export THIGGLE_API_KEY=your-api-key

If you are using a client library, you can pass the API key as a parameter to the client constructor. If you are using the REST API directly, you can pass the API key in the Authorization header (be sure to include the Bearer prefix).

curl -X POST "https://api.thiggle.com/v1/categorize" \
   -H "Content-Type: application/json" \
   -H "Authorization: Bearer $THIGGLE_API_KEY" \
   -d '{
       "prompt": "What animal barks?",
       "categories": ["Dog", "Cat", "Bird", "Fish"]
   }'

Examples

AI Agents

Use the categorization API to choose the relevant tools to complete the task. Use this as a reliable building block for higher-order AI agents. Never worry about the API returning extraneous text or unknown categories that break your agent.

{
  "prompt": "What tools do I need to complete the following task? Task: find the best restaurant in San Francisco. Tools:",
  "categories": [
    "google-maps-api",
    "python-repl",
    "calculator",
    "yelp-api",
    "ffmpeg"
  ]
}

{
  "choices": ["google-maps-api", "yelp-api"]
}

Multiple-Choice Questions

Answer multiple-choice questions. For questions with more than one correct answer, use the allow_multiple_classes flag.

{
  "prompt": "What animals have four legs?",
  "categories": ["cat", "dog", "bird", "fish", "elephant", "snake"],
  "allow_multiple_classes": true
}

{
  "choices": ["cat", "dog", "elephant"]
}

Labeling Training Data

You can use the categorization API to label text for training data. For example, you could use it to label text for a text classifier. This example bins text into different buckets: ['finance', 'sports', 'politics', 'science', 'technology', 'entertainment', 'health', 'other'].

{
  "prompt": "What category does this text belong to? Text: The Dow Jones Industrial Average fell 200 points on Monday.",
  "categories": [
    "finance",
    "sports",
    "politics",
    "science",
    "technology",
    "entertainment",
    "health",
    "other"
  ]
}

{
  "choices": ["finance"]
}

Sentiment Analysis

Classify any text into sentiment classes.

{
  "prompt": "Is this a positive or negative review of Star Wars? The more one sees the main characters, the less appealing they become. Luke Skywalker is a whiner, Han Solo a sarcastic clod, Princess Leia a nag, and C-3PO just a drone",
  "categories": ["positive", "negative"]
}

{
  "choices": ["negative"]
}

Use any sentiment categories you like. For example, you could use ["positive", "neutral", "negative"] or ["positive", "negative", "very positive", "very negative"]. Or even ["happy", "sad", "angry", "surprised", "disgusted", "fearful"].

Rate Limits

The API is rate limited to 100 requests per minute. If you exceed this limit, you will receive a 429 Too Many Requests response. If you need a higher rate limit, please contact us at quota@thiggle.com.

Your current rate limit usage is returned in the X-RateLimit-Remaining header. If you are using a client library, you can use this to determine when you are approaching the rate limit.

Quota

Quotas are determined by your current plan. You can view your current plan on your account page. The quota is reset at the beginning of each month. If you exceed your quota, you will receive a 402 Payment Required response. Your current quota usage is returned in the X-Quota-Remaining header. If you are using a client library, you can use this to determine when you are approaching your quota.