Use ChatGPT over Twilio to create an AI phone agent (works for incoming or outgoing calls).
Twilio Webhook -> Flask app -> Twilio Media Stream (websocket) -> Whisper -> ChatGPT API -> Google TTS -> Twilio Play Audio
pip install git+https://github.com/sshh12/llm_convo
- Environment Variables
OPENAI_API_KEY=
TWILIO_ACCOUNT_SID=
TWILIO_AUTH_TOKEN=
TWILIO_PHONE_NUMBER=
Try python examples\keyboard_chat_with_gpt.py
to chat with GPT through terminal.
Try python examples\twilio_ngrok_ml_rhyme_hotline.py --preload_whisper --start_ngrok
. This requires whisper installed locally.
This will create an ngrok tunnel and provide a webhook URL to point to in Twilio settings for a purchased phone number.
![chrome_VZSfJHN6FV](https://private-user-images.githubusercontent.com/6625384/263423630-1fe9468d-0eb3-4309-9b81-1d2f3d02c353.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxMjQ1NzIsIm5iZiI6MTcyNjEyNDI3MiwicGF0aCI6Ii82NjI1Mzg0LzI2MzQyMzYzMC0xZmU5NDY4ZC0wZWIzLTQzMDktOWI4MS0xZDJmM2QwMmMzNTMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkxMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MTJUMDY1NzUyWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9Mjc3OTQ2ZDMxNWZiZTU4Y2RhNmMyMzQwZDFkYmE5MjBmZWM1YWYyYTVjZGI1NTc2ZjVmYjhkZGJhZTUzMWI4MCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.ZAZtudMP2NOB3q5ENZw7ouOPL5cyWGqPYBH8WMNZ1Pk)
Recording.mp4
Try python examples\twilio_ngrok_pizza_order.py --preload_whisper --start_ngrok --phone_number "+1.........."
. This requires whisper installed locally.
This will create an ngrok tunnel and provide a webhook URL to point to in Twilio settings for a purchased phone number. Once the webhook is updated, it will start an outgoing call to the provided phone number.
Setup a Haiku hotline with Twilio that can be called like any other phone number.
from gevent import monkey
monkey.patch_all()
from llm_convo.agents import OpenAIChat, TwilioCaller
from llm_convo.twilio_io import TwilioServer
from llm_convo.conversation import run_conversation
import logging
import time
logging.getLogger().setLevel(logging.INFO)
tws = TwilioServer(remote_host="abcdef.ngrok.app", port=8080, static_dir=r"/path/to/static")
# Point twilio voice webhook to https://abcdef.ngrok.app/audio/incoming-voice
tws.start()
agent_a = OpenAIChat(
system_prompt="You are a Haiku Assistant. Answer whatever the user wants but always in a rhyming Haiku.",
init_phrase="This is Haiku Bot, how can I help you.",
)
# agent_a = OpenAIChat(
# system_prompt="""
# You are an ordering bot that is going to call a pizza place an order a pizza.
# When you need to say numbers space them out (e.g. 1 2 3) and do not respond with abbreviations.
# If they ask for information not known, make something up that's reasonable.
# The customer's details are:
# * Address: 1234 Candyland Road, Apt 506
# * Credit Card: 1234 5555 8888 9999 (CVV: 010)
# * Name: Bob Joe
# * Order: 1 large pizza with only pepperoni
# """,
# init_phrase="Hi, I would like to order a pizza.",
# )
def run_chat(sess):
agent_b = TwilioCaller(sess)
while not agent_b.session.media_stream_connected():
time.sleep(0.1)
run_conversation(agent_a, agent_b)
tws.on_session = run_chat
# You can also have ChatGPT actually start the call, e.g. for automated ordering
# tws.start_call("+18321231234")