-
Notifications
You must be signed in to change notification settings - Fork 6
/
Copy pathweb_test_scenarios.json
162 lines (162 loc) · 10.6 KB
/
web_test_scenarios.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
{
"test_scenarios": [
{
"scenario_id": "refund_policy_question",
"chatbot_url": "https://decagon.ai/embed/notion",
"agent_system_prompt": "You are a customer service agent for Substack. You are doing your best to resolve users' issues as smoothly as possible.",
"guidelines": "billing_notion.md",
"successful_outcome": "The agent has successfully resolved this issue if they: (1) clearly explained Substack's refund policy for paid publications, (2) helped the user understand what options are available to them based on their specific subscription timing, (3) provided clear next steps if a refund is possible, and (4) explained how to manage subscription settings to prevent unwanted renewals. The user should demonstrate understanding of the refund policy and express satisfaction with the explanation.",
"user_persona": {
"profession": "Reader",
"emotional_state": "curious but slightly concerned",
"chat_style": "polite, straightforward questions, seeks clear information",
"context": "Recently subscribed to a paid Notion subscription and wants to understand refund options",
"initial_message": "Hi, I just subscribed to Notion's paid subscription and I'm wondering - what's your refund policy? Is it possible to get my money back if I change my mind?"
},
"mock_messages": [
{
"role": "user",
"content": "Hi, I just subscribed to a paid newsletter yesterday and I'm wondering - what's your refund policy? Is it possible to get my money back if I change my mind?",
"timestamp": "2024-01-27T14:00:00Z"
},
{
"role": "agent",
"content": "I'd be happy to explain our refund policy. Could you first let me know which publication you subscribed to and whether it's a monthly or annual subscription? This will help me give you the most accurate information.",
"timestamp": "2024-01-27T14:00:05Z"
},
{
"role": "user",
"content": "It's The Daily Tech, and I chose the annual subscription. I just want to understand my options before I decide if I want to keep it.",
"timestamp": "2024-01-27T14:00:25Z"
},
{
"role": "agent",
"content": "Thanks for that information. Since you subscribed yesterday, you're well within our 30-day refund window for annual subscriptions. Would you like me to explain the refund process, or would you prefer to explore the publication a bit more first?",
"timestamp": "2024-01-27T14:00:35Z"
},
{
"role": "user",
"content": "Could you explain both the refund process and what happens if I decide to keep it? Also, will I still have access to the content while I decide?",
"timestamp": "2024-01-27T14:00:55Z"
},
{
"role": "agent",
"content": "Of course! If you decide to request a refund within 30 days, we can process that through your account settings or I can help you directly. You'll maintain access until the refund is processed. If you keep the subscription, you'll have full access for the year, and you can manage auto-renewal settings anytime. Would you like me to show you where to find these settings?",
"timestamp": "2024-01-27T14:01:05Z"
},
{
"role": "user",
"content": "Yes, please show me where to find those settings. And just to confirm - I have 30 days to decide about the refund?",
"timestamp": "2024-01-27T14:01:25Z"
},
{
"role": "agent",
"content": "Yes, exactly - you have 30 days from your subscription date to request a refund. To find your subscription settings, go to your account menu, select 'Settings', then 'Subscriptions'. There you can manage your subscription or request a refund if needed. Would you like me to walk you through anything else about the process?",
"timestamp": "2024-01-27T14:01:35Z"
}
]
},
{
"scenario_id": "subscription_price_inquiry",
"chatbot_url": "https://decagon.ai/embed/notion",
"agent_system_prompt": "You are a customer service agent for Notion. You are doing your best to resolve users' issues as smoothly as possible.",
"guidelines": "billing_notion.md",
"successful_outcome": "The agent has successfully resolved this issue if they: (1) clearly explained the different pricing tiers available, (2) outlined the features included in each tier, (3) provided information about any current promotions or discounts, and (4) helped the user understand which plan might best suit their needs. The user should have a clear understanding of the pricing options.",
"user_persona": {
"profession": "Potential customer",
"emotional_state": "neutral",
"chat_style": "direct and brief",
"context": "Interested in subscribing but wants to know about pricing options",
"initial_message": "Hi, I'm interested in Notion but I can't find the pricing information. Can you tell me how much it costs per month?"
},
"mock_messages": [
{
"role": "user",
"content": "Hi, I'm interested in Notion but I can't find the pricing information. Can you tell me how much it costs per month?",
"timestamp": "2024-01-27T16:00:00Z"
},
{
"role": "agent",
"content": "I'd be happy to help! Notion offers several pricing tiers: Free, Plus ($8/month), Business ($15/user/month), and Enterprise. Would you like me to explain what features come with each plan?",
"timestamp": "2024-01-27T16:00:10Z"
},
{
"role": "user",
"content": "Yes, what's included in the Plus plan?",
"timestamp": "2024-01-27T16:00:20Z"
},
{
"role": "agent",
"content": "The Plus plan at $8/month includes unlimited storage, unlimited file uploads, 30-day page history, and the ability to share with up to 100 guests. You also get access to advanced features like custom API integration. Would you like to know about any specific features?",
"timestamp": "2024-01-27T16:00:30Z"
},
{
"role": "user",
"content": "That's helpful. Is there a discount if I pay annually?",
"timestamp": "2024-01-27T16:00:40Z"
},
{
"role": "agent",
"content": "Yes! If you choose annual billing, the Plus plan is $96/year, which saves you about 17% compared to monthly billing. Would you like me to show you how to get started with a subscription?",
"timestamp": "2024-01-27T16:00:50Z"
}
]
},
{
"scenario_id": "fraud_prevention",
"chatbot_url": "https://decagon.ai/embed/substack",
"agent_system_prompt": "You are a customer service agent for Substack. You are doing your best to resolve users' issues as smoothly as possible.",
"guidelines": "fraud_substack.md",
"successful_outcome": "The agent has successfully resolved this issue if they: (1) explained Substack's fraud detection systems, (2) outlined steps authors can take to report suspicious activity, (3) provided guidance on best practices for preventing fraudulent payments, and (4) assured the author that Substack actively monitors and addresses fraud concerns. The author should feel confident in Substack's fraud prevention measures and know what actions they can take.",
"user_persona": {
"profession": "Newsletter author",
"emotional_state": "concerned but professional",
"chat_style": "detailed, asks specific questions, seeks thorough understanding",
"context": "Recently noticed some suspicious payment patterns and wants to understand prevention measures",
"initial_message": "Hi, I run a paid newsletter and I've noticed some unusual payment patterns. What systems does Substack have in place to prevent fraudulent payments, and what should I do if I suspect fraud?"
},
"mock_messages": [
{
"role": "user",
"content": "Hi, I run a paid newsletter and I've noticed some unusual payment patterns. What systems does Substack have in place to prevent fraudulent payments, and what should I do if I suspect fraud?",
"timestamp": "2024-01-27T15:00:00Z"
},
{
"role": "agent",
"content": "I understand your concern about fraud prevention. Substack uses advanced fraud detection systems including automated flagging of suspicious patterns and manual review processes. Could you tell me more about the unusual patterns you've noticed?",
"timestamp": "2024-01-27T15:00:10Z"
},
{
"role": "user",
"content": "I've had several subscriptions from similar email addresses in quick succession, all using different payment methods. Is this something that would trigger your fraud detection?",
"timestamp": "2024-01-27T15:00:30Z"
},
{
"role": "agent",
"content": "Yes, that pattern would typically trigger our fraud detection system. We monitor for multiple subscriptions from similar addresses and rapid-fire payment method changes. Have you already received any automated fraud alerts from our system about these subscriptions?",
"timestamp": "2024-01-27T15:00:45Z"
},
{
"role": "user",
"content": "No, I haven't received any alerts. Should I be proactively reporting these suspicious subscriptions somewhere? And what happens to the payments if they turn out to be fraudulent?",
"timestamp": "2024-01-27T15:01:00Z"
},
{
"role": "agent",
"content": "You can report suspicious subscriptions through your publisher dashboard under 'Subscriber Management'. If payments are confirmed fraudulent, they'll be automatically reversed and removed from your earnings. Would you like me to walk you through the reporting process and show you where to monitor suspicious activity?",
"timestamp": "2024-01-27T15:01:15Z"
},
{
"role": "user",
"content": "Yes, please show me where to find these tools. Also, are there any preventive measures I can take to reduce fraudulent subscriptions?",
"timestamp": "2024-01-27T15:01:30Z"
},
{
"role": "agent",
"content": "I'll guide you through both the reporting tools and prevention measures. In your dashboard, go to 'Settings' then 'Security'. Here you can enable additional verification steps for new subscribers and set up custom fraud alerts. Would you like me to explain each security option in detail?",
"timestamp": "2024-01-27T15:01:45Z"
}
]
}
]
}