🏆 CASY - Chatbot Survey Tool

CASY is a chatbot-based survey tool designed to collect insights from technical professionals like data scientists, software developers, and LLM developers. It focuses on gathering survey responses to identify and analyze technical challenges in software development projects.

📋 Project Overview

• Purpose: To enable dynamic and interactive data collection through a chatbot.
• Target Users: Researchers, Master's and PhD students, and HR professionals.
• Participants: Technical professionals (data scientists, developers, etc.).
• Tech Stack:
  • Streamlit (1.39.0)
  • Python
  • Google Cloud

🛠️ Features

• Interactive Chatbot: Conversational survey experience for users.
• Dynamic Insights: Collects responses to analyze technical challenges.
• Dynamic Question Flow: Adapts follow-up questions based on user responses to gather comprehensive data.
• Adversarial Resistance: Maintains survey focus with up to 90% success rate.

💻 Requirements

• Python 3.x
• Required libraries listed in requirements.txt

🚀 How to Run the Project

Install dependencies and run the app using the following commands:

1. Clone the project.
2. Install the required libraries:
   pip install -r requirements.txt
3. Create an access token in Hugging Face.
4. Add access token into .env file in your local folder (check env.example file)
5. Run the app:
   streamlit run app.py

🧩 Core Features

Create a survey session

Join a created survey session

Complete the survey

View details of completed survey sessions

Resume surveys from where you left off

🛠️ Trial Features

Try CASY right away!

Simulate a survey

Due to the nature of our application, it is not possible to create a testing environment using predefined question-answer pairs to measure the chatbot's accuracy. Therefore, we developed a simulation environment that allows us to evaluate the chatbot's performance by generating survey participant agents using real data from technical debt surveys.

In this environment, you can select a participant profile and specify the number of questions the chatbot should ask, then run the survey simulation. To make the testing environment more realistic, we introduced two parameters: Offtopic Answer Weight and Uninformative Answer Weight. Based on these weights, additional agents may take over the survey, producing off-topic or uninformative answers, enabling us to observe how the chatbot handles such situations.

"Log Data" checkbox can be checked to save the simulation data and parameters in your Weights And Biases account (requires authentication).

Further details about the implementation can be found in tests/survey_test.py.

Change chatbot parameters

📜 License

This project is licensed under the MIT License.

👤 Contact

• Owner: Ahmet Bekcan
• Contributors: Sıla Işık, Ceyda Suna
• Email: ahmet.bekcan@metu.edu.tr, isik.sila@metu.edu.tr, ceyda.suna@metu.edu.tr