Skip to content

Commit

Permalink
Merge remote-tracking branch 'Farmerobot/main'
Browse files Browse the repository at this point in the history
# Conflicts:
#	src/llm_postor/game/game_state.py
  • Loading branch information
Luncenok committed Oct 24, 2024
2 parents 6e14fd8 + f9c8e66 commit 7ec6830
Show file tree
Hide file tree
Showing 39 changed files with 250 additions and 182 deletions.
115 changes: 82 additions & 33 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Analysis of Persuasive Capabilities of Large Language Models in Computer Games
# LLMPostor

[![Tests](https://github.com/Farmerobot/mk-ai-agents/actions/workflows/test.yml/badge.svg)](https://github.com/Farmerobot/mk-ai-agents/actions/workflows/test.yml)

Expand All @@ -10,58 +10,108 @@ This project aims to test and compare the persuasive abilities of large language

## Current Features

1. **Web GUI**: A user-friendly interface built using [insert framework used, e.g., Flask, Streamlit] that allows:
- Observation of game progress in real-time.
- Analysis of specific agent actions at various points in the game, including transcripts of conversations and decision-making processes.
- Insight into the thought and decision-making process of LLMs through logging of their internal states (if available from the LLM).
TODO

2. **Interactive Map**: A visual representation of the game environment, showing locations and movements of AI agents, using [insert library used, e.g., Pygame, a JavaScript library].
## Project Architecture

3. **Post-Game Analysis**: After the game concludes, the system provides an in-depth analysis of the persuasive abilities demonstrated by AI agents, including metrics such as:
* Success rate of persuasive attempts.
* Frequency of different persuasive techniques used.
* Correlation between LLM model and persuasive success.
The project is structured into several modules:

4. **"Among Us" Simulation**: The project uses the popular game "Among Us" as a framework to test AI agents' abilities in interaction, deception, and persuasion. The core mechanics of tasks, meetings, voting, and impostors are replicated.
* **`agents`**: Contains implementations of different AI agents, each using a specific LLM.
* **`models`**: Defines data structures for tasks and game state.
* **`players`**: Defines the players.

5. **Multiple AI Agents**: The system supports multiple AI agents, each of which can be powered by a different LLM (e.g., GPT-3, GPT-4, other open-source models), enabling comparative analysis of their persuasive capabilities.
## Table of Contents

## Project Architecture
- Installation
- Usage
- Configuration
- Running Tests
- Contributing
- License

The project is structured into several modules:
## Installation

* **`game_engine`**: Manages the game logic, including task assignment, player interactions, and game state transitions.
* **`agents`**: Contains implementations of different AI agents, each using a specific LLM.
* **`models`**: Defines data structures for players, tasks, and game state.
* **`gui`**: Handles the user interface and visualization.
* **`utils`**: Contains helper functions.
This project requires Python 3.11 or higher.

## Technologies Used
### 1. Install Poetry

* **Python 3.11**: Programming language.
* **[List Libraries/Frameworks Used, e.g., Pydantic, Flask, Pygame, etc.]**: Specific libraries and frameworks used for data modeling, web development, game visualization, etc.
* **OpenAI API**: For accessing large language models.
If you don't have Poetry installed, you can install it by following these steps:

- For Unix/macOS:
```bash
curl -sSL https://install.python-poetry.org | python3 -
```
- For Windows:

## Running Instructions
Visit the official [Poetry installation page](https://python-poetry.org/docs/#installation) for Windows-specific instructions.

Required Python version: 3.11
Once installed, verify Poetry's installation by running:
```bash
poetry --version
```
### 2. Clone the Repository and Install Dependencies

The project uses the `poetry` package manager, which can be installed following the instructions at [https://python-poetry.org/docs/](https://python-poetry.org/docs/)
- Clone the repository:

To install all dependencies using `poetry`, execute the following command:
```bash
poetry install
git clone https://github.com/username/llm_poster.git
cd llm_poster
```

Due to the use of large language models, it is necessary to provide an OpenAI API key. The key can be obtained at [https://beta.openai.com/signup/](https://beta.openai.com/signup/). After obtaining the key, assign it to the `OPENAI_API_KEY` environment variable.
- Install dependencies using Poetry:

To run the demonstration version, execute the following command:
```bash
cd src
poetry run python demo.py
poetry install
```

### 3. Activate the Virtual Environment

Once the dependencies are installed, activate the virtual environment:

```bash
poetry shell
```

## Usage

To run the simulation with the GUI:

```bash
poetry run run-gui
```

## Configuration

The project requires API keys for LLMs such as OpenAI. Set the following environment variables in your shell:

```bash
export OPENAI_API_KEY=your-api-key
```

Alternatively, you can set these in a `.env` file at the project root.

## Running Tests

To run all tests, use the following command:

```bash
poetry run pytest
```

## Contributing

Contributions are welcome! Please follow these steps to contribute:

1. Fork the repository.
2. Create a new branch (git checkout -b feature-branch).
3. Commit your changes (git commit -m 'Add new feature').
4. Push to the branch (git push origin feature-branch).
5. Open a pull request on GitHub.

## License

This project is licensed under the MIT License. See the LICENSE file for details.

## Future Enhancements (Roadmap)

We are planning to add the following features to enhance the project:
Expand All @@ -76,7 +126,6 @@ We are planning to add the following features to enhance the project:
- **API Documentation**: Comprehensive API documentation for integrating custom LLM models or analytical tools.
- **Automated Testing**: Implementation of automated tests to ensure the robustness and reliability of the game engine and AI agents.


These planned features aim to make the project more comprehensive and versatile for researchers and enthusiasts in AI and NLP fields.

This project offers a unique and interactive way to study the persuasive abilities of LLMs in a controlled, game-like environment, providing valuable insights for researchers in the field of artificial intelligence and natural language processing.
2 changes: 1 addition & 1 deletion poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

26 changes: 14 additions & 12 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
[tool.poetry]
name = "mk-ai-agent-test"
name = "llm_postor"
version = "0.1.0"
description = ""
authors = ["Marcin Korcz, Andrii Chmutov, Vasyl Korzavatykh, Mateusz Idziejczak, Mateusz Stawicki"]
description = "This project tests and compares the persuasive abilities of large language models (LLMs) in a game-like environment simulating 'Among Us', where AI agents powered by LLMs interact, make decisions, and attempt to persuade each other."
authors = ["Marcin Korcz", "Andrii Chmutov", "Vasyl Korzavatykh", "Mateusz Idziejczak", "Mateusz Stawicki"]
readme = "README.md"
package-mode = false
package-mode = true

[tool.poetry.dependencies]
python = "^3.11"
numpy = "^1.26.4"
openai = "^1.23.1"
openai = "^1.23.1"
pandas = "^2.2.2"
torch = "^2.2.2"
torchvision = "^0.17.2"
Expand All @@ -22,22 +22,24 @@ llama-index-embeddings-huggingface = "^0.2.0"
ipykernel = "^6.29.4"
ipywidgets = "^8.1.2"
langchain = "^0.3.3"
langchain-community = "^0.3.2"
langchain-community = "^0.3.2"
langchain-openai = "^0.2.2"
langchain-google-genai = "^2.0.1"
streamlit = "^1.39.0"
pytest = "^8.3.3"
st-annotated-text = "^4.0.1"

[tool.pytest.ini_options]
pythonpath = ["src"]
testpaths = ["tests"]

[tool.poetry.group.dev.dependencies]
ipykernel = "^6.29.4"
pytest = "^8.3.3"
pytest-mock = "^3.14.0"

[tool.pytest.ini_options]
pythonpath = ["src"]
testpaths = ["tests"]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
build-backend = "poetry.core.masonry.api"

[tool.poetry.scripts]
run-gui = "scripts.run_gui:run_gui"
19 changes: 19 additions & 0 deletions scripts/run_gui.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import os
import subprocess

def run_gui():
# Define the project root directory (this assumes the script is located in the scripts/ folder)
project_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))

# Navigate to the project root directory
os.chdir(project_root)

# Define the path to the demo_gui.py script relative to the project root
demo_gui_path = os.path.join(project_root, "src", "llm_postor", "demo_gui.py")

# Check if the demo_gui.py file exists
if not os.path.exists(demo_gui_path):
raise FileNotFoundError(f"File does not exist: {demo_gui_path}")

# Run the Streamlit app with the correct path
subprocess.run(["poetry", "run", "streamlit", "run", demo_gui_path], check=True)
14 changes: 7 additions & 7 deletions scripts/run_gui.sh
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
#!/bin/bash
#!/bin/bash

# Get the directory of the current script
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Get the directory of the current script
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

# Navigate to the project root (assuming this script is located inside `scripts/` directory)
cd "$SCRIPT_DIR/.."
# Navigate to the project root (assuming this script is located inside `scripts/` directory)
cd "$SCRIPT_DIR/.."

# Run the Streamlit app in the Poetry environment
poetry run streamlit run src/demo_gui.py
# Run the Streamlit app in the Poetry environment
poetry run streamlit run src/demo_gui.py
File renamed without changes.
4 changes: 2 additions & 2 deletions src/annotation.py → src/llm_postor/annotation.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
import os
import json
from typing import List

from langchain_openai import ChatOpenAI
from game.llm_prompts import ANNOTATION_TEMPLATE
from langchain.schema import HumanMessage

from llm_postor.game.llm_prompts import ANNOTATION_TEMPLATE

def annotate_dialogue(dialogue: str, llm_model_name: str = "gpt-4o-mini") -> str:
"""
Annotates a dialogue with persuasion techniques using OpenAI API.
Expand Down
22 changes: 9 additions & 13 deletions src/demo_gui.py → src/llm_postor/demo_gui.py
Original file line number Diff line number Diff line change
@@ -1,26 +1,22 @@
import json
import time
from openai import OpenAIError
import streamlit as st
from game.game_engine import GameEngine
from game.players.ai import AIPlayer
from game.gui_handler import GUIHandler
from game.game_state import GameState
from openai import OpenAIError
from types import SimpleNamespace as Namespace
from game.players.fake_ai import FakeAIPlayer
from game.models.engine import GamePhase
from game.chat_analyzer import ChatAnalyzer


from llm_postor.game.game_engine import GameEngine
from llm_postor.game.players.ai import AIPlayer
from llm_postor.game.gui_handler import GUIHandler
from llm_postor.game.game_state import GameState
from llm_postor.game.players.fake_ai import FakeAIPlayer
from llm_postor.game.models.engine import GamePhase
from llm_postor.game.chat_analyzer import ChatAnalyzer

# To run this script, you need to poetry install and then run the following command:
# streamlit run src/demo_gui.py




def main():
st.title("Among Us Game - susGPT")
st.title("Among Us Game - LLMPostor")
gui_handler = GUIHandler()
game_engine = GameEngine()

Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
import re
from typing import List, Any

from langchain_openai import ChatOpenAI

from game.consts import ASCII_MAP
from .base_agent import Agent
from langchain.schema import HumanMessage
from game.llm_prompts import ADVENTURE_PLAN_TEMPLATE, ADVENTURE_ACTION_TEMPLATE
import re
from langchain_openai import ChatOpenAI

from llm_postor.game.agents.base_agent import Agent
from llm_postor.game.consts import ASCII_MAP
from llm_postor.game.llm_prompts import ADVENTURE_PLAN_TEMPLATE, ADVENTURE_ACTION_TEMPLATE

class AdventureAgent(Agent):
def update_state(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@
from typing import Any, List
from pydantic import BaseModel, Field

from game.agents.usage_metadata import UsageMetadata
from game.consts import TOKEN_COSTS

from llm_postor.game.agents.usage_metadata import UsageMetadata
from llm_postor.game.consts import TOKEN_COSTS

class AgentState(BaseModel):
history: str = Field(default_factory=str)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
from typing import List, Any
from .base_agent import Agent
from langchain.schema import HumanMessage
from game.llm_prompts import DISCUSSION_TEMPLATE, DISCUSSION_RESPONSE_TEMPLATE

from llm_postor.game.llm_prompts import DISCUSSION_TEMPLATE
from llm_postor.game.llm_prompts import DISCUSSION_RESPONSE_TEMPLATE

class DiscussionAgent(Agent):
def update_state(
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import re
from typing import List, Any
from .base_agent import Agent
from langchain.schema import HumanMessage
from game.llm_prompts import VOTING_TEMPLATE
import re

from llm_postor.game.llm_prompts import VOTING_TEMPLATE

class VotingAgent(Agent):
def update_state(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
import re
from typing import List, Dict
from pydantic import BaseModel
from game.players.base_player import Player
from langchain_openai import ChatOpenAI
from game.llm_prompts import PERSUASION_TECHNIQUES
from langchain.schema import HumanMessage
import re

from llm_postor.game.players.base_player import Player
from llm_postor.game.llm_prompts import PERSUASION_TECHNIQUES

class ChatAnalyzer(BaseModel):
players: List[Player]
Expand Down
File renamed without changes.
Loading

0 comments on commit 7ec6830

Please sign in to comment.