SAEM: Managing Memories, and SAM: Capturing Memories

🥈 2024 Elice AI Hellothon Excellence Award (2nd Place)

Theme: Putting Heart into AI

In the era of artificial intelligence, focusing on human "empathy," "emotions," and other heartfelt aspects.

1. Project Overview

1.1 Project Title

SAEM: Managing Memories, and SAM: Capturing Memories

SAEM: Managing Memories

SAM: Capturing Memories

1.2 Project Logo

1.3 Project Information

Development Background

• The rapid growth of the elderly population, transitioning into a super-aged society, and increasing dementia cases.
• A growing demand for caregivers and a shortage of personnel.
• By 2025, South Korea is expected to become a super-aged society, where over 20% of the total population (more than 10 million people) will be elderly.
• Caregivers are required by regulation to conduct cognitive activities at least 3 times a week or 12 times a month.
• However, the current number of caregivers is only 600,000, far from meeting the demand.

Development Goal

• Provide a simple voice-based journaling system for seniors as part of their cognitive activities.
• Build a system to help caregivers efficiently manage seniors’ statuses and seamlessly facilitate cognitive activity education.

SAM: Capturing Memories

• A service enabling caregivers to create customized cognitive activity guides based on seniors' records.
• Check dementia progression through tailored cognitive activities.

SAEM: Managing Memories

• A voice-based memo service for seniors to record their memories and daily life.
• Activities like daily journaling and reminiscing can stimulate cognitive function, contributing to dementia prevention or slowing its progression.
• Supports heartfelt experiences through senior-friendly prompts.

Application

• Caregivers can use seniors’ records to smoothly conduct personalized cognitive activities.
• Seniors’ conditions can be monitored via activity similarity analysis, and cognitive activities can help prevent or slow dementia progression.

Expected Benefits

• Seniors can maintain cognitive function by recording their daily lives.
• Caregivers can streamline their daily tasks and effectively manage seniors' conditions, alleviating personnel shortages to some extent.

1.4 Development Period (2 weeks)

⏰ November 10, 2024 – November 24, 2024

1.5 Team Introduction

AI Alice Hellothon 2024 Team 1

👩‍💻 Yukyung Shim (PM)

👩‍💻 Seungjae Lim (BE)

👩‍💻 Seokhun Eom (FE)

👩‍💻 Songyi Park (Designer)

1.6 Demo Video (Optional)

🎞 Watch Demo Video Here

2. Project Architecture

2.1 System Architecture Diagram

2.2 Use Case Flow Charts

2.3 ERD (Entity Relationship Diagram)

2.4 API Documentation

SAM API Documentation

3. Tech Stack

3. Tech Stack

🌐 Languages

🎨 Frontend

🔧 Backend

🗄️ Database

⚙️ DevOps & Deployment

📦 Package Managers

🤝 Collaboration Tools

🛠️ Development Tools

4. Screenshots

4.1 SAM: Managing Memories

Main Page	Preparing Records	Guide Creation
Cognitive Activity (Before)	Cognitive Activity (During)	Report
:------------:	:------------:	:------------:

4.2 SAM: Capturing Memories

Main Page	Record Memories	My Memories

5. Feature Descriptions

5.1 SAM: Managing Memories

5.1.1 Main Page

• Management page for seniors’ cognitive activity records, guide creation, and education.

5.1.2 Preparing Records Page

• Prepares records necessary for creating cognitive activity guides.

5.1.3 Guide Creation Page

• AI generates questions for cognitive activity based on records, with manual question addition.

5.1.4 Cognitive Activity Page

• Conduct cognitive activity sessions using generated guide questions.

5.1.5 Report Page

• Automatically generates reports upon completing 3 sessions weekly.

5.2 SAM: Capturing Memories

5.2.1 Main Page

• Displays a list of recorded memories.

5.2.2 Record Memories Page

• Allows seniors to journal daily lives and memories as cognitive stimulation.

5.2.3 My Memories Page

• AI generates personalized images and keywords based on recorded memories.

6. How to Run

6.1. Backend Setup

6.1.1. Run FastAPI Server

1. Clone the Repository:

git clone https://github.com/SeungjaeLim/SaemSam.git
cd SaemSam

2. Set Up the Environment:

python -m venv venv
source venv/bin/activate  # On Windows, use venv\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt

4. Run FastAPI server:

uvicorn main:app --host 0.0.0.0 --port 8000 --reload

6.1.2. Create the Database:

1. Start your MySQL server.

2. Create the database and tables using the following commands:

CREATE DATABASE saem CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
USE saem;

-- Table: elders
CREATE TABLE elders (
    id INT AUTO_INCREMENT PRIMARY KEY,
    name VARCHAR(255) NOT NULL,
    birth_date DATE NOT NULL,
    gender ENUM('M', 'F') NOT NULL,
    care_level ENUM('1', '2', '3', '4', '5') NOT NULL,
    contact_info VARCHAR(255),
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Table: activity_guides
CREATE TABLE activity_guides (
    id INT AUTO_INCREMENT PRIMARY KEY,
    elder_id INT NOT NULL,
    title VARCHAR(255) NOT NULL,
    have_studied BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (elder_id) REFERENCES elders(id)
);

-- Table: guide_questions
CREATE TABLE guide_questions (
    id INT AUTO_INCREMENT PRIMARY KEY,
    guide_id INT NOT NULL,
    question_id INT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (guide_id) REFERENCES activity_guides(id),
    FOREIGN KEY (question_id) REFERENCES questions(id)
);

-- Table: questions
CREATE TABLE questions (
    id INT AUTO_INCREMENT PRIMARY KEY,
    text TEXT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Table: answers
CREATE TABLE answers (
    id INT AUTO_INCREMENT PRIMARY KEY,
    question_id INT NOT NULL,
    elder_id INT NOT NULL,
    response TEXT NOT NULL,
    response_date DATE NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (question_id) REFERENCES questions(id),
    FOREIGN KEY (elder_id) REFERENCES elders(id)
);

-- Table: records
CREATE TABLE records (
    id INT AUTO_INCREMENT PRIMARY KEY,
    title VARCHAR(255) NOT NULL,
    content TEXT,
    elder_id INT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (elder_id) REFERENCES elders(id)
);

-- Table: record_questions
CREATE TABLE record_questions (
    id INT AUTO_INCREMENT PRIMARY KEY,
    record_id INT NOT NULL,
    question_id INT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (record_id) REFERENCES records(id),
    FOREIGN KEY (question_id) REFERENCES questions(id)
);

-- Table: record_keywords
CREATE TABLE record_keywords (
    id INT AUTO_INCREMENT PRIMARY KEY,
    record_id INT NOT NULL,
    keyword_id INT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (record_id) REFERENCES records(id),
    FOREIGN KEY (keyword_id) REFERENCES keywords(id)
);

-- Table: keywords
CREATE TABLE keywords (
    id INT AUTO_INCREMENT PRIMARY KEY,
    keyword VARCHAR(255) UNIQUE NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Table: keyword_preferences
CREATE TABLE keyword_preferences (
    id INT AUTO_INCREMENT PRIMARY KEY,
    elder_id INT NOT NULL,
    keyword_id INT NOT NULL,
    is_preferred BOOLEAN DEFAULT TRUE,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (elder_id) REFERENCES elders(id),
    FOREIGN KEY (keyword_id) REFERENCES keywords(id)
);

-- Table: images
CREATE TABLE images (
    id INT AUTO_INCREMENT PRIMARY KEY,
    record_id INT NOT NULL,
    url TEXT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (record_id) REFERENCES records(id)
);

-- Table: analyses
CREATE TABLE analyses (
    id INT AUTO_INCREMENT PRIMARY KEY,
    elder_id INT NOT NULL,
    question_id INT NOT NULL,
    first_answer_id INT NOT NULL,
    last_answer_id INT NOT NULL,
    similarity FLOAT NOT NULL,
    report_id INT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (elder_id) REFERENCES elders(id),
    FOREIGN KEY (question_id) REFERENCES questions(id),
    FOREIGN KEY (first_answer_id) REFERENCES answers(id),
    FOREIGN KEY (last_answer_id) REFERENCES answers(id),
    FOREIGN KEY (report_id) REFERENCES reports(id)
);

-- Table: reports
CREATE TABLE reports (
    id INT AUTO_INCREMENT PRIMARY KEY,
    elder_id INT NOT NULL,
    year INT NOT NULL,
    week_number INT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (elder_id) REFERENCES elders(id)
);

-- Table: tasks
CREATE TABLE tasks (
    id INT AUTO_INCREMENT PRIMARY KEY,
    elder_id INT NOT NULL,
    year INT NOT NULL,
    week_number INT NOT NULL,
    status INT DEFAULT 0,
    iteration INT DEFAULT 0,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (elder_id) REFERENCES elders(id)
);

6.1.3. Running VLLM Servers for LLM and Embedding

1. Navigate to the server Directory:

cd server

2. Run the VLLM Servers: Start the LLM and embedding servers using the script:

./vllm_serve.sh

• The script will:
  • Create a tmux session named Ssaem.
  • Start two VLLM servers:
    • Model 1: meta-llama/Meta-Llama-3-70B-Instruct
    • Model 2: BAAI/bge-multilingual-gemma2
  • Automatically split the tmux panes and attach to the session.

6.2. Frontend Setup

1. Navigate to the Frontend Directory:

cd fe

2. Ensure pnpm is installed. If not, install it first:

npm install -g pnpm

3. Install frontend dependencies:

pnpm install

4. Run the Frontend Development Server:

pnpm dev --host 0.0.0.0 --port 5173

7. Reflections

7.1 Team Retrospective

• Defining the target audience as seniors and caregivers using the 5W1H method helped focus and detail our service planning.
• Realized the diverse nature of seniors beyond just age brackets and the importance of cognitive activities in combating dementia.
• Although we implemented a voice-based service, it could expand into multimedia formats like photos or videos for richer memory capturing.

7.2 Individual Reflections

👩‍💻 Songyi Park

👩‍💻 Yukyung Shim

👩‍💻 Seokhun Eom

👩‍💻 Seungjae Lim