AI Modelling ( Multi-modal Chatbot )

1. Task

Real-time Text and Image generation corresponding to user's message

2. Model

2.1. Text Generation: OpenAI GPT (Chat) API

• Making a chatting agent with user's pet information (user giving when joining the app).

2.2. Image Generation: Dreambooth Fast Stable Diffusion

• Using pre-training checkpoint: runwayml/stable-diffusion-v1-5
• Finetuning a model with user's 5 pet photos (user giving when joining the app).

3. Dataset (Demo Examples)

Dreambooth(kind of sd finetuning method) makes it possible to finetune the model with only a few images. Therefore, we do not use a large dataset. We only use data of 5 images for finetuning. Finetuning takes about 30 minutes. The finetuning outputs(ckpts) are as follows.

강아지 양파

1. Checkpoint: dhdbsrlw/pet-onion
2. Finetuning Images
(*src: Instagram @pom_onion)

강아지 제로

1. Checkpoint: dhdbsrlw/pet-zero
2. Finetuning Images
(*src: Google)

강아지 몽자

1. Checkpoint: dhdbsrlw/pet-monga
2. Finetuning Images
(*src: Instagram @mongja0408)

4. Generation Output

More generation outputs can be accessed in here !

---

(Last Update: 2024/02/10)
Contact: dhdbsrlw@korea.ac.kr