KakaoBrain project KoGPT

• KakaoBrain project KoGPT (Korean Generative Pre-trained Transformer)
  • https://github.com/kakaobrain/kogpt
  • https://huggingface.co/kakaobrain/kogpt

Model Descriptions

KoGPT6B-ryan1.5b

• [huggingface][kakaobrain/kogpt][KoGPT6B-ryan1.5b]
• [huggingface][kakaobrain/kogpt][KoGPT6B-ryan1.5b-float16]

Hyperparameter	Value
	6,166,502,400
	28
	4,096
	16,384
	16
	256
	2,048
	64,512
Positional Encoding	Rotary Position Embedding (RoPE)
RoPE Dimensions	64

Hardware requirements

KoGPT6B-ryan1.5b

GPU

The following is the recommended minimum GPU hardware guidance for a handful of example KoGPT.

• 32GB GPU RAM in the required minimum memory size

KoGPT6B-ryan1.5b-float16

GPU

The following is the recommended minimum GPU hardware guidance for a handful of example KoGPT.

• half-precision requires NVIDIA GPUS based on Volta, Turing or Ampere
• 16GB GPU RAM in the required minimum memory size

Usage

prompt