- KakaoBrain project KoGPT (Korean Generative Pre-trained Transformer)
- [huggingface][kakaobrain/kogpt][KoGPT6B-ryan1.5b]
- [huggingface][kakaobrain/kogpt][KoGPT6B-ryan1.5b-float16]
Hyperparameter | Value |
---|---|
6,166,502,400 | |
28 | |
4,096 | |
16,384 | |
16 | |
256 | |
2,048 | |
64,512 | |
Positional Encoding | Rotary Position Embedding (RoPE) |
RoPE Dimensions | 64 |
The following is the recommended minimum GPU hardware guidance for a handful of example KoGPT.
32GB GPU RAM
in the required minimum memory size
The following is the recommended minimum GPU hardware guidance for a handful of example KoGPT.
- half-precision requires NVIDIA GPUS based on Volta, Turing or Ampere
16GB GPU RAM
in the required minimum memory size