The repo is based on gpt-tfjs and its TypeScript fork by lukemovement. A lot of models share common layers and preprocessing methods so it makes sense to continue the work in a single repo rather than copy-pasting the same transformer architecture over and over again or having to maintain multiple repos for each layer. Currently not a lot to see here.
🔴 Work in progress 🔴
- GPT
- Vision Transformer
- CLIP - in progress
- Logging
- Range
- Positional embedding
- Class embedding
- Slice
- GELU
- QuickGELU
- Multihead attention
- MLP
- Residual Attention block
- Transformer
- AdamW