Recently, text generation models like GPT and T5 have been the talk of the town. However, I wanted to explore the capabilities of text generation using BERT - one of the first large language models that has been instrumental in the progress of NLP. While BERT is primarily known for sentimental and classification tasks, I was curious to see if it could generate a few words or sentences as well.
BERT is indeed capable of generating words within a sentence by predicting masked words in a reasonable manner. However, when it comes to generating full sentences, it faces certain obstacles such as tokenizer limitations and the issue of hallucination.
This repository aims to compile all known techniques that can be employed to generate text of any length using the original BERT core architecture.
I created this small project to experiment with well-known NLP models. There's a lot to learn from attempting tasks with a model that isn't specifically designed for it. Even a little success can be rewarding!
I welcome contributions to this project! Please feel free to add any additional techniques that I may have missed.