A Pytorch LSTM Language model trained on Penn Tree Bank Dataset.
- Introduction
- Setup
- Run the code
- Training
- Sample Generation
- Play with the model
- Connect with me
- License
This is a PyTorch Implementation for an LSTM-based language model. The model is trained on Penn Tree Bank dataset using Adam optimizer with a learning rate 0.001
and 10 epochs.
The code is using pipenv
as a virtual environment and package manager. To run the code, all you need is to install the necessary dependencies. open the terminal and type:
git clone https://github.com/Khamies/LSTM-Language-Generator.git
cd LSTM-Language-Generator
pipenv install
And you should be ready to go to play with code and build upon it!
-
To train the model, run:
python main.py
-
To train the model with specific arguments, run:
python main.py --batch_size=64
. The following command-line arguments are available:- Batch size:
--batch_size
- bptt:
--bptt
- Learning rate:
--lr
- Embedding size:
--embed_size
- Hidden size:
--hidden_size
- Latent size:
--latent_size
- Batch size:
The model is trained on 10 epochs
using Adam as an optimizer with a learning rate = 0.001
and batch size = 32
, you can find all the model settings in settings.py. Here is the loss curve for the training step:
Here are some generated samples from the model:
he said <pad> is n't expected to be the first quarter of
the company said will be used to repay the company's
To play with the model, a jupyter notebook has been provided, you can find it here
@misc{Khamies2021LSTM-Language-Generator, author = {Khamies, Waleed}, title = {A PyTorch Implementation for an LSTM-based language model}, year = {2021}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/Khamies/LSTM-Language-Generator}}, }
For any question or a collaboration, drop me a message here
Follow me on Linkedin!
Thank you ❤️