This project is built in order to explore the natural language processing field and understand the working of transformers and their uses to develop various projects like language models, text summarization, text classification and more. We have developed sentimental classification model, word vectors for nepali words, probabilistic language model and GPT2-based language model for nepali language in this project and the work is on going.
- To develop nepali language model using probabilistic and sequential model.
- Explore the areas of word embeddings and classification of nepali texts.
- Develop a spelling correction model of nepali texts
Frontent is developed using React and Backend is developed using django and django rest framework.
- Currently working on.
-
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems (I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, eds.), vol. 30, Curran Associates, Inc., 2017.
-
Y. Bengio, R. Ducharme, and P. Vincent, “A neural probabilistic language model,” Advances in neural information processing systems, vol. 13, 2000.
-
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” Advances in neural information processing systems, vol. 26, 2013.
-
S. Timilsina, M. Gautam, and B. Bhattarai, “Nepberta: Nepali language model trained in a large corpus,” in Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, pp. 273–284, 2022.