From Small to Mighty: Advancing Math Reasoning in SLMs

Overview

This project explores techniques to enhance mathematical reasoning in Small Language Models (SLMs) like T5-small. We investigate and implement methods such as:

• Low-Rank Adaptation (LoRA)
• LoRA with Stacked Layers
• QLoRA

Artifacts:

• Poster: Poster_Advancing_Math_Reasoning_SLMs.pdf
• Report: Report_Advancing_Math_Reasoning_SLMs.pdf

---

Setup and Reproducibility

Install Dependencies

1. Clone the repository:

   git clone https://github.com/saikaryekar/math-reasoning-slms.git
   cd math-reasoning-slms

2. Create and activate the environment using the environment.yml file:

   conda env create -f environment.yml
   conda activate proj

---

Contributors

• Sai Paresh Karyekar
• Sharika Menon
• Alyan Khan

---