Skip to content

Latest commit

 

History

History
36 lines (28 loc) · 1.37 KB

README.md

File metadata and controls

36 lines (28 loc) · 1.37 KB
Raw

This repository contains the source code and resources for an LLM-based mathematical problem solver. It combines the generative capabilities of large language models with the discriminative power of classical machine learning techniques.

Project Structure

  • inference.py: Contains the main inference pipeline for solving mathematical problems using the two-stage approach.
  • qlora_training.py: Script for fine-tuning the base language model using QLoRA (Quantized Low-Rank Adaptation).
  • statistical_features_classifier.py: Trains the second-stage logistic regression classifier using aggregated statistics from the QLoRA model's outputs.
  • model/: Directory containing the base language model and fine-tuned QLoRA adapters.
  • data/: Contains training, validation, and test datasets.

Setup

  1. Clone the Repository: 
    git clone https://github.com/yourusername/AIMO.git
cd AIMO
  2. Install Dependencies: 
    pip install -r requirements.txt
  3. Download the Base Model: 
    Download the base language model (e.g., DeepSeek-Math-7B-RL) and place it in the model/ directory.

Running the Project

  1. Fine-tune the QLoRA Model: 
    python qlora_training.py
  2. Train the statistical features classifier: 
    python aggregated_statistics_classifier.py
  3. Run inference: 
    python inference.py