Skip to content

Latest commit

 

History

History
92 lines (66 loc) · 2.67 KB

README.md

File metadata and controls

92 lines (66 loc) · 2.67 KB
Raw

Scheduling Simulator

Scheduling simulator is based on runtime data and prediction results located at prediction/final/.

Usage

$ python3 simulator.py [-h] -n NUM_JOBS -a ALGO -d DISTRIBUTION [-r RATE] [-s STD] [-c COEFFICIENT_OF_VARIATION] [--llm] [-m MODEL] [--per_token_latency LATENCY] [--const_latency LATENCY]

Required Arguments:

  • -n: Number of jobs to create
  • -a: Scheduling algorithm, chosen from FCFS and SJF
  • -d: Job arrival distribution, chosen from Poisson, Uniform, and Gamma

Optional Arguments:

  • -r: Average job arrival rate, default 1
  • -s: Standard deviation of job arrival rate, used for Unifrom distribution, default 1
  • -c: Coefficient of variance of job arrival rate, used for Gamma distribution, default 2
  • --llm: Create jobs for LLM
  • -m: LLM model
  • --per_token_latency: Average per token generation latency for the LLM model, default 0.02
  • --const_latency: Constant latency for token generation, amortized across all generated tokens for the LLM model, default 0.1

Batching-related parameters:

MAX_BATCH_SIZE = 1
BATCH_WAIT_TIMEOUT = 5

No Batching

To run the simulator with no batching, simply set the MAX_BATCH_SIZE to 1.

Static Batching

To run the simulator with static batching, simply set the MAX_BATCH_SIZE > 1 and choose a BATCH_WAIT_TIMEOUT based on the expected batch size and the average job duration.

Dynamic Batching

To run the simulator with dynamic batching, set the DYNAMIC_BATCHING = True with MAX_BATCH_SIZE > 1. Dynamic batching is also called continuous batching. Rebatching is performed when at least 1 job in the batch has finished. The benefit is that the early-finished jobs in a batch don't need to wait for the remaining jobs in the batch to finish.

Evaluation

Configure the parameters in util.py; an example could be:

LLM_DATA_PATH = 'prediction/final/predictions_all.csv'  # multiple predictors
CLASSIFIERS = ['reg-l1', 'reg-mse', 'cls', 'multi-cls', 'multi-cls-l1', 'multi-cls-mse']
RESULTS_DIR = 'results/cleaned_datasets_no_batching_single_round/'

Run the evaluation script and plot the results on a barplot:

No Batching

Set these parameters in util.py:

  • MODE = 'NO_BATCHING'
  • MAX_BATCH_SIZE = 1
  • DYNAMIC_BATCHING = False
python auto_eval.py
python auto_eval_barplot.py

Static Batching

Set these parameters in util.py:

  • MODE = 'STATIC_BATCHING'
  • MAX_BATCH_SIZE = 2
  • DYNAMIC_BATCHING = False
python auto_eval.py
python auto_eval_barplot.py

Dynamic Batching

Set these parameters in util.py:

  • MODE = 'DYNAMIC_BATCHING'
  • MAX_BATCH_SIZE = 2
  • DYNAMIC_BATCHING = True
python auto_eval.py
python auto_eval_barplot.py