Add Benchmarking Compatibility to PaddingFree Plugin

[achew010]

Description

This PR provides support to run the benchmark for the AADP plugin

Key items:

• data_processing stanza in scenarios.yaml to configure the benchmark dataset and define formatting using Jinja template
• modifications to benchmark.py
• additional AADP sample configurations

Benchmarks (Mistral 7B)

Benchmarks on Flan shows the improvement from PF is consistent with Transformers

• 46% improvement in training runtime with PaddingFree for full FT
• 20% improvement in training runtime with PaddingFree for QPEFT

NOTE: there are some regressions that affected the below results #70

Subset of Flan

run bash scripts/run_benchmarks.sh "1 2" "benchmark_outputs/flan" "scenarios-flan.yaml "none""

Framework Type	Num Device	Device Batch Size	Train Runtime (sec)	Throughput (toks/sec)	% Runtime Improvement
Full FT	2	4	1430	992	base
Full FT + PF	2	4	774	1875	+46
BNB	2	4	1612	880	base
BNB + PF	2	4	1271	1142	+21
BNB + FOAK	2	4	1068	1328	base
BNB + FOAK + PF	2	4	605	2400	+43
GPTQ	2	4	1569	904	base
GPTQ + PF	2	4	1255	1156	+20
GPTQ + FOAK	2	4	1034	1372	base
GPTQ + FOAK + PF	2	4	587	2472	+43

Subset of Orca-Math

run bash scripts/run_benchmarks.sh "1 2" "benchmark_outputs/orca" "scenarios-orca.yaml "none""
Benchmarks on Orca-Math show improvement from PaddingFree matches Transformers

• 23% improvement in training runtime with PaddingFree on full FT
• PaddingFree however doesn't show improvement in train runtime with QPEFT but there is significant improvement when FOAK is applied, the issue is documented in Inconsistency in Padding-Free Benchmarks with Different Transformers Versions #70

Single Device

Framework Type	Num Device	Device Batch Size	Train Runtime (sec)	Throughput (toks/sec)	% Runtime Improvement
Full FT	1	4	362	1520	base
Full FT + PF	1	4	291	1874	+19

Two Device

Framework Type	Num Device	Device Batch Size	Train Runtime (sec)	Throughput (toks/sec)	% Runtime Improvement
Full FT	2	4	231	1188	base
Full FT + PF	2	4	177	1584	+23
BNB	2	4	392	709	base
BNB + PF	2	4	398	704	+0
BNB + FOAK	2	4	190	1468	base
BNB + FOAK + PF	2	4	155	1802	+18
GPTQ	2	4	388	720	base
GPTQ + PF	2	4	386	725	+0
GPTQ + FOAK	2	4	186	1359	base
GPTQ + FOAK + PF	2	4	158	1771	+15

[fabianlim]

have questions and concerns

[fabianlim]

Please fix the DCO

[fabianlim]

@achew010 if we figure out what is the issue and if it requires some significant change in the design of the accelerated-peft plugin, this needs to be documented