Include queue time in benchmark report

[sachanub]

Description

Please read our CONTRIBUTING.md prior to creating your first pull request.

Creating PR to include queue time in the benchmark report.

Tests run:

Ran benchmark locally with resnet50.yaml with batch sizes 1, 2, 4, 8, 16 and concurrency 4. 10000 requests are used for inference.

---
---
resnet50:
    scripted_mode:
        benchmark_engine: "ab"
        url: "file:///home/model-server/torch_compile_ab_benchmarks/resnet50/resnet-50-scripted.mar"
        workers:
            - 4
        batch_delay: 100
        batch_size:
            - 1
            - 2
            - 4
            - 8
            - 16
        requests: 10000
        concurrency: 100
        input: "/serve/examples/image_classifier/kitten.jpg"
        handler_profiling: True
        backend_profiling: True
        exec_env: "local"
        processors:
            - "cpu"
            - "gpus": "all"

The AB report and metrics can be found be in this tar file:
resnet_reports.tar.gz

As evident from the reports, the P50, P90 and P99 of the queue times can be found in the ab_report.csv and stats_metrics.json files.

[agunapal]

LGTM