Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

benchmark.py ERROR: TypeError: cannot concatenate 'str' and 'NoneType' objects #242

Closed
ysh329 opened this issue Jan 12, 2018 · 7 comments
Closed

benchmark.py ERROR: TypeError: cannot concatenate 'str' and 'NoneType' objects #242

ysh329 opened this issue Jan 12, 2018 · 7 comments
Labels
correctness

Comments

@ysh329
Copy link

ysh329 commented Jan 12, 2018 

gpu@gpu-FP4:~/yuanshuai/code/CLBlast/scripts/benchmark$ python benchmark_all.py -p 0 -d 1
Traceback (most recent call last):
  File "benchmark_all.py", line 45, in <module>
    benchmark_all(**parsed_arguments)
  File "benchmark_all.py", line 39, in benchmark_all
    plot_title, tight_plot, output_folder, verbose)
  File "/home/yuanshuai/code/CLBlast/scripts/benchmark/benchmark.py", line 105, in benchmark_single
    plot_title = benchmark_name if plot_title is "" else benchmark_name + ": " + plot_title
TypeError: cannot concatenate 'str' and 'NoneType' objects
@ysh329 ysh329 changed the title TypeError: cannot concatenate 'str' and 'NoneType' objects benchmark.py ERROR: TypeError: cannot concatenate 'str' and 'NoneType' objects Jan 12, 2018
@CNugteren
Copy link
Owner

Does it help if you pass the -t some_title option to the script?

@ysh329
Copy link
Author

ysh329 commented Jan 13, 2018

I tried again using command python benchmark.py -b gemm -p 0 -d 0, but error as below:

gpu@gpu-FP4:~/yuanshuai/code/CLBlast/scripts/benchmark$ python benchmark.py -b gemm -p 0 -d 0                                                                                                                                [5/553]
[benchmark] Running on platform 0, device 0
[benchmark] Running 6 benchmarks for settings 'gemm'
[benchmark] Running benchmark 'gemm:multiples of 128'
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 128 -layout 102 -num_steps 20 -k 128 -m 128 -transA 111 -transB 111 -n 128
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Running benchmark 'gemm:multiples of 129'
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 129 -layout 102 -num_steps 20 -k 129 -m 129 -transA 111 -transB 111 -n 129
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Running benchmark 'gemm:around 512'
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 1 -layout 102 -num_steps 16 -k 512 -m 512 -transA 111 -transB 111 -n 512
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Running benchmark 'gemm:around 2048'
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 1 -layout 102 -num_steps 16 -k 2048 -m 2048 -transA 111 -transB 111 -n 2048
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Running benchmark 'gemm:layouts/transpose'
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 101 -num_steps 1 -k 1024 -m 1024 -transA 111 -transB 111 -n 1024
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 101 -num_steps 1 -k 1024 -m 1024 -transA 111 -transB 112 -n 1024
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 101 -num_steps 1 -k 1024 -m 1024 -transA 112 -transB 111 -n 1024
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 101 -num_steps 1 -k 1024 -m 1024 -transA 112 -transB 112 -n 1024
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 1024 -m 1024 -transA 111 -transB 111 -n 1024
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 1024 -m 1024 -transA 111 -transB 112 -n 1024
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 1024 -m 1024 -transA 112 -transB 111 -n 1024
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 1024 -m 1024 -transA 112 -transB 112 -n 1024
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Running benchmark 'gemm:powers of 2'
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 8 -m 8 -transA 111 -transB 111 -n 8
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 16 -m 16 -transA 111 -transB 111 -n 16
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 32 -m 32 -transA 111 -transB 111 -n 32
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 64 -m 64 -transA 111 -transB 111 -n 64
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 128 -m 128 -transA 111 -transB 111 -n 128
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 256 -m 256 -transA 111 -transB 111 -n 256
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 512 -m 512 -transA 111 -transB 111 -n 512
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 1024 -m 1024 -transA 111 -transB 111 -n 1024
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 2048 -m 2048 -transA 111 -transB 111 -n 2048
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Calling binary: ./clblast_client_xgemm -platform 0 -device 0 -precision 32 -runs 10 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -step 0 -layout 102 -num_steps 1 -k 4096 -m 4096 -transA 111 -transB 111 -n 4096
/bin/sh: 1: ./clblast_client_xgemm: not found
[benchmark] Saving benchmark results to '/home/yuanshuai/code/CLBlast/scripts/benchmark/sgemm_benchmarks.json'
[plot] Plotting subplot 0
Traceback (most recent call last):
  File "benchmark.py", line 175, in <module>
    benchmark_single(**parsed_arguments)
  File "benchmark.py", line 168, in benchmark_single
    label_names, plot_title, tight_plot, verbose)
  File "/home/yuanshuai/code/CLBlast/scripts/benchmark/plot.py", line 89, in plot_graphs
    y_max = max([max(y) for y in y_list])
ValueError: max() arg is an empty sequence

./clblast_client_xgemm: not found, Does this mean my build progress is not complete?

@ysh329
Copy link
Author

ysh329 commented Jan 13, 2018

Similar error (./clblast_client_xaxpy: not found) when executing command: python benchmark_all.py -p 0 -d 0 -t some_title:

gpu@gpu-FP4:~/yuanshuai/code/CLBlast/scripts/benchmark$ python benchmark_all.py -p 0 -d 0 -t some_title
[benchmark] Running on platform 0, device 0
[benchmark] Running 6 benchmarks for settings 'axpy'
[benchmark] Running benchmark 'axpy:multiples of 256K'
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 262144 -num_steps 16 -n 262144
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Running benchmark 'axpy:multiples of 256K+1'
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 262145 -num_steps 16 -n 262145
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Running benchmark 'axpy:around 1M'
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 1 -num_steps 16 -n 1048576
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Running benchmark 'axpy:around 16M'
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 1 -num_steps 16 -n 16777216
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Running benchmark 'axpy:strides n=8M'
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 2 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 4 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 2 -incy 1 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 2 -incy 2 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 2 -incy 4 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 4 -incy 1 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 4 -incy 2 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 20 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 4 -incy 4 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Running benchmark 'axpy:powers of 2'
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 32768
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 65536
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 131072
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 262144
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 524288
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 1048576
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 2097152
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 4194304
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 8388608
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 16777216
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 33554432
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Calling binary: ./clblast_client_xaxpy -platform 0 -device 0 -precision 32 -runs 40 -warm_up -q -no_abbrv -clblas 0 -cblas 0 -cublas 0 -incx 1 -incy 1 -step 0 -num_steps 1 -n 67108864
/bin/sh: 1: ./clblast_client_xaxpy: not found
[benchmark] Saving benchmark results to '/home/yuanshuai/code/CLBlast/scripts/benchmark/saxpy_benchmarks.json'
[plot] Plotting subplot 0
Traceback (most recent call last):
  File "benchmark_all.py", line 45, in <module>
    benchmark_all(**parsed_arguments)
  File "benchmark_all.py", line 39, in benchmark_all
    plot_title, tight_plot, output_folder, verbose)
  File "/home/yuanshuai/code/CLBlast/scripts/benchmark/benchmark.py", line 168, in benchmark_single
    label_names, plot_title, tight_plot, verbose)
  File "/home/yuanshuai/code/CLBlast/scripts/benchmark/plot.py", line 89, in plot_graphs
    y_max = max([max(y) for y in y_list])
ValueError: max() arg is an empty sequence

@ysh329
Copy link
Author

ysh329 commented Jan 13, 2018 

gpu@gpu-FP4:~/yuanshuai/code/CLBlast/build$ ./clblast_tuner_xgemm
* Options given/available:
    -platform 0 [=default]
    -device 0 [=default]
    -precision 32 (single) [=default]
    -m 1024 [=default]
    -n 1024 [=default]
    -k 1024 [=default]
    -alpha 2.00 [=default]
    -beta 2.00 [=default]
    -fraction 1.00 [=default]
    -runs 2 [=default]
    -max_l2_norm 0.00 [=default]

* Found 578 configuration(s)
* Parameters explored: MWG NWG KWG MDIMC NDIMC MDIMA NDIMB KWI VWM VWN STRM STRN SA SB 

|   ID | total |                                                                 param |       compiles |         time | GFLOPS |            status |
x------x-------x-----------------------------------------------------------------------x----------------x--------------x--------x-------------------x
Segmentation fault (core dumped)

@CNugteren
Copy link
Owner

CNugteren commented Jan 13, 2018 •
edited
Loading

I guess you didn't build the clients? See the README for more information.

As for the tuner issue, could you try another tuner? Or run on your other device? Did you manage to run other OpenCL programs properly?

@ysh329
Copy link
Author

ysh329 commented Jan 14, 2018

Thanks my big brother, I'll have a try 🤣

@CNugteren
Copy link
Owner

The original issue reported in the benchmark script is solved (b35e3d1). The other issue of segfaults is further discussed in #243, so I'm closing this now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
correctness
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@CNugteren @ysh329