Get benchmarks working again

[thomasfaingnaert]

No description provided.

[maleadt]

Hmm, this is unfortunate. It's only there for the validation, right? Can't we do something lazy, like https://github.com/JuliaGPU/GPUCompiler.jl/blob/962b84ed84cffc268ce1c28b1c71037566622f24/src/reflection.jl#L60-L62 ?

[thomasfaingnaert]

It's only in the benchmarks/Project.toml, not in the Project.toml in the root, so that doesn't matter, right?

[maleadt]

Ah, I missed that. Carry on.

[thomasfaingnaert]

Benchmarks bump into an illegal memory access now, which is weird, because the configuration where it happens should also be run in CI, and it's fine there:

[ Info: Running benchmark WMMA GEMM Float16*Float16+Float32=Float32 (256×256) · (256×256) (TN) Block (256, 64, 64) Warps (2, 2) OP (16, 16, 16)...
ERROR: LoadError: CUDA error: an illegal memory access was encountered (code 700, ERROR_ILLEGAL_ADDRESS)
throw_api_error(res::CUDA.cudaError_enum)
   @ CUDA ~/.cache/julia-buildkite-plugin/depots/3105e5d3-28f0-4cf0-b90b-02786f04b8f6/packages/CUDA/htRwP/lib/cudadrv/libcuda.jl:27
check
   @ ~/.cache/julia-buildkite-plugin/depots/3105e5d3-28f0-4cf0-b90b-02786f04b8f6/packages/CUDA/htRwP/lib/cudadrv/libcuda.jl:34 [inlined]
cuCtxSynchronize
   @ ~/.cache/julia-buildkite-plugin/depots/3105e5d3-28f0-4cf0-b90b-02786f04b8f6/packages/CUDA/htRwP/lib/utils/call.jl:26 [inlined]
profile_internally(f::var"#473#profiled_code#24"{Configuration, CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CuArray{Float16, 2, CUDA.Mem.DeviceBuffer}, CuArray{Float16, 2, CUDA.Mem.DeviceBuffer}}; concurrent::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ CUDA.Profile ~/.cache/julia-buildkite-plugin/depots/3105e5d3-28f0-4cf0-b90b-02786f04b8f6/packages/CUDA/htRwP/src/profile.jl:269
profile_internally(f::Function)
   @ CUDA.Profile ~/.cache/julia-buildkite-plugin/depots/3105e5d3-28f0-4cf0-b90b-02786f04b8f6/packages/CUDA/htRwP/src/profile.jl:239
macro expansion
   @ ~/.cache/julia-buildkite-plugin/depots/3105e5d3-28f0-4cf0-b90b-02786f04b8f6/packages/CUDA/htRwP/src/profile.jl:62 [inlined]
   @ /var/lib/buildkite-agent/builds/ripper/julialang/gemmkernels-dot-jl/benchmarks/runbenchmarks.jl:120
include(fname::String)
   @ Base.MainInclude ./client.jl:478
   @ none:10
🚨 Error: The command exited with status 1

[maleadt]

Benchmarks bump into an illegal memory access now, which is weird, because the configuration where it happens should also be run in CI

Benchmarks don't run with --check-bounds=yes.

[thomasfaingnaert]

I can only reproduce the illegal memory access locally on 1.9, seems to be fixed in 1.10. Let's bump benchmarks to Julia 1.10 and see if that fixes things.

[maleadt]

I can only reproduce the illegal memory access locally on 1.9, seems to be fixed in 1.10. Let's bump benchmarks to Julia 1.10 and see if that fixes things.

[thomasfaingnaert]

@maleadt Expired GH token?

2024-01-26 18:56:26 CEST	ERROR: LoadError: Error found in GitHub reponse:
2024-01-26 18:56:26 CEST		Status Code: 422
2024-01-26 18:56:26 CEST		Message: Validation Failed
2024-01-26 18:56:26 CEST		Docs URL: https://docs.github.com/rest/issues/comments#create-an-issue-comment
2024-01-26 18:56:26 CEST		Errors: Any[Dict{String, Any}("message" => "Body is too long (maximum is 65536 characters)", "field" => "data", "code" => "unprocessable", "resource" => "IssueComment")]

EDIT: Nevermind, I missed the error message: Body is too long.

[thomasfaingnaert]

Let's just merge this for now so we at least have some form of benchmarking. I'll keep track of the remaining problems in an issue.