Skip to content

Latest commit

 

History

History
319 lines (298 loc) · 17.8 KB

report.md

File metadata and controls

319 lines (298 loc) · 17.8 KB

Benchmark Report

Job Properties

Commits: JuliaLang/julia@91b045ad4b6339337f0439686528346c17bce9a5 vs JuliaLang/julia@0f04b33e01d3139ea103d9153dc442418c206b8a

Comparison Diff: link

Triggered By: link

Tag Predicate: !"scalar"

Results

Note: If Chrome is your browser, I strongly recommend installing the Wide GitHub extension, which makes the result table easier to read.

Below is a table of this job's results, obtained by running the benchmarks found in JuliaCI/BaseBenchmarks.jl. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.

The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value.

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID time ratio memory ratio
["alloc", "grow_array"] 1.05 (5%) ❌ 1.00 (1%)
["alloc", "strings"] 0.95 (5%) ✅ 1.00 (1%)
["array", "bool", "boolarray_bool_load!"] 1.05 (5%) ❌ 1.00 (1%)
["array", "reductions", ("perf_reduce", "Int64")] 1.06 (5%) ❌ 1.00 (1%)
["array", "setindex!", ("setindex!", 4)] 1.06 (5%) ❌ 1.00 (1%)
["broadcast", "dotop", ("Float64", "(1000000,)", 1)] 0.87 (5%) ✅ 1.00 (1%)
["broadcast", "dotop", ("Float64", "(1000000,)", 2)] 0.86 (5%) ✅ 1.00 (1%)
["broadcast", "mix_scalar_tuple", (10, "scal_tup_x3")] 0.94 (5%) ✅ 1.00 (1%)
["broadcast", "mix_scalar_tuple", (5, "scal_tup_x3")] 1.35 (5%) ❌ 1.00 (1%)
["broadcast", "typeargs", ("array", 10)] 1.07 (5%) ❌ 1.00 (1%)
["broadcast", "typeargs", ("tuple", 10)] 1.09 (5%) ❌ 1.00 (1%)
["collection", "deletion", ("Set", "String", "filter!")] 0.65 (25%) ✅ 1.00 (1%)
["dates", "accessor", "millisecond"] 1.18 (5%) ❌ 1.00 (1%)
["dates", "arithmetic", ("Date", "Day")] 0.91 (5%) ✅ 1.00 (1%)
["dates", "arithmetic", ("DateTime", "Day")] 0.94 (5%) ✅ 1.00 (1%)
["dates", "arithmetic", ("DateTime", "Hour")] 1.10 (5%) ❌ 1.00 (1%)
["dates", "arithmetic", ("DateTime", "Millisecond")] 0.91 (5%) ✅ 1.00 (1%)
["dates", "arithmetic", ("DateTime", "Minute")] 1.10 (5%) ❌ 1.00 (1%)
["dates", "arithmetic", ("DateTime", "Second")] 1.10 (5%) ❌ 1.00 (1%)
["dates", "conversion", "Date -> DateTime"] 0.82 (5%) ✅ 1.00 (1%)
["dates", "parse", ("Date", "DateFormat")] 0.90 (5%) ✅ 1.00 (1%)
["dates", "parse", ("DateTime", "DateFormat")] 0.88 (5%) ✅ 1.00 (1%)
["find", "findall", ("Vector{Bool}", "10-90")] 1.46 (5%) ❌ 1.00 (1%)
["find", "findall", ("Vector{Bool}", "90-10")] 1.08 (5%) ❌ 1.00 (1%)
["find", "findall", ("ispos", "Vector{Float32}")] 1.07 (5%) ❌ 1.00 (1%)
["find", "findall", ("ispos", "Vector{UInt64}")] 1.10 (5%) ❌ 1.00 (1%)
["find", "findnext", ("ispos", "Vector{UInt64}")] 1.13 (5%) ❌ 1.00 (1%)
["find", "findprev", ("BitVector", "90-10")] 0.94 (5%) ✅ 1.00 (1%)
["inference", "abstract interpretation", "many_invoke_calls"] 1.05 (5%) ❌ 1.00 (1%)
["inference", "abstract interpretation", "many_method_matches"] 1.06 (5%) ❌ 1.00 (1%)
["inference", "abstract interpretation", "many_opaque_closures"] 1.01 (5%) 0.98 (1%) ✅
["inference", "optimization", "many_const_calls"] 1.05 (5%) ❌ 1.00 (1%)
["linalg", "small exp #29116"] 1.05 (5%) ❌ 1.00 (1%)
["micro", "fib"] 1.11 (5%) ❌ 1.00 (1%)
["misc", "23042", "ComplexF32"] 0.86 (5%) ✅ 1.00 (1%)
["misc", "allocation elision view", "conditional"] 0.66 (5%) ✅ 1.00 (1%)
["misc", "allocation elision view", "no conditional"] 0.66 (5%) ✅ 1.00 (1%)
["misc", "bitshift", ("UInt32", "UInt32")] 0.84 (5%) ✅ 1.00 (1%)
["misc", "foldl", "foldl(+, filter(...))"] 0.87 (5%) ✅ 1.00 (1%)
["misc", "perf highdim generator"] 1.05 (5%) ❌ 1.00 (1%)
["problem", "fem", "sparse_fem"] 0.91 (5%) ✅ 1.00 (1%)
["random", "ranges", ("rand", "MersenneTwister", "Int", "1:1000")] 0.59 (25%) ✅ 1.00 (1%)
["simd", ("Cartesian", "inner", "Int32", 3, 31)] 1.28 (20%) ❌ 1.00 (1%)
["simd", ("CartesianPartition", "conditional_loop!", "Int32", 2, 31)] 0.52 (20%) ✅ 1.00 (1%)
["simd", ("Linear", "conditional_loop!", "Int32", 4095)] 0.74 (20%) ✅ 1.00 (1%)
["sort", "issues", "partialsort!(rand(10_000), 1:3, rev=true)"] 1.00 (20%) 1.01 (1%) ❌
["sparse", "constructors", ("Diagonal", 1000)] 1.07 (5%) ❌ 1.00 (1%)
["string", "==(::AbstractString, ::AbstractString)", "identical"] 1.10 (5%) ❌ 1.00 (1%)
["string", "==(::SubString, ::String)", "different length"] 0.92 (5%) ✅ 1.00 (1%)
["string", "readuntil", "target length 1"] 0.93 (5%) ✅ 1.00 (1%)
["tuple", "linear algebra", ("matmat", "(16, 16)", "(16, 16)")] 1.08 (5%) ❌ 1.00 (1%)
["tuple", "linear algebra", ("matvec", "(4, 4)", "(4,)")] 1.12 (5%) ❌ 1.00 (1%)
["tuple", "linear algebra", ("matvec", "(8, 8)", "(8,)")] 0.73 (5%) ✅ 1.00 (1%)
["tuple", "misc", "11899"] 0.91 (5%) ✅ 1.00 (1%)
["tuple", "reduction", ("minimum", "(2, 2)")] 1.06 (5%) ❌ 1.00 (1%)
["tuple", "reduction", ("sum", "(4,)")] 1.07 (5%) ❌ 1.00 (1%)
["tuple", "reduction", ("sumabs", "(2,)")] 1.07 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "*", "BigFloat", "(false, false)")] 1.34 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "*", "BigFloat", "(false, true)")] 1.32 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "*", "BigFloat", "(true, true)")] 1.32 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "*", "Bool", "(false, true)")] 0.93 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "*", "ComplexF64", "(false, true)")] 0.95 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "*", "ComplexF64", "(true, true)")] 0.90 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "*", "Float64", "(false, true)")] 1.08 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "*", "Float64", "(true, true)")] 0.88 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "*", "Int64", "(true, true)")] 1.15 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "abs", "Bool", 1)] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "abs", "Int8", 1)] 0.86 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "identity", "BigInt", 1)] 1.13 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "identity", "Float32", 0)] 1.08 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "identity", "Int8", 0)] 1.07 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "identity", "Int8", 1)] 0.94 (5%) ✅ 1.00 (1%)
["union", "array", ("collect", "all", "Int64", 1)] 1.06 (5%) ❌ 1.00 (1%)
["union", "array", ("collect", "filter", "Float64", 0)] 0.92 (5%) ✅ 1.00 (1%)
["union", "array", ("map", "*", "BigFloat", "(false, false)")] 1.33 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "*", "BigFloat", "(false, true)")] 1.33 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "*", "BigFloat", "(true, true)")] 1.34 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "*", "ComplexF64", "(true, true)")] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "identity", "Int64", 1)] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "BigFloat", "(false, false)")] 1.34 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "BigFloat", "(false, true)")] 1.33 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "BigFloat", "(true, true)")] 1.33 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "Bool", "(true, true)")] 1.06 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "Float32", "(false, true)")] 1.19 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "Float32", "(true, true)")] 1.11 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "Int64", "(true, true)")] 0.90 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_binaryop", "*", "Int8", "(false, true)")] 1.08 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_simplecopy", "BigInt", 1)] 1.06 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_simplecopy", "Int8", 1)] 0.92 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum", "BigFloat", 0)] 1.07 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum", "BigFloat", 1)] 1.06 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum", "Int8", 0)] 0.91 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum", "Int8", 1)] 0.83 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum2", "Int8", 1)] 0.91 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum3", "BigFloat", 0)] 1.11 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum3", "BigFloat", 1)] 1.11 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum3", "Bool", 1)] 1.24 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum3", "ComplexF64", 1)] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum3", "Float32", 1)] 1.09 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum3", "Float64", 1)] 1.12 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum3", "Int64", 1)] 0.87 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum4", "BigFloat", 0)] 1.07 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum4", "BigFloat", 1)] 1.06 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_sum4", "Int8", 0)] 0.95 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum4", "Int8", 1)] 0.94 (5%) ✅ 1.00 (1%)
["union", "array", ("skipmissing", "keys", "Union{Missing, Int8}", 1)] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("skipmissing", "perf_sumskipmissing", "BigFloat", 0)] 1.09 (5%) ❌ 1.00 (1%)
["union", "array", ("skipmissing", "perf_sumskipmissing", "Int64", 0)] 0.94 (5%) ✅ 1.00 (1%)
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, ComplexF64}", 1)] 1.19 (5%) ❌ 1.00 (1%)
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, Float32}", 1)] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Nothing, BigFloat}", 0)] 1.05 (5%) ❌ 1.00 (1%)
["union", "array", ("skipmissing", "sum", "Union{Nothing, BigFloat}", 0)] 1.06 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["alloc"]
  • ["array", "accumulate"]
  • ["array", "any/all"]
  • ["array", "bool"]
  • ["array", "cat"]
  • ["array", "comprehension"]
  • ["array", "convert"]
  • ["array", "equality"]
  • ["array", "growth"]
  • ["array", "index"]
  • ["array", "reductions"]
  • ["array", "reverse"]
  • ["array", "setindex!"]
  • ["array", "subarray"]
  • ["broadcast"]
  • ["broadcast", "dotop"]
  • ["broadcast", "fusion"]
  • ["broadcast", "mix_scalar_tuple"]
  • ["broadcast", "sparse"]
  • ["broadcast", "typeargs"]
  • ["collection", "deletion"]
  • ["collection", "initialization"]
  • ["collection", "iteration"]
  • ["collection", "optimizations"]
  • ["collection", "queries & updates"]
  • ["collection", "set operations"]
  • ["dates", "accessor"]
  • ["dates", "arithmetic"]
  • ["dates", "construction"]
  • ["dates", "conversion"]
  • ["dates", "parse"]
  • ["dates", "query"]
  • ["dates", "string"]
  • ["find", "findall"]
  • ["find", "findnext"]
  • ["find", "findprev"]
  • ["frontend"]
  • ["inference", "abstract interpretation"]
  • ["inference", "allinference"]
  • ["inference", "optimization"]
  • ["io", "array_limit"]
  • ["io", "read"]
  • ["io", "serialization"]
  • ["io"]
  • ["linalg", "arithmetic"]
  • ["linalg", "blas"]
  • ["linalg", "factorization"]
  • ["linalg"]
  • ["micro"]
  • ["misc"]
  • ["misc", "23042"]
  • ["misc", "afoldl"]
  • ["misc", "allocation elision view"]
  • ["misc", "bitshift"]
  • ["misc", "foldl"]
  • ["misc", "issue 12165"]
  • ["misc", "iterators"]
  • ["misc", "julia"]
  • ["misc", "parse"]
  • ["misc", "repeat"]
  • ["misc", "splatting"]
  • ["problem", "chaosgame"]
  • ["problem", "fem"]
  • ["problem", "go"]
  • ["problem", "grigoriadis khachiyan"]
  • ["problem", "imdb"]
  • ["problem", "json"]
  • ["problem", "laplacian"]
  • ["problem", "monte carlo"]
  • ["problem", "raytrace"]
  • ["problem", "seismic"]
  • ["problem", "simplex"]
  • ["problem", "spellcheck"]
  • ["problem", "stockcorr"]
  • ["problem", "ziggurat"]
  • ["random", "collections"]
  • ["random", "randstring"]
  • ["random", "ranges"]
  • ["random", "sequences"]
  • ["random", "types"]
  • ["shootout"]
  • ["simd"]
  • ["sort", "insertionsort"]
  • ["sort", "issorted"]
  • ["sort", "issues"]
  • ["sort", "length = 10"]
  • ["sort", "length = 100"]
  • ["sort", "length = 1000"]
  • ["sort", "length = 10000"]
  • ["sort", "length = 3"]
  • ["sort", "length = 30"]
  • ["sort", "mergesort"]
  • ["sort", "quicksort"]
  • ["sparse", "arithmetic"]
  • ["sparse", "constructors"]
  • ["sparse", "index"]
  • ["sparse", "matmul"]
  • ["sparse", "sparse matvec"]
  • ["sparse", "sparse solves"]
  • ["sparse", "transpose"]
  • ["string", "==(::AbstractString, ::AbstractString)"]
  • ["string", "==(::SubString, ::String)"]
  • ["string", "findfirst"]
  • ["string"]
  • ["string", "readuntil"]
  • ["string", "repeat"]
  • ["tuple", "index"]
  • ["tuple", "linear algebra"]
  • ["tuple", "misc"]
  • ["tuple", "reduction"]
  • ["union", "array"]

Version Info

Primary Build

Julia Version 1.12.0-DEV.214
Commit 91b045ad4b (2024-03-14 19:34 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.3 LTS
  uname: Linux 5.15.0-76-generic #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz: 
              speed         user         nice          sys         idle          irq
       #1  3500 MHz     975544 s        458 s     235598 s  216511129 s          0 s
       #2  3500 MHz   15490191 s        232 s     398525 s  201942027 s          0 s
       #3  3501 MHz     763946 s        402 s     170994 s  216805379 s          0 s
       #4  3500 MHz     632338 s        269 s     179871 s  216810120 s          0 s
       #5  3501 MHz     624975 s        209 s     182810 s  216126597 s          0 s
       #6  3501 MHz     659468 s        222 s     178177 s  216896387 s          0 s
       #7  3502 MHz     695646 s        181 s     199940 s  216898942 s          0 s
       #8  3505 MHz     677748 s        154 s     165699 s  216949579 s          0 s
  Memory: 31.301593780517578 GB (18106.9375 MB free)
  Uptime: 2.179875035e7 sec
  Load Avg:  1.0  1.0  1.0
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, haswell)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)

Comparison Build

Julia Version 1.12.0-DEV.204
Commit 0f04b33e01 (2024-03-14 19:30 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.3 LTS
  uname: Linux 5.15.0-76-generic #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz: 
              speed         user         nice          sys         idle          irq
       #1  3500 MHz     976180 s        458 s     235793 s  216621948 s          0 s
       #2  3500 MHz   15599375 s        232 s     400446 s  201943051 s          0 s
       #3  3501 MHz     764585 s        402 s     171016 s  216916828 s          0 s
       #4  3499 MHz     632538 s        269 s     179880 s  216921939 s          0 s
       #5  3501 MHz     625075 s        209 s     182817 s  216238384 s          0 s
       #6  3502 MHz     659554 s        222 s     178185 s  217008410 s          0 s
       #7  3503 MHz     695698 s        181 s     199949 s  217010999 s          0 s
       #8  3500 MHz     677838 s        154 s     165710 s  217061576 s          0 s
  Memory: 31.301593780517578 GB (18115.66015625 MB free)
  Uptime: 2.180996339e7 sec
  Load Avg:  1.07  1.02  1.0
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, haswell)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)