Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add test comparison to Test262 result compare #1279

Merged
merged 11 commits into from
May 29, 2021
Merged

Add test comparison to Test262 result compare #1279

merged 11 commits into from
May 29, 2021

Conversation

Razican
Copy link
Member

@Razican Razican commented May 24, 2021

This adds some result comparison to Test262 results. It will add 4 lists (if not empty) to conformance results:

  • List of tests that were previously broken/ignored but now pass.
  • List of tests that were passing but now fail
  • List of panics that no longer panic
  • List of tests that were not panicking but now panic.

Note that this will only check for tests that already existed, so if we update the Test262 submodule, new tests will not appear in this list.

@Razican Razican added enhancement New feature or request test Issues and PRs related to the tests. labels May 24, 2021
@Razican Razican added this to the v0.12.0 milestone May 24, 2021
@Razican
Copy link
Member Author

Razican commented May 24, 2021

I will be testing if this actually works now with a couple of commits making things fail/panic.

@github-actions
Copy link

Benchmark for 2a85a3c

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 340.4±16.79ns 331.7±12.54ns +2.62%
Arithmetic operations (Full) 276.7±13.42µs 285.2±16.53µs -2.98%
Array access (Execution) 6.5±0.36µs 6.7±0.36µs -2.99%
Array access (Full) 319.9±10.74µs 317.0±68.22µs +0.91%
Array creation (Execution) 2.7±0.16ms 2.7±0.13ms 0.00%
Array creation (Full) 3.3±0.23ms 3.0±0.16ms +10.00%
Array pop (Execution) 885.6±44.92µs 890.5±49.46µs -0.55%
Array pop (Full) 1429.1±57.08µs 1336.0±65.38µs +6.97%
Boolean Object Access (Execution) 5.4±0.25µs 5.7±0.56µs -5.26%
Boolean Object Access (Full) 293.7±14.78µs 308.6±25.10µs -4.83%
Clean js (Execution) 647.7±31.95µs 641.9±26.86µs +0.90%
Clean js (Full) 953.8±50.75µs 980.1±64.89µs -2.68%
Clean js (Parser) 41.2±3.04µs 41.3±1.90µs -0.24%
Create Realm 415.6±24.76ns 397.7±21.67ns +4.50%
Dynamic Object Property Access (Execution) 5.4±0.35µs 5.3±0.30µs +1.89%
Dynamic Object Property Access (Full) 319.3±13.03µs 312.3±17.87µs +2.24%
Expression (Parser) 6.9±0.30µs 7.6±0.60µs -9.21%
Fibonacci (Execution) 834.9±47.34µs 816.3±42.88µs +2.28%
Fibonacci (Full) 1189.9±47.68µs 1104.2±54.15µs +7.76%
For loop (Execution) 22.0±0.81µs 23.6±2.67µs -6.78%
For loop (Full) 333.6±13.98µs 323.0±15.83µs +3.28%
For loop (Parser) 20.1±2.43µs 19.8±1.02µs +1.52%
Goal Symbols (Parser) 14.1±0.96µs 13.5±0.56µs +4.44%
Hello World (Parser) 3.9±0.23µs 3.8±0.20µs +2.63%
Long file (Parser) 740.4±32.94ns 764.5±41.48ns -3.15%
Mini js (Execution) 597.7±38.69µs 583.1±89.21µs +2.50%
Mini js (Full) 896.9±48.71µs 891.1±40.88µs +0.65%
Mini js (Parser) 36.9±4.45µs 37.2±7.63µs -0.81%
Number Object Access (Execution) 4.2±0.30µs 4.2±0.23µs 0.00%
Number Object Access (Full) 291.2±15.19µs 301.0±16.74µs -3.26%
Object Creation (Execution) 4.5±0.24µs 4.5±0.38µs 0.00%
Object Creation (Full) 314.1±11.09µs 302.6±16.50µs +3.80%
RegExp (Execution) 11.0±0.65µs 11.2±0.62µs -1.79%
RegExp (Full) 299.0±11.18µs 318.4±16.83µs -6.09%
RegExp Literal (Execution) 11.0±0.47µs 11.0±0.70µs 0.00%
RegExp Literal (Full) 311.6±18.10µs 325.8±151.67µs -4.36%
RegExp Literal Creation (Execution) 9.6±0.53µs 10.1±0.75µs -4.95%
RegExp Literal Creation (Full) 320.8±16.37µs 309.3±19.24µs +3.72%
Static Object Property Access (Execution) 5.0±0.20µs 4.7±0.26µs +6.38%
Static Object Property Access (Full) 311.5±12.78µs 299.3±12.94µs +4.08%
String Object Access (Execution) 7.3±0.68µs 7.2±0.38µs +1.39%
String Object Access (Full) 305.8±20.30µs 304.2±15.57µs +0.53%
String comparison (Execution) 6.5±0.28µs 6.4±0.25µs +1.56%
String comparison (Full) 304.0±30.55µs 304.6±20.08µs -0.20%
String concatenation (Execution) 5.4±0.38µs 5.4±0.33µs 0.00%
String concatenation (Full) 287.8±12.33µs 294.8±15.75µs -2.37%
String copy (Execution) 4.0±0.25µs 4.1±0.29µs -2.44%
String copy (Full) 284.9±13.85µs 294.6±15.66µs -3.29%
Symbols (Execution) 3.6±0.33µs 3.4±0.20µs +5.88%
Symbols (Full) 295.6±33.75µs 284.8±23.99µs +3.79%

@github-actions
Copy link

Benchmark for f58d835

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 349.7±18.02ns 357.6±20.95ns -2.21%
Arithmetic operations (Full) 276.5±18.59µs 275.6±15.18µs +0.33%
Array access (Execution) 6.4±0.32µs 6.6±0.34µs -3.03%
Array access (Full) 307.7±23.01µs 302.6±16.86µs +1.69%
Array creation (Execution) 2.7±0.17ms 2.6±0.12ms +3.85%
Array creation (Full) 3.0±0.18ms 3.0±0.21ms 0.00%
Array pop (Execution) 190.0±13.28µs 846.9±51.37µs -77.57%
Array pop (Full) 680.3±32.38µs 1334.0±81.56µs -49.00%
Boolean Object Access (Execution) 5.2±0.28µs 5.6±0.35µs -7.14%
Boolean Object Access (Full) 296.7±19.03µs 295.4±24.03µs +0.44%
Clean js (Execution) 665.6±51.38µs 647.3±42.91µs +2.83%
Clean js (Full) 933.7±54.29µs 964.7±49.33µs -3.21%
Clean js (Parser) 41.5±3.33µs 40.3±2.09µs +2.98%
Create Realm 401.0±28.43ns 421.8±36.33ns -4.93%
Dynamic Object Property Access (Execution) 5.4±0.33µs 5.5±0.35µs -1.82%
Dynamic Object Property Access (Full) 322.4±33.58µs 315.4±20.49µs +2.22%
Expression (Parser) 6.8±0.48µs 6.7±0.50µs +1.49%
Fibonacci (Execution) 841.1±43.02µs 863.6±157.51µs -2.61%
Fibonacci (Full) 1161.7±72.66µs 1137.1±82.53µs +2.16%
For loop (Execution) 22.2±1.52µs 23.2±2.23µs -4.31%
For loop (Full) 316.4±18.46µs 323.8±20.84µs -2.29%
For loop (Parser) 19.8±1.59µs 18.7±1.14µs +5.88%
Goal Symbols (Parser) 13.7±1.07µs 13.8±1.01µs -0.72%
Hello World (Parser) 3.7±0.28µs 3.7±0.23µs 0.00%
Long file (Parser) 783.5±127.68ns 762.4±60.66ns +2.77%
Mini js (Execution) 572.1±33.42µs 583.3±40.85µs -1.92%
Mini js (Full) 894.6±54.90µs 906.8±48.67µs -1.35%
Mini js (Parser) 35.6±3.04µs 34.7±2.59µs +2.59%
Number Object Access (Execution) 4.1±0.25µs 4.1±0.25µs 0.00%
Number Object Access (Full) 304.5±16.50µs 295.3±20.66µs +3.12%
Object Creation (Execution) 4.6±0.22µs 4.7±0.44µs -2.13%
Object Creation (Full) 299.5±16.70µs 296.3±19.12µs +1.08%
RegExp (Execution) 11.4±0.84µs 11.2±0.73µs +1.79%
RegExp (Full) 326.4±30.53µs 328.7±22.32µs -0.70%
RegExp Literal (Execution) 11.0±1.38µs 10.8±0.65µs +1.85%
RegExp Literal (Full) 318.5±21.85µs 309.6±18.60µs +2.87%
RegExp Literal Creation (Execution) 9.6±0.62µs 9.9±0.59µs -3.03%
RegExp Literal Creation (Full) 335.1±45.47µs 318.1±16.23µs +5.34%
Static Object Property Access (Execution) 4.8±0.27µs 4.8±0.29µs 0.00%
Static Object Property Access (Full) 326.4±41.08µs 299.7±16.91µs +8.91%
String Object Access (Execution) 7.3±0.46µs 7.2±0.38µs +1.39%
String Object Access (Full) 304.7±17.16µs 300.6±18.02µs +1.36%
String comparison (Execution) 6.5±0.39µs 6.6±0.42µs -1.52%
String comparison (Full) 306.4±21.19µs 295.1±17.23µs +3.83%
String concatenation (Execution) 5.8±0.89µs 5.4±0.37µs +7.41%
String concatenation (Full) 311.3±36.92µs 295.8±23.03µs +5.24%
String copy (Execution) 4.2±0.31µs 4.0±0.23µs +5.00%
String copy (Full) 287.4±17.06µs 278.8±16.21µs +3.08%
Symbols (Execution) 3.5±0.20µs 3.5±0.29µs 0.00%
Symbols (Full) 290.8±19.80µs 278.2±16.79µs +4.53%

@github-actions
Copy link

Benchmark for f41cc14

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 368.9±0.44ns 364.6±0.45ns +1.18%
Arithmetic operations (Full) 272.0±0.44µs 268.9±0.70µs +1.15%
Array access (Execution) 6.1±0.03µs 6.1±0.02µs 0.00%
Array access (Full) 300.2±0.60µs 297.8±1.07µs +0.81%
Array creation (Execution) 3.1±0.00ms 2.9±0.00ms +6.90%
Array creation (Full) 3.4±0.01ms 3.3±0.01ms +3.03%
Array pop (Execution) 232.1±1.30µs 930.0±2.19µs -75.04%
Array pop (Full) 719.9±1.33µs 1437.6±3.07µs -49.92%
Boolean Object Access (Execution) 5.2±0.01µs 5.3±0.02µs -1.89%
Boolean Object Access (Full) 293.7±0.84µs 291.3±0.58µs +0.82%
Clean js (Execution) 668.0±3.40µs 660.4±3.27µs +1.15%
Clean js (Full) 986.2±3.75µs 995.8±14.04µs -0.96%
Clean js (Parser) 41.3±0.08µs 41.4±0.09µs -0.24%
Create Realm 433.7±0.75ns 425.5±0.30ns +1.93%
Dynamic Object Property Access (Execution) 5.1±0.06µs 5.0±0.03µs +2.00%
Dynamic Object Property Access (Full) 296.6±0.84µs 289.9±0.74µs +2.31%
Expression (Parser) 7.4±0.01µs 7.3±0.01µs +1.37%
Fibonacci (Execution) 754.9±2.36µs 757.0±2.90µs -0.28%
Fibonacci (Full) 1074.2±10.04µs 1062.4±1.93µs +1.11%
For loop (Execution) 22.0±0.07µs 21.9±0.07µs +0.46%
For loop (Full) 306.7±0.44µs 304.1±0.53µs +0.85%
For loop (Parser) 19.9±0.04µs 19.9±0.07µs 0.00%
Goal Symbols (Parser) 14.2±0.04µs 14.2±0.04µs 0.00%
Hello World (Parser) 4.0±0.01µs 4.0±0.01µs 0.00%
Long file (Parser) 786.4±6.01ns 793.3±3.18ns -0.87%
Mini js (Execution) 603.6±3.15µs 597.0±2.78µs +1.11%
Mini js (Full) 925.1±5.76µs 924.9±4.59µs +0.02%
Mini js (Parser) 35.9±0.08µs 35.9±0.06µs 0.00%
Number Object Access (Execution) 4.1±0.01µs 4.1±0.01µs 0.00%
Number Object Access (Full) 286.1±0.42µs 286.4±0.93µs -0.10%
Object Creation (Execution) 4.3±0.01µs 4.4±0.02µs -2.27%
Object Creation (Full) 287.0±0.47µs 288.9±0.78µs -0.66%
RegExp (Execution) 10.9±0.03µs 11.9±0.04µs -8.40%
RegExp (Full) 302.6±0.85µs 302.7±0.49µs -0.03%
RegExp Literal (Execution) 10.9±0.04µs 11.9±0.06µs -8.40%
RegExp Literal (Full) 300.6±0.64µs 297.0±0.67µs +1.21%
RegExp Literal Creation (Execution) 9.4±0.03µs 10.5±0.03µs -10.48%
RegExp Literal Creation (Full) 292.5±0.58µs 289.8±0.52µs +0.93%
Static Object Property Access (Execution) 4.4±0.03µs 4.5±0.02µs -2.22%
Static Object Property Access (Full) 289.6±0.67µs 287.3±1.00µs +0.80%
String Object Access (Execution) 7.2±0.01µs 7.2±0.02µs 0.00%
String Object Access (Full) 294.2±0.98µs 291.7±0.64µs +0.86%
String comparison (Execution) 6.5±0.03µs 6.6±0.04µs -1.52%
String comparison (Full) 295.8±1.16µs 288.2±0.75µs +2.64%
String concatenation (Execution) 5.2±0.04µs 5.2±0.02µs 0.00%
String concatenation (Full) 288.8±0.69µs 285.8±0.75µs +1.05%
String copy (Execution) 3.9±0.01µs 4.0±0.02µs -2.50%
String copy (Full) 283.4±0.69µs 277.5±0.54µs +2.13%
Symbols (Execution) 3.3±0.01µs 3.3±0.01µs 0.00%
Symbols (Full) 275.4±0.60µs 272.4±2.12µs +1.10%

@github-actions
Copy link

Benchmark for 328153f

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 302.9±0.20ns 303.1±0.16ns -0.07%
Arithmetic operations (Full) 226.5±0.42µs 232.2±0.67µs -2.45%
Array access (Execution) 5.1±0.02µs 5.2±0.02µs -1.92%
Array access (Full) 248.1±1.08µs 251.5±0.48µs -1.35%
Array creation (Execution) 2.6±0.01ms 2.4±0.00ms +8.33%
Array creation (Full) 2.8±0.00ms 2.7±0.00ms +3.70%
Array pop (Execution) 192.8±1.79µs 776.9±1.75µs -75.18%
Array pop (Full) 606.7±0.91µs 1203.6±1.32µs -49.59%
Boolean Object Access (Execution) 4.4±0.01µs 4.5±0.01µs -2.22%
Boolean Object Access (Full) 242.5±0.43µs 248.1±2.22µs -2.26%
Clean js (Execution) 556.5±7.55µs 549.7±4.15µs +1.24%
Clean js (Full) 825.6±3.58µs 823.9±3.62µs +0.21%
Clean js (Parser) 36.3±0.14µs 36.5±0.07µs -0.55%
Create Realm 361.8±0.41ns 354.9±3.47ns +1.94%
Dynamic Object Property Access (Execution) 4.2±0.01µs 4.3±0.02µs -2.33%
Dynamic Object Property Access (Full) 246.8±0.51µs 245.5±0.35µs +0.53%
Expression (Parser) 6.2±0.04µs 6.2±0.00µs 0.00%
Fibonacci (Execution) 644.0±15.57µs 636.8±1.83µs +1.13%
Fibonacci (Full) 902.4±1.47µs 905.6±4.99µs -0.35%
For loop (Execution) 18.2±0.05µs 18.2±0.04µs 0.00%
For loop (Full) 259.5±0.49µs 259.5±0.43µs 0.00%
For loop (Parser) 17.4±0.02µs 17.6±0.03µs -1.14%
Goal Symbols (Parser) 12.2±0.02µs 12.4±0.17µs -1.61%
Hello World (Parser) 3.4±0.01µs 3.5±0.01µs -2.86%
Long file (Parser) 709.5±6.50ns 731.2±6.30ns -2.97%
Mini js (Execution) 501.8±2.56µs 497.5±2.89µs +0.86%
Mini js (Full) 769.0±2.55µs 771.9±2.75µs -0.38%
Mini js (Parser) 31.6±0.05µs 31.8±0.02µs -0.63%
Number Object Access (Execution) 3.4±0.01µs 3.4±0.00µs 0.00%
Number Object Access (Full) 237.9±0.42µs 244.3±0.32µs -2.62%
Object Creation (Execution) 3.6±0.01µs 3.6±0.01µs 0.00%
Object Creation (Full) 239.7±2.05µs 243.6±7.70µs -1.60%
RegExp (Execution) 9.1±0.02µs 9.9±0.03µs -8.08%
RegExp (Full) 254.5±0.68µs 258.4±0.70µs -1.51%
RegExp Literal (Execution) 9.1±0.02µs 9.9±0.02µs -8.08%
RegExp Literal (Full) 249.6±0.45µs 255.1±0.35µs -2.16%
RegExp Literal Creation (Execution) 8.0±0.02µs 8.7±0.02µs -8.05%
RegExp Literal Creation (Full) 243.4±1.05µs 248.8±0.62µs -2.17%
Static Object Property Access (Execution) 3.7±0.01µs 3.8±0.02µs -2.63%
Static Object Property Access (Full) 247.6±0.60µs 242.9±0.36µs +1.93%
String Object Access (Execution) 6.1±0.07µs 6.0±0.04µs +1.67%
String Object Access (Full) 248.3±0.40µs 248.1±0.35µs +0.08%
String comparison (Execution) 5.5±0.02µs 5.4±0.01µs +1.85%
String comparison (Full) 243.7±0.49µs 246.4±1.20µs -1.10%
String concatenation (Execution) 4.4±0.02µs 4.4±0.01µs 0.00%
String concatenation (Full) 240.5±1.18µs 243.0±0.41µs -1.03%
String copy (Execution) 3.3±0.01µs 3.2±0.01µs +3.12%
String copy (Full) 232.0±0.27µs 235.5±0.39µs -1.49%
Symbols (Execution) 2.8±0.00µs 2.8±0.01µs 0.00%
Symbols (Full) 228.0±0.54µs 230.1±0.61µs -0.91%

@github-actions
Copy link

github-actions bot commented May 24, 2021 •
edited
Loading

Test262 conformance changes:

Test result master count PR count difference
Total 78,873 78,873 0
Passed 26,665 26,665 0
Ignored 15,604 15,604 0
Failed 36,604 36,604 0
Panics 14 14 0
Conformance 33.81% 33.81% 0.00%

@github-actions
Copy link

Benchmark for fb8db27

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 307.8±0.31ns 315.6±0.20ns -2.47%
Arithmetic operations (Full) 227.3±0.36µs 225.1±0.35µs +0.98%
Array access (Execution) 5.2±0.02µs 5.2±0.02µs 0.00%
Array access (Full) 252.7±0.56µs 247.6±0.32µs +2.06%
Array creation (Execution) 2.6±0.00ms 2.4±0.00ms +8.33%
Array creation (Full) 2.8±0.02ms 2.8±0.00ms 0.00%
Array pop (Execution) 193.9±1.44µs 777.4±1.63µs -75.06%
Array pop (Full) 606.1±0.81µs 1230.9±2.06µs -50.76%
Boolean Object Access (Execution) 4.4±0.00µs 4.4±0.01µs 0.00%
Boolean Object Access (Full) 243.4±0.37µs 241.2±0.32µs +0.91%
Clean js (Execution) 557.2±2.85µs 549.5±4.48µs +1.40%
Clean js (Full) 836.9±2.75µs 852.1±5.12µs -1.78%
Clean js (Parser) 36.1±0.04µs 36.4±0.05µs -0.82%
Create Realm 361.0±0.32ns 353.8±0.15ns +2.04%
Dynamic Object Property Access (Execution) 4.2±0.01µs 4.2±0.01µs 0.00%
Dynamic Object Property Access (Full) 248.1±0.65µs 247.7±0.57µs +0.16%
Expression (Parser) 6.2±0.06µs 6.2±0.01µs 0.00%
Fibonacci (Execution) 637.1±1.82µs 634.3±1.10µs +0.44%
Fibonacci (Full) 906.8±1.54µs 901.6±1.17µs +0.58%
For loop (Execution) 18.3±0.19µs 18.3±0.04µs 0.00%
For loop (Full) 257.8±0.67µs 258.1±0.44µs -0.12%
For loop (Parser) 17.4±0.03µs 17.3±0.07µs +0.58%
Goal Symbols (Parser) 12.2±0.02µs 12.2±0.02µs 0.00%
Hello World (Parser) 3.4±0.01µs 3.4±0.02µs 0.00%
Long file (Parser) 715.7±4.59ns 714.7±4.97ns +0.14%
Mini js (Execution) 506.2±5.83µs 496.5±2.65µs +1.95%
Mini js (Full) 777.6±2.42µs 795.2±2.47µs -2.21%
Mini js (Parser) 31.6±0.07µs 31.8±0.04µs -0.63%
Number Object Access (Execution) 3.4±0.00µs 3.4±0.00µs 0.00%
Number Object Access (Full) 239.9±0.58µs 238.5±0.29µs +0.59%
Object Creation (Execution) 3.6±0.01µs 3.6±0.01µs 0.00%
Object Creation (Full) 245.4±0.44µs 241.9±0.44µs +1.45%
RegExp (Execution) 9.1±0.03µs 10.0±0.05µs -9.00%
RegExp (Full) 258.3±1.11µs 252.8±0.42µs +2.18%
RegExp Literal (Execution) 9.2±0.02µs 10.0±0.02µs -8.00%
RegExp Literal (Full) 254.5±0.78µs 251.7±2.85µs +1.11%
RegExp Literal Creation (Execution) 8.0±0.09µs 8.7±0.08µs -8.05%
RegExp Literal Creation (Full) 250.6±0.44µs 245.1±0.25µs +2.24%
Static Object Property Access (Execution) 3.7±0.01µs 3.8±0.01µs -2.63%
Static Object Property Access (Full) 244.6±0.42µs 241.6±0.69µs +1.24%
String Object Access (Execution) 6.0±0.01µs 6.0±0.04µs 0.00%
String Object Access (Full) 249.3±0.69µs 247.5±0.35µs +0.73%
String comparison (Execution) 5.4±0.02µs 5.5±0.01µs -1.82%
String comparison (Full) 246.8±0.70µs 246.9±0.46µs -0.04%
String concatenation (Execution) 4.4±0.01µs 4.3±0.01µs +2.33%
String concatenation (Full) 239.4±0.31µs 236.9±0.31µs +1.06%
String copy (Execution) 3.3±0.01µs 3.3±0.01µs 0.00%
String copy (Full) 234.7±1.22µs 232.6±3.70µs +0.90%
Symbols (Execution) 2.8±0.02µs 2.8±0.01µs 0.00%
Symbols (Full) 229.3±2.57µs 227.9±0.29µs +0.61%

@Razican Razican requested review from 0x7D2B, HalidOdat and RageKnify May 24, 2021 20:36
@Razican Razican linked an issue May 24, 2021 that may be closed by this pull request
More detailed information for test conformance changes #1277
Closed
@boa-dev boa-dev deleted a comment from github-actions bot May 24, 2021
@Razican
Copy link
Member Author

Razican commented May 24, 2021 •
edited
Loading

Example output:

Test result master count PR count difference
Total 78,873 78,873 0
Passed 26,498 26,398 -100
Ignored 15,604 15,604 0
Failed 36,771 36,871 +100
Panics 26 27 +1
Conformance 33.60% 33.47% -0.13%

Broken tests:

test/language/expressions/typeof/bigint.js [strict mode] (previously Passed)
test/language/expressions/typeof/bigint.js (previously Passed)
test/intl402/BigInt/prototype/toLocaleString/this-value-invalid.js [strict mode] (previously Passed)
test/intl402/BigInt/prototype/toLocaleString/this-value-invalid.js (previously Passed)
test/built-ins/BigInt/constructor-from-hex-string.js [strict mode] (previously Passed)
test/built-ins/BigInt/constructor-from-hex-string.js (previously Passed)
test/built-ins/BigInt/call-value-of-when-to-string-present.js [strict mode] (previously Passed)
test/built-ins/BigInt/call-value-of-when-to-string-present.js (previously Passed)
test/built-ins/BigInt/proto.js [strict mode] (previously Passed)
test/built-ins/BigInt/proto.js (previously Passed)
test/built-ins/BigInt/constructor-empty-string.js [strict mode] (previously Passed)
test/built-ins/BigInt/constructor-empty-string.js (previously Passed)
test/built-ins/BigInt/valueof-throws.js [strict mode] (previously Passed)
test/built-ins/BigInt/valueof-throws.js (previously Passed)
test/built-ins/BigInt/constructor-from-string-syntax-errors.js [strict mode] (previously Passed)
test/built-ins/BigInt/constructor-from-string-syntax-errors.js (previously Passed)
test/built-ins/BigInt/name.js [strict mode] (previously Passed)
test/built-ins/BigInt/name.js (previously Passed)
test/built-ins/BigInt/constructor-integer.js [strict mode] (previously Passed)
test/built-ins/BigInt/constructor-integer.js (previously Passed)
test/built-ins/BigInt/prop-desc.js [strict mode] (previously Passed)
test/built-ins/BigInt/prop-desc.js (previously Passed)
test/built-ins/BigInt/length.js [strict mode] (previously Passed)
test/built-ins/BigInt/length.js (previously Passed)
test/built-ins/BigInt/constructor-from-decimal-string.js [strict mode] (previously Passed)
test/built-ins/BigInt/constructor-from-decimal-string.js (previously Passed)
test/built-ins/BigInt/tostring-throws.js [strict mode] (previously Passed)
test/built-ins/BigInt/tostring-throws.js (previously Passed)
test/built-ins/BigInt/constructor-from-binary-string.js [strict mode] (previously Passed)
test/built-ins/BigInt/constructor-from-binary-string.js (previously Passed)
test/built-ins/BigInt/constructor-from-octal-string.js [strict mode] (previously Passed)
test/built-ins/BigInt/constructor-from-octal-string.js (previously Passed)
test/built-ins/BigInt/asIntN/arithmetic.js [strict mode] (previously Passed)
test/built-ins/BigInt/asIntN/arithmetic.js (previously Passed)
test/built-ins/BigInt/asIntN/bits-toindex.js [strict mode] (previously Passed)
test/built-ins/BigInt/asIntN/bits-toindex.js (previously Passed)
test/built-ins/BigInt/asIntN/not-a-constructor.js [strict mode] (previously Passed)
test/built-ins/BigInt/asIntN/not-a-constructor.js (previously Passed)
test/built-ins/BigInt/asIntN/name.js [strict mode] (previously Passed)
test/built-ins/BigInt/asIntN/name.js (previously Passed)
test/built-ins/BigInt/asIntN/length.js [strict mode] (previously Passed)
test/built-ins/BigInt/asIntN/length.js (previously Passed)
test/built-ins/BigInt/asIntN/order-of-steps.js [strict mode] (previously Passed)
test/built-ins/BigInt/asIntN/order-of-steps.js (previously Passed)
test/built-ins/BigInt/asIntN/asIntN.js [strict mode] (previously Passed)
test/built-ins/BigInt/asIntN/asIntN.js (previously Passed)
test/built-ins/BigInt/prototype/proto.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/proto.js (previously Passed)
test/built-ins/BigInt/prototype/Symbol.toStringTag.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/Symbol.toStringTag.js (previously Passed)
test/built-ins/BigInt/prototype/prop-desc.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/prop-desc.js (previously Passed)
test/built-ins/BigInt/prototype/constructor.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/constructor.js (previously Passed)
test/built-ins/BigInt/prototype/toString/not-a-constructor.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/toString/not-a-constructor.js (previously Passed)
test/built-ins/BigInt/prototype/toString/name.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/toString/name.js (previously Passed)
test/built-ins/BigInt/prototype/toString/prop-desc.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/toString/prop-desc.js (previously Passed)
test/built-ins/BigInt/prototype/toString/string-is-code-units-of-decimal-digits-only.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/toString/string-is-code-units-of-decimal-digits-only.js (previously Passed)
test/built-ins/BigInt/prototype/toLocaleString/not-a-constructor.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/toLocaleString/not-a-constructor.js (previously Passed)
test/built-ins/BigInt/prototype/valueOf/this-value-invalid-primitive-throws.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/valueOf/this-value-invalid-primitive-throws.js (previously Passed)
test/built-ins/BigInt/prototype/valueOf/not-a-constructor.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/valueOf/not-a-constructor.js (previously Passed)
test/built-ins/BigInt/prototype/valueOf/this-value-invalid-object-throws.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/valueOf/this-value-invalid-object-throws.js (previously Passed)
test/built-ins/BigInt/prototype/valueOf/name.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/valueOf/name.js (previously Passed)
test/built-ins/BigInt/prototype/valueOf/prop-desc.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/valueOf/prop-desc.js (previously Passed)
test/built-ins/BigInt/prototype/valueOf/length.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/valueOf/length.js (previously Passed)
test/built-ins/BigInt/prototype/valueOf/return.js [strict mode] (previously Passed)
test/built-ins/BigInt/prototype/valueOf/return.js (previously Passed)
test/built-ins/BigInt/parseInt/nonexistent.js [strict mode] (previously Passed)
test/built-ins/BigInt/parseInt/nonexistent.js (previously Passed)
test/built-ins/BigInt/asUintN/asUintN.js [strict mode] (previously Passed)
test/built-ins/BigInt/asUintN/asUintN.js (previously Passed)
test/built-ins/BigInt/asUintN/arithmetic.js [strict mode] (previously Passed)
test/built-ins/BigInt/asUintN/arithmetic.js (previously Passed)
test/built-ins/BigInt/asUintN/bits-toindex.js [strict mode] (previously Passed)
test/built-ins/BigInt/asUintN/bits-toindex.js (previously Passed)
test/built-ins/BigInt/asUintN/not-a-constructor.js [strict mode] (previously Passed)
test/built-ins/BigInt/asUintN/not-a-constructor.js (previously Passed)
test/built-ins/BigInt/asUintN/name.js [strict mode] (previously Passed)
test/built-ins/BigInt/asUintN/name.js (previously Passed)
test/built-ins/BigInt/asUintN/length.js [strict mode] (previously Passed)
test/built-ins/BigInt/asUintN/length.js (previously Passed)
test/built-ins/BigInt/asUintN/order-of-steps.js [strict mode] (previously Passed)
test/built-ins/BigInt/asUintN/order-of-steps.js (previously Passed)
test/built-ins/Object/bigint.js [strict mode] (previously Passed)
test/built-ins/Object/bigint.js (previously Passed)
test/built-ins/Object/prototype/toString/Object.prototype.toString.call-bigint.js [strict mode] (previously Passed)
test/built-ins/Object/prototype/toString/Object.prototype.toString.call-bigint.js (previously Passed)
test/built-ins/Object/prototype/toString/symbol-tag-override-bigint.js [strict mode] (previously Passed)
test/built-ins/Object/prototype/toString/symbol-tag-override-bigint.js (previously Passed)

New panics:

test/built-ins/RegExp/prototype/exec/failure-g-lastindex-reset.js (previously Failed)

@github-actions
Copy link

Benchmark for 08cc474

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 328.2±14.88ns 330.1±12.47ns -0.58%
Arithmetic operations (Full) 265.3±11.46µs 268.5±9.37µs -1.19%
Array access (Execution) 6.2±0.28µs 6.3±0.22µs -1.59%
Array access (Full) 292.9±9.58µs 309.2±33.87µs -5.27%
Array creation (Execution) 2.6±0.07ms 2.6±0.06ms 0.00%
Array creation (Full) 2.9±0.11ms 2.9±0.15ms 0.00%
Array pop (Execution) 847.6±64.42µs 862.4±28.54µs -1.72%
Array pop (Full) 1312.4±69.52µs 1301.5±41.40µs +0.84%
Boolean Object Access (Execution) 5.1±0.21µs 5.1±0.37µs 0.00%
Boolean Object Access (Full) 284.0±12.90µs 292.9±13.04µs -3.04%
Clean js (Execution) 625.8±22.57µs 630.4±23.29µs -0.73%
Clean js (Full) 923.1±31.53µs 928.4±26.37µs -0.57%
Clean js (Parser) 39.7±1.33µs 45.6±2.41µs -12.94%
Create Realm 410.7±19.55ns 385.5±11.39ns +6.54%
Dynamic Object Property Access (Execution) 5.1±0.18µs 5.2±0.22µs -1.92%
Dynamic Object Property Access (Full) 289.3±12.01µs 293.2±20.61µs -1.33%
Expression (Parser) 6.8±0.27µs 7.9±0.95µs -13.92%
Fibonacci (Execution) 796.0±37.29µs 801.2±25.17µs -0.65%
Fibonacci (Full) 1073.1±41.95µs 1114.8±66.24µs -3.74%
For loop (Execution) 21.6±1.02µs 21.8±0.71µs -0.92%
For loop (Full) 301.9±9.88µs 300.8±10.03µs +0.37%
For loop (Parser) 19.0±0.57µs 19.5±1.10µs -2.56%
Goal Symbols (Parser) 13.7±1.26µs 15.2±0.99µs -9.87%
Hello World (Parser) 3.8±0.15µs 3.8±0.15µs 0.00%
Long file (Parser) 716.5±29.47ns 807.6±26.94ns -11.28%
Mini js (Execution) 552.6±13.88µs 576.0±30.29µs -4.06%
Mini js (Full) 864.3±30.65µs 912.1±61.47µs -5.24%
Mini js (Parser) 34.7±1.47µs 40.2±2.85µs -13.68%
Number Object Access (Execution) 4.1±0.44µs 4.1±0.22µs 0.00%
Number Object Access (Full) 279.9±11.10µs 283.5±11.74µs -1.27%
Object Creation (Execution) 4.3±0.15µs 4.4±0.21µs -2.27%
Object Creation (Full) 288.9±15.25µs 285.2±11.01µs +1.30%
RegExp (Execution) 10.7±0.32µs 11.0±0.63µs -2.73%
RegExp (Full) 310.1±13.93µs 300.0±20.83µs +3.37%
RegExp Literal (Execution) 10.9±0.45µs 10.9±0.59µs 0.00%
RegExp Literal (Full) 295.1±8.42µs 298.8±12.21µs -1.24%
RegExp Literal Creation (Execution) 9.4±0.41µs 9.4±0.40µs 0.00%
RegExp Literal Creation (Full) 288.7±8.87µs 293.0±9.83µs -1.47%
Static Object Property Access (Execution) 4.7±0.19µs 4.7±0.23µs 0.00%
Static Object Property Access (Full) 297.1±20.96µs 286.8±10.86µs +3.59%
String Object Access (Execution) 7.3±0.49µs 7.1±0.21µs +2.82%
String Object Access (Full) 290.6±11.57µs 288.6±8.99µs +0.69%
String comparison (Execution) 6.4±0.36µs 6.4±0.27µs 0.00%
String comparison (Full) 294.9±15.30µs 295.2±14.04µs -0.10%
String concatenation (Execution) 5.1±0.20µs 5.2±0.17µs -1.92%
String concatenation (Full) 282.6±9.74µs 283.4±10.09µs -0.28%
String copy (Execution) 4.0±0.17µs 4.0±0.18µs 0.00%
String copy (Full) 281.3±33.90µs 284.4±19.90µs -1.09%
Symbols (Execution) 3.3±0.11µs 3.4±0.09µs -2.94%
Symbols (Full) 271.2±24.27µs 269.4±14.43µs +0.67%

@github-actions
Copy link

Benchmark for e092ba4

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 360.2±4.67ns 362.6±0.53ns -0.66%
Arithmetic operations (Full) 270.1±2.51µs 272.3±0.85µs -0.81%
Array access (Execution) 6.2±0.03µs 6.2±0.02µs 0.00%
Array access (Full) 295.3±0.78µs 293.4±1.19µs +0.65%
Array creation (Execution) 2.9±0.01ms 2.9±0.01ms 0.00%
Array creation (Full) 3.3±0.06ms 3.3±0.02ms 0.00%
Array pop (Execution) 933.9±5.22µs 944.3±9.86µs -1.10%
Array pop (Full) 1436.2±7.15µs 1443.4±5.07µs -0.50%
Boolean Object Access (Execution) 5.3±0.05µs 5.3±0.02µs 0.00%
Boolean Object Access (Full) 292.2±3.21µs 291.3±0.66µs +0.31%
Clean js (Execution) 659.3±6.28µs 662.5±3.91µs -0.48%
Clean js (Full) 987.7±4.31µs 1013.4±33.09µs -2.54%
Clean js (Parser) 41.2±0.08µs 41.1±0.21µs +0.24%
Create Realm 422.9±4.49ns 425.8±5.51ns -0.68%
Dynamic Object Property Access (Execution) 5.1±0.04µs 5.0±0.04µs +2.00%
Dynamic Object Property Access (Full) 289.9±0.95µs 292.6±1.92µs -0.92%
Expression (Parser) 7.3±0.03µs 7.3±0.04µs 0.00%
Fibonacci (Execution) 754.5±3.87µs 757.8±5.07µs -0.44%
Fibonacci (Full) 1074.9±4.88µs 1063.4±3.19µs +1.08%
For loop (Execution) 22.3±0.10µs 21.7±0.11µs +2.76%
For loop (Full) 305.9±6.51µs 310.1±0.71µs -1.35%
For loop (Parser) 19.9±0.07µs 19.8±0.11µs +0.51%
Goal Symbols (Parser) 14.1±0.04µs 14.1±0.06µs 0.00%
Hello World (Parser) 4.0±0.02µs 4.0±0.04µs 0.00%
Long file (Parser) 801.3±5.92ns 802.8±5.97ns -0.19%
Mini js (Execution) 595.4±8.83µs 600.2±4.41µs -0.80%
Mini js (Full) 914.6±4.18µs 926.9±6.32µs -1.33%
Mini js (Parser) 36.0±0.13µs 35.8±0.34µs +0.56%
Number Object Access (Execution) 4.1±0.04µs 4.1±0.03µs 0.00%
Number Object Access (Full) 286.2±3.85µs 287.1±2.36µs -0.31%
Object Creation (Execution) 4.4±0.03µs 4.3±0.01µs +2.33%
Object Creation (Full) 285.1±0.82µs 291.1±4.02µs -2.06%
RegExp (Execution) 11.9±0.06µs 11.8±0.10µs +0.85%
RegExp (Full) 300.4±2.32µs 300.9±3.51µs -0.17%
RegExp Literal (Execution) 11.9±0.06µs 11.9±0.13µs 0.00%
RegExp Literal (Full) 296.1±2.17µs 299.7±2.57µs -1.20%
RegExp Literal Creation (Execution) 10.5±0.04µs 10.4±0.04µs +0.96%
RegExp Literal Creation (Full) 292.8±2.94µs 288.9±3.64µs +1.35%
Static Object Property Access (Execution) 4.6±0.02µs 4.6±0.04µs 0.00%
Static Object Property Access (Full) 287.1±1.19µs 287.4±2.02µs -0.10%
String Object Access (Execution) 7.2±0.06µs 7.2±0.03µs 0.00%
String Object Access (Full) 292.7±2.76µs 296.1±0.82µs -1.15%
String comparison (Execution) 6.6±0.06µs 6.5±0.07µs +1.54%
String comparison (Full) 293.7±2.11µs 290.4±6.56µs +1.14%
String concatenation (Execution) 5.2±0.02µs 5.3±0.02µs -1.89%
String concatenation (Full) 283.1±2.80µs 287.9±4.22µs -1.67%
String copy (Execution) 4.0±0.02µs 4.0±0.03µs 0.00%
String copy (Full) 277.7±1.87µs 280.6±0.92µs -1.03%
Symbols (Execution) 3.3±0.02µs 3.3±0.03µs 0.00%
Symbols (Full) 273.4±5.09µs 272.6±0.63µs +0.29%

@github-actions
Copy link

Benchmark for 5b08782

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 307.9±0.12ns 303.4±0.30ns +1.48%
Arithmetic operations (Full) 225.9±0.22µs 225.7±0.34µs +0.09%
Array access (Execution) 5.1±0.03µs 5.2±0.02µs -1.92%
Array access (Full) 247.0±0.67µs 248.8±0.75µs -0.72%
Array creation (Execution) 2.4±0.00ms 2.4±0.00ms 0.00%
Array creation (Full) 2.8±0.00ms 2.8±0.00ms 0.00%
Array pop (Execution) 778.1±1.98µs 776.6±3.25µs +0.19%
Array pop (Full) 1203.0±1.78µs 1204.4±2.01µs -0.12%
Boolean Object Access (Execution) 4.4±0.03µs 4.4±0.01µs 0.00%
Boolean Object Access (Full) 244.6±3.60µs 246.3±0.21µs -0.69%
Clean js (Execution) 552.0±2.77µs 550.3±2.62µs +0.31%
Clean js (Full) 831.0±3.93µs 824.8±3.60µs +0.75%
Clean js (Parser) 36.5±0.03µs 36.5±0.03µs 0.00%
Create Realm 355.6±0.31ns 354.3±3.20ns +0.37%
Dynamic Object Property Access (Execution) 4.1±0.01µs 4.3±0.02µs -4.65%
Dynamic Object Property Access (Full) 246.5±0.37µs 243.9±0.28µs +1.07%
Expression (Parser) 6.1±0.01µs 6.2±0.01µs -1.61%
Fibonacci (Execution) 644.2±5.67µs 640.6±1.54µs +0.56%
Fibonacci (Full) 902.6±2.03µs 895.1±0.61µs +0.84%
For loop (Execution) 18.2±0.04µs 18.3±0.11µs -0.55%
For loop (Full) 256.1±0.91µs 256.4±0.50µs -0.12%
For loop (Parser) 17.3±0.02µs 17.5±0.04µs -1.14%
Goal Symbols (Parser) 12.3±0.07µs 12.3±0.02µs 0.00%
Hello World (Parser) 3.4±0.00µs 3.4±0.01µs 0.00%
Long file (Parser) 691.3±3.93ns 692.0±4.32ns -0.10%
Mini js (Execution) 497.4±5.25µs 497.2±2.03µs +0.04%
Mini js (Full) 776.3±2.34µs 770.1±2.73µs +0.81%
Mini js (Parser) 31.7±0.04µs 31.8±0.19µs -0.31%
Number Object Access (Execution) 3.5±0.00µs 3.5±0.00µs 0.00%
Number Object Access (Full) 239.2±0.39µs 239.5±0.33µs -0.13%
Object Creation (Execution) 3.6±0.01µs 3.6±0.01µs 0.00%
Object Creation (Full) 240.3±1.27µs 242.2±0.43µs -0.78%
RegExp (Execution) 9.9±0.01µs 10.0±0.02µs -1.00%
RegExp (Full) 254.7±0.60µs 253.7±0.30µs +0.39%
RegExp Literal (Execution) 9.9±0.02µs 10.1±0.02µs -1.98%
RegExp Literal (Full) 250.0±0.44µs 253.3±0.84µs -1.30%
RegExp Literal Creation (Execution) 8.7±0.02µs 8.7±0.02µs 0.00%
RegExp Literal Creation (Full) 244.2±0.36µs 246.9±0.44µs -1.09%
Static Object Property Access (Execution) 3.7±0.00µs 3.8±0.02µs -2.63%
Static Object Property Access (Full) 244.4±0.28µs 240.7±0.28µs +1.54%
String Object Access (Execution) 6.1±0.03µs 6.0±0.01µs +1.67%
String Object Access (Full) 247.7±0.30µs 249.3±0.32µs -0.64%
String comparison (Execution) 5.4±0.01µs 5.5±0.02µs -1.82%
String comparison (Full) 245.8±0.58µs 244.9±0.27µs +0.37%
String concatenation (Execution) 4.3±0.01µs 4.4±0.01µs -2.27%
String concatenation (Full) 239.4±0.31µs 239.0±0.49µs +0.17%
String copy (Execution) 3.3±0.01µs 3.3±0.01µs 0.00%
String copy (Full) 232.4±0.27µs 233.2±0.42µs -0.34%
Symbols (Execution) 2.8±0.01µs 2.8±0.01µs 0.00%
Symbols (Full) 228.5±0.70µs 228.8±0.47µs -0.13%

@github-actions
Copy link

Benchmark for d6c5a64

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 399.1±19.67ns 377.4±11.38ns +5.75%
Arithmetic operations (Full) 312.1±14.04µs 310.5±13.21µs +0.52%
Array access (Execution) 7.2±0.27µs 7.4±0.35µs -2.70%
Array access (Full) 351.3±39.63µs 348.5±25.48µs +0.80%
Array creation (Execution) 3.0±0.10ms 3.0±0.16ms 0.00%
Array creation (Full) 3.3±0.12ms 3.3±0.15ms 0.00%
Array pop (Execution) 975.0±30.43µs 972.7±29.86µs +0.24%
Array pop (Full) 1506.1±59.95µs 1507.5±61.17µs -0.09%
Boolean Object Access (Execution) 6.0±0.39µs 5.9±0.30µs +1.69%
Boolean Object Access (Full) 331.7±17.25µs 330.7±10.46µs +0.30%
Clean js (Execution) 738.0±34.23µs 731.0±25.33µs +0.96%
Clean js (Full) 1072.3±33.67µs 1063.8±37.80µs +0.80%
Clean js (Parser) 46.6±2.19µs 45.8±1.91µs +1.75%
Create Realm 466.2±14.19ns 476.1±22.14ns -2.08%
Dynamic Object Property Access (Execution) 6.1±0.36µs 6.0±0.33µs +1.67%
Dynamic Object Property Access (Full) 341.5±21.31µs 341.0±11.40µs +0.15%
Expression (Parser) 7.8±0.38µs 7.7±0.50µs +1.30%
Fibonacci (Execution) 923.4±36.85µs 925.8±33.46µs -0.26%
Fibonacci (Full) 1257.0±47.04µs 1268.4±68.29µs -0.90%
For loop (Execution) 24.9±1.15µs 24.9±0.99µs 0.00%
For loop (Full) 358.9±18.14µs 348.8±18.41µs +2.90%
For loop (Parser) 22.5±1.34µs 22.4±2.00µs +0.45%
Goal Symbols (Parser) 15.7±0.89µs 15.3±0.94µs +2.61%
Hello World (Parser) 4.4±0.22µs 4.2±0.14µs +4.76%
Long file (Parser) 846.7±40.79ns 823.1±25.25ns +2.87%
Mini js (Execution) 660.1±35.87µs 648.8±28.84µs +1.74%
Mini js (Full) 1000.8±45.24µs 991.9±29.94µs +0.90%
Mini js (Parser) 40.5±1.37µs 39.6±1.63µs +2.27%
Number Object Access (Execution) 4.7±0.24µs 4.7±0.18µs 0.00%
Number Object Access (Full) 325.0±12.77µs 334.7±20.65µs -2.90%
Object Creation (Execution) 5.2±0.41µs 5.1±0.30µs +1.96%
Object Creation (Full) 329.8±12.25µs 331.7±10.64µs -0.57%
RegExp (Execution) 12.5±0.69µs 12.7±0.71µs -1.57%
RegExp (Full) 347.0±22.64µs 351.1±20.73µs -1.17%
RegExp Literal (Execution) 12.5±1.08µs 12.3±0.51µs +1.63%
RegExp Literal (Full) 346.5±19.30µs 347.0±22.03µs -0.14%
RegExp Literal Creation (Execution) 10.8±0.57µs 10.7±0.67µs +0.93%
RegExp Literal Creation (Full) 348.0±16.59µs 349.7±81.88µs -0.49%
Static Object Property Access (Execution) 5.3±0.22µs 5.4±0.31µs -1.85%
Static Object Property Access (Full) 330.6±12.42µs 338.4±15.12µs -2.30%
String Object Access (Execution) 8.3±0.54µs 8.2±0.41µs +1.22%
String Object Access (Full) 335.3±18.52µs 339.7±16.21µs -1.30%
String comparison (Execution) 7.4±0.41µs 7.4±0.40µs 0.00%
String comparison (Full) 335.1±11.54µs 335.0±14.50µs +0.03%
String concatenation (Execution) 6.0±0.32µs 6.0±0.24µs 0.00%
String concatenation (Full) 331.6±16.73µs 333.5±19.21µs -0.57%
String copy (Execution) 4.7±0.27µs 4.7±0.50µs 0.00%
String copy (Full) 323.9±15.14µs 327.6±21.90µs -1.13%
Symbols (Execution) 3.8±0.14µs 4.0±0.18µs -5.00%
Symbols (Full) 313.5±12.15µs 317.6±23.03µs -1.29%

@0x7D2B
Copy link
Contributor

0x7D2B commented May 25, 2021 •
edited
Loading

Can you put those tests lists inside a <details> tag like what happens with the benchmark?

RageKnify
RageKnify approved these changes May 25, 2021
View reviewed changes
Copy link
Contributor

@RageKnify RageKnify left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x7D2B's idea sounds like a nice quality-of-life improvement, other than that it looks great.

0x7D2B
0x7D2B suggested changes May 25, 2021
View reviewed changes
Copy link
Contributor

@0x7D2B 0x7D2B left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think test names should be wrapped in <details> tags like the benchmarks. Also I wanted to try leaving a review~

@Razican
Copy link
Member Author

Razican commented May 25, 2021

I think test names should be wrapped in <details> tags like the benchmarks. Also I wanted to try leaving a review~

Should be ready now :)

@github-actions
Copy link

Benchmark for 806064d

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 359.1±10.65ns 367.1±3.60ns -2.18%
Arithmetic operations (Full) 269.0±4.71µs 268.9±2.67µs +0.04%
Array access (Execution) 6.1±0.03µs 6.1±0.05µs 0.00%
Array access (Full) 296.4±2.27µs 295.4±1.82µs +0.34%
Array creation (Execution) 3.1±0.02ms 3.1±0.01ms 0.00%
Array creation (Full) 3.3±0.02ms 3.3±0.04ms 0.00%
Array pop (Execution) 983.0±6.69µs 982.3±5.43µs +0.07%
Array pop (Full) 1451.8±21.65µs 1470.2±9.50µs -1.25%
Boolean Object Access (Execution) 5.2±0.06µs 5.2±0.05µs 0.00%
Boolean Object Access (Full) 292.5±0.98µs 288.2±2.07µs +1.49%
Clean js (Execution) 665.9±5.19µs 662.8±10.19µs +0.47%
Clean js (Full) 978.5±8.21µs 986.3±5.67µs -0.79%
Clean js (Parser) 40.9±0.37µs 41.3±0.10µs -0.97%
Create Realm 431.4±1.19ns 428.9±6.92ns +0.58%
Dynamic Object Property Access (Execution) 5.1±0.06µs 5.1±0.04µs 0.00%
Dynamic Object Property Access (Full) 292.5±1.99µs 294.9±0.44µs -0.81%
Expression (Parser) 7.3±0.04µs 7.4±0.08µs -1.35%
Fibonacci (Execution) 752.7±6.32µs 756.8±19.56µs -0.54%
Fibonacci (Full) 1067.0±11.38µs 1066.2±9.96µs +0.08%
For loop (Execution) 21.9±0.11µs 21.8±0.20µs +0.46%
For loop (Full) 303.2±3.11µs 305.5±3.09µs -0.75%
For loop (Parser) 19.8±0.25µs 19.9±0.19µs -0.50%
Goal Symbols (Parser) 14.1±0.11µs 14.1±0.07µs 0.00%
Hello World (Parser) 4.0±0.03µs 4.0±0.02µs 0.00%
Long file (Parser) 785.9±9.17ns 803.2±7.60ns -2.15%
Mini js (Execution) 603.8±6.48µs 602.7±8.81µs +0.18%
Mini js (Full) 916.2±7.65µs 916.4±8.39µs -0.02%
Mini js (Parser) 35.7±0.21µs 36.0±0.07µs -0.83%
Number Object Access (Execution) 4.1±0.03µs 4.1±0.03µs 0.00%
Number Object Access (Full) 283.7±1.63µs 285.3±1.73µs -0.56%
Object Creation (Execution) 4.3±0.04µs 4.3±0.03µs 0.00%
Object Creation (Full) 286.5±2.94µs 288.0±1.68µs -0.52%
RegExp (Execution) 10.9±0.08µs 10.9±0.15µs 0.00%
RegExp (Full) 300.4±2.01µs 304.7±4.53µs -1.41%
RegExp Literal (Execution) 10.8±0.21µs 10.9±0.10µs -0.92%
RegExp Literal (Full) 296.9±2.64µs 299.3±2.99µs -0.80%
RegExp Literal Creation (Execution) 9.4±0.06µs 9.4±0.08µs 0.00%
RegExp Literal Creation (Full) 291.6±5.18µs 292.3±0.64µs -0.24%
Static Object Property Access (Execution) 4.6±0.51µs 4.6±0.03µs 0.00%
Static Object Property Access (Full) 289.9±7.20µs 292.4±0.91µs -0.85%
String Object Access (Execution) 7.2±0.06µs 7.2±0.09µs 0.00%
String Object Access (Full) 295.4±1.35µs 292.2±2.28µs +1.10%
String comparison (Execution) 6.5±0.04µs 6.5±0.07µs 0.00%
String comparison (Full) 288.7±1.37µs 291.5±3.62µs -0.96%
String concatenation (Execution) 5.2±0.06µs 5.4±0.06µs -3.70%
String concatenation (Full) 285.2±3.56µs 283.7±2.03µs +0.53%
String copy (Execution) 4.0±0.03µs 3.9±0.04µs +2.56%
String copy (Full) 277.3±1.05µs 280.6±5.20µs -1.18%
Symbols (Execution) 3.3±0.02µs 3.3±0.04µs 0.00%
Symbols (Full) 271.5±2.99µs 271.6±2.11µs -0.04%

0x7D2B
0x7D2B suggested changes May 25, 2021
View reviewed changes
boa_tester/src/results.rs Outdated Show resolved Hide resolved
boa_tester/src/results.rs Outdated Show resolved Hide resolved
boa_tester/src/results.rs Outdated Show resolved Hide resolved
boa_tester/src/results.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

Benchmark for fe18fd7

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 305.4±2.69ns 310.0±0.28ns -1.48%
Arithmetic operations (Full) 225.6±0.21µs 224.0±0.41µs +0.71%
Array access (Execution) 5.4±0.02µs 5.4±0.02µs 0.00%
Array access (Full) 249.8±0.65µs 248.1±0.63µs +0.69%
Array creation (Execution) 2.4±0.00ms 2.4±0.00ms 0.00%
Array creation (Full) 2.8±0.01ms 2.8±0.00ms 0.00%
Array pop (Execution) 775.5±3.21µs 774.0±1.90µs +0.19%
Array pop (Full) 1204.7±10.50µs 1203.3±1.90µs +0.12%
Boolean Object Access (Execution) 4.4±0.00µs 4.4±0.01µs 0.00%
Boolean Object Access (Full) 240.7±1.14µs 244.2±0.48µs -1.43%
Clean js (Execution) 511.2±2.14µs 512.7±2.61µs -0.29%
Clean js (Full) 791.7±2.30µs 790.1±3.57µs +0.20%
Clean js (Parser) 36.2±0.04µs 36.2±0.07µs 0.00%
Create Realm 360.9±0.19ns 358.7±0.32ns +0.61%
Dynamic Object Property Access (Execution) 4.4±0.03µs 4.5±0.01µs -2.22%
Dynamic Object Property Access (Full) 245.2±0.22µs 245.5±0.60µs -0.12%
Expression (Parser) 6.2±0.00µs 6.2±0.00µs 0.00%
Fibonacci (Execution) 690.6±1.72µs 682.1±0.93µs +1.25%
Fibonacci (Full) 950.9±2.12µs 949.7±1.17µs +0.13%
For loop (Execution) 18.4±0.04µs 18.5±0.02µs -0.54%
For loop (Full) 258.8±7.05µs 257.2±0.62µs +0.62%
For loop (Parser) 17.3±0.03µs 17.2±0.02µs +0.58%
Goal Symbols (Parser) 12.3±0.02µs 12.2±0.02µs +0.82%
Hello World (Parser) 3.4±0.02µs 3.4±0.01µs 0.00%
Long file (Parser) 718.6±4.71ns 719.4±5.92ns -0.11%
Mini js (Execution) 460.0±2.37µs 466.2±2.72µs -1.33%
Mini js (Full) 733.6±2.11µs 734.9±1.93µs -0.18%
Mini js (Parser) 31.5±0.07µs 31.7±0.11µs -0.63%
Number Object Access (Execution) 3.4±0.00µs 3.4±0.00µs 0.00%
Number Object Access (Full) 237.7±0.20µs 240.0±3.63µs -0.96%
Object Creation (Execution) 3.8±0.00µs 3.8±0.02µs 0.00%
Object Creation (Full) 240.8±0.42µs 240.1±0.72µs +0.29%
RegExp (Execution) 9.3±0.04µs 9.4±0.02µs -1.06%
RegExp (Full) 252.4±0.61µs 253.4±3.34µs -0.39%
RegExp Literal (Execution) 9.3±0.03µs 9.3±0.03µs 0.00%
RegExp Literal (Full) 249.0±0.43µs 249.0±0.46µs 0.00%
RegExp Literal Creation (Execution) 8.1±0.02µs 8.2±0.03µs -1.22%
RegExp Literal Creation (Full) 244.4±0.49µs 245.2±1.60µs -0.33%
Static Object Property Access (Execution) 4.0±0.01µs 4.0±0.01µs 0.00%
Static Object Property Access (Full) 239.8±0.27µs 244.2±0.83µs -1.80%
String Object Access (Execution) 6.0±0.01µs 6.1±0.02µs -1.64%
String Object Access (Full) 244.4±0.30µs 244.5±0.29µs -0.04%
String comparison (Execution) 5.6±0.02µs 5.5±0.02µs +1.82%
String comparison (Full) 242.3±1.61µs 242.5±0.32µs -0.08%
String concatenation (Execution) 4.5±0.02µs 4.4±0.02µs +2.27%
String concatenation (Full) 235.8±0.50µs 236.7±0.34µs -0.38%
String copy (Execution) 3.5±0.01µs 3.4±0.01µs +2.94%
String copy (Full) 232.4±0.56µs 231.9±0.46µs +0.22%
Symbols (Execution) 2.9±0.01µs 2.9±0.01µs 0.00%
Symbols (Full) 228.5±0.42µs 227.7±0.33µs +0.35%

@0x7D2B 0x7D2B merged commit 1a58c9f into master May 29, 2021
@RageKnify RageKnify deleted the test262_compare branch May 29, 2021 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@0x7D2B 0x7D2B 0x7D2B approved these changes

@RageKnify RageKnify RageKnify approved these changes

@HalidOdat HalidOdat HalidOdat approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request test Issues and PRs related to the tests.
Projects
None yet
Milestone
v0.12.0
Development

Successfully merging this pull request may close these issues.

More detailed information for test conformance changes
4 participants
@Razican @0x7D2B @HalidOdat @RageKnify