Babel tests

[nicolo-ribaudo]

We run test262 on Babel and core-js regularly, and it would be very nice to show the results on https://test262.fyi.

I just noticed that our test262 runner on CI is completely broken and it's not reporting results, but for the purpose of this issue let's assume that it works, while I fix it 😅

We run test262 on every commit, and store the result in a test262-result artifact -- you can find the last one https://github.com/babel/babel/actions/runs/5179541745.

We run tests by compiling them to ES5 using Babel, injecting the core-js polyfill, and running the resulting code in a very old version of Node.js (0.10) which only supported ES5. This means that our test results aren't significant for pre-ES6 behavior, but they are useful for features introduced since then.

We use https://github.com/babel/babel-test262-runner/ to run them, which produces tap output. The readme doesn't mention it, but it's possible to specify the path to a different test262 checkout using the TEST262_PATH=.... env variable.

[nicolo-ribaudo]

Babel CI and test runner fixed! The test262-results artifact at https://github.com/babel/babel/actions/runs/5198849397 not contains the results.

[nicolo-ribaudo]

Something is still broken, many more tests than expected are failing.

[CanadaHonk]

Going to try and add this using the existing harness today :)

[CanadaHonk]

Also btw @nicolo-ribaudo, I maintain a fork of eshost (no node changes yet) and test262-stream (more accurate results) if you have requests / want to switch to them later in Babel's runner for best results.

[nicolo-ribaudo]

Thank you, I'll take a look!

I fixed for real Babel's test262 runner, it's now generating the results at https://github.com/babel/babel/actions/runs/5198849397

[CanadaHonk]

After a lot of tinkering, Babel builds should now be generated with the same harness (after some updates) while using Babel's test262 transpile script.

[CanadaHonk]

Results are on site! Not perfect/very accurate results but works :)

[nicolo-ribaudo]

Oh awesome :) Let's keep this issue open for a while, so I can check if there is anything that can be improved for the Babel runner and I'll post comments here.

[linusg]

This seems to be working as intended :)