Flaky tests runner

[wrnrlr]

Version: Deno 1.41.3

Here are 3 screenshots I took one after the other where for exactly the same test Deno responded with 3 different behaviours only one of which is correct.

This is not the only way in which Deno is broken. Another behaviour that I see it simply won't run the code inside the tests:

[mash-graz]

yes -- as long as the progress indicators are still circling around the yellow dots, you feel excited like in an online casino ;)

Just use git commit --amend --no-edit and git push -f to get another chance if the error looks utterly unbelievable.

[wrnrlr]

Sorry I don't understand your riddle or how or why this is solving my issue.

[mash-graz]

sorry -- I was writing about very similar experiences while contributing PRs to the deno github repo and getting similar confusing and always changing error reports back from the CI system, but you are talking about the behavior on your own local hardware.

So it's definitely not an issue related to your local setup but happening also in other execution contexts. But I have to say, that I did not see any of those flaky error reports during many local test runs her on my machine recently.

[0f-0b]

describe is used to group related tests. Actual test code should be put in an it block or a before*/after* hook. For example, you can initialize node in a beforeAll hook:

import { beforeAll, describe, it } from "jsr:@std/testing@^0.220.1/bdd";
import { assertEquals } from "jsr:@std/assert@^0.220.1/assert_equals";

describe("DOMNode", () => {
  let node;

  beforeAll(async () => {
    node = await xmlFromString(…);
  });

  it("in root namespace", () => assertEquals(…));
  it("in leaf namespace", () => assertEquals(…));
});

Deno.test equivalent

import { assertEquals } from "jsr:@std/assert@^0.220.1/assert_equals";

Deno.test("DOMNode", async (t) => {
  let node = await xmlFromString(…);

  await t.step("in root namespace", () => assertEquals(…));
  await t.step("in leaf namespace", () => assertEquals(…));
});

The error message could definitely be improved. Jest throws a helpful error if the callback passed to describe returns a promise or anything other than undefined.

[lucacasonato]

@0f-0b is right. I am closing this and have opened a follow up in std to add an error message when one passes an async function to describe: denoland/std#5034