Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: fix failing tests #1248

Merged
merged 21 commits into from
Jun 21, 2024
Merged

ci: fix failing tests #1248

merged 21 commits into from
Jun 21, 2024

Conversation

sandydoo
Copy link
Member

@sandydoo sandydoo commented Jun 1, 2024

Fixed examples:

  • examples/rust-wasm-cross
  • examples/phoenix
  • examples/python-django
  • examples/opentelemety-collector
    won't launch on macos-13. firewall issue? Tried disabling firewall, still nada.
    the readiness_probe on clickhouse is broken on macos13. curl-ing against the http port works. Only happens in GHA 🤷 clickhouse-client works as expected.
  • examples/supported-languages
    odin needs nixpkgs bump for darwin fixes.
    unison-ucm doesn't have aarch64-linux builds.
  • examples/ruby-on-rails: flaky on macos. timeouts too short? Seems to be working now.

Fixed languages:

  • languages/odin made gdb and ols optional depending on system.

Broken tests:

  • devenv-test-cli
  • devenv test
  • tests/mysql fails on macos

Other changes:

  • Changed .setup.sh to run inside the devenv shell. This helps test manual user setup, like initializing a project and installing dependencies.
  • Implement .patch.sh script support in devenv-run-tests. This runs outside of the shell, before .setup.sh. This should be used to patch files, like devenv.nix.
  • Replaced uses of example.com with localhost.

Copy link

cloudflare-workers-and-pages bot commented Jun 1, 2024

Deploying devenv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 5a3b934
Status: ✅  Deploy successful!
Preview URL: https://ade4f31b.devenv.pages.dev
Branch Preview URL: https://fixup-tests.devenv.pages.dev

View logs

@sandydoo sandydoo force-pushed the fixup-tests branch 3 times, most recently from 5318690 to 1577956 Compare June 1, 2024 17:00
@sandydoo
Copy link
Member Author

sandydoo commented Jun 1, 2024

thinking about it some more, the .patch.sh is more of a workaround for whatever the underlying issue is. Why does running the setup stuff break in enterShell, but not when run before the processes are launched? Race condition?

@sandydoo
Copy link
Member Author

sandydoo commented Jun 4, 2024

Alright, I'm out of steam. The last two failures are on macOS.

  1. examples/opentelemetry-collector — the clickhouse healthcheck fails to go green on the hosted x86 runner. I used tmate to run it manually, and as a separate process, and it works correctly in those situations. The http health check works as expected as well. Super weird!
  2. tests/mysql — fails on our self-hosted macos runner. There's a warning about not being able to resolve the hostname. Haven't looked into it too much.

@sandydoo sandydoo marked this pull request as ready for review June 4, 2024 14:12
@sandydoo sandydoo marked this pull request as draft June 4, 2024 14:12
.args(devenv_override)
.args(overrides.clone())
.arg("shell")
.arg("./.setup.sh")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is the same as enterShell, but it's not racy so that's why it works better?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. The problem we have now is that this sets things up in .devenv, but then devenv test uses a temporary directory. We can work around this by librarifying devenv and setting up our own temp dir.

@sandydoo
Copy link
Member Author

Needs #1263.

@sandydoo sandydoo force-pushed the fixup-tests branch 4 times, most recently from 7bf30ec to a35e90e Compare June 19, 2024 12:35
@domenkozar domenkozar marked this pull request as ready for review June 20, 2024 11:24
@sandydoo sandydoo merged commit efd3f22 into main Jun 21, 2024
262 of 272 checks passed
@sandydoo sandydoo deleted the fixup-tests branch June 21, 2024 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants