Misc test flakyness fixes

[koivunej]

Assorted flakyness fixes from #5198, might not be flaky on main.

Migrate some tests using neon_simple_env to just neon_env_builder and using initial_tenant to make flakyness understanding easier. (Did not understand the flakyness of test_timeline_create_break_after_uninit_mark.)

test_download_remote_layers_api is flaky because we have no atomic "wait for WAL, checkpoint, wait for upload and do not receive any more WAL".

test_tenant_size fixes are just boilerplate which should had always existed; we should wait for the tenant to be active. similarly for test_timeline_delete.

test_timeline_size_post_checkpoint fails often for me with reading zero from metrics. Give it a few attempts.

[github-actions]

1644 tests run: 1571 passed, 0 failed, 73 skipped (full report)

---

Code coverage (full report)

• functions: 53.1% (7601 of 14319 functions)
• lines: 81.4% (44732 of 54934 lines)

---

_{The comment gets automatically updated with the latest test results
64a1749 at 2023-09-11T08:20:45.328Z :recycle:}

[koivunej]

• test_delete_tenant_exercise_crash_safety_failpoints[Check.RETRY_WITH_RESTART-noop-tenant-delete-before-create-local-mark-False]: release

and

• test_delete_tenant_exercise_crash_safety_failpoints[Check.RETRY_WITH_RESTART-noop-tenant-delete-before-create-local-mark-False]: release

Looks like a missing ... wait? I'll handle it later, it seems I am permanently collecting these forever PRs if I want to always fix the next flaky.

It seems to have been marked as flaky in the prev test run, so it needs more wait_tenant_become_active.

[koivunej]

f7a0114 was green, but cloud e2e failed, don't want to wait for it.

[koivunej]

Do we now have a requirement that branches need to be up to date? I needed to rebase but there were no conflicts.

[arpad-m]

Do we now have a requirement that branches need to be up to date? I needed to rebase but there were no conflicts.

From my understanding the message is not there because of a requirement for PRs, but to expose the button, so that one can get CI improvements more quickly to contributor PRs. It's faster to press the button than to ask contributors to update, or to push the update yourself manually. See this slack message.