Move logic for ingest benchmark from GitHub workflow into python testcase

[Bodobolero]

Problem

The first version of the ingest benchmark had some parsing and reporting logic in shell script inside GitHub workflow.
it is better to move that logic into a python testcase so that we can also run it locally.

Summary of changes

• Create new python testcase

• invoke pgcopydb inside python test case

• move the following logic into python testcase

  • determine backpressure
  • invoke pgcopydb and report its progress
  • parse pgcopydb log and extract metrics
  • insert metrics into perf test database

• add additional column to perf test database that can receive endpoint ID used for pgcopydb run to have it available in grafana dashboard when retrieving other metrics for an endpoint

Example run

https://github.com/neondatabase/neon/actions/runs/11860622170/job/33056264386

[github-actions]

5499 tests run: 5273 passed, 0 failed, 226 skipped (full report)

---

Flaky tests (1)

Postgres 14

• test_isolation[None]: release-arm64

Code coverage* (full report)

• functions: 31.5% (7934 of 25219 functions)
• lines: 49.6% (62948 of 126918 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
8dab650 at 2024-11-19T09:50:17.038Z :recycle:}

[bayandin]

I've added a couple of comments where I spot a different approach with the current setup of performance tests

[bayandin]

Looks good!
Added a couple of comments, most of them for consistency with other benchmarks