Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upload core dumps from failed tests in CI #1850

Merged
merged 3 commits into from
Nov 2, 2023
+29 −12
Merged

upload core dumps from failed tests in CI #1850

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c56dfa5
upgrade to actions/checkout@v4
spoonincode Oct 26, 2023
ec22785
upgrade parallel-ctest-containers action to node20
spoonincode Oct 26, 2023
72e0e0d
upload core files from failed tests
spoonincode Oct 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/parallel-ctest-containers/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ inputs:
test-timeout:
required: true
runs:
using: 'node16'
using: 'node20'
main: 'dist/index.mjs'
35 changes: 26 additions & 9 deletions .github/workflows/build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ jobs:
runs-on: ubuntu-latest
container: ${{fromJSON(needs.platform-cache.outputs.platforms)[matrix.platform].image}}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Download builddir
Expand Down Expand Up @@ -138,9 +138,9 @@ jobs:
runs-on: ["self-hosted", "enf-x86-hightier"]
container:
image: ${{fromJSON(needs.platform-cache.outputs.platforms)[matrix.cfg.base].image}}
options: --security-opt seccomp=unconfined
options: --security-opt seccomp=unconfined --mount type=bind,source=/var/lib/systemd/coredump,target=/cores
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Download builddir
uses: actions/download-artifact@v3
with:
Expand All @@ -152,6 +152,13 @@ jobs:
zstdcat build.tar.zst | tar x
cd build
ctest --output-on-failure -j $(nproc) -LE "(nonparallelizable_tests|long_running_tests)" --timeout 420
- name: Upload core files from failed tests
uses: actions/upload-artifact@v3
if: failure()
with:
name: ${{matrix.cfg.name}}-tests-logs
if-no-files-found: ignore
path: /cores
- name: Check CPU Features
run: awk 'BEGIN {err = 1} /bmi2/ && /adx/ {err = 0} END {exit err}' /proc/cpuinfo

Expand All @@ -168,7 +175,7 @@ jobs:
- cfg: {name: 'ubuntu22repro', base: 'ubuntu22', builddir: 'reproducible'}
runs-on: ["self-hosted", "enf-x86-midtier"]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Download builddir
uses: actions/download-artifact@v3
with:
Expand All @@ -181,12 +188,17 @@ jobs:
log-tarball-prefix: ${{matrix.cfg.name}}
tests-label: nonparallelizable_tests
test-timeout: 420
- name: Export core dumps
run: docker run --mount type=bind,source=/var/lib/systemd/coredump,target=/cores alpine sh -c 'tar -C /cores/ -c .' | tar x
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this looks weird, and to add more confusion it's different than the parallel tests job because NP/LR tests don't run the job in a container until parallel-ctest-containers step.

On ENF runners the user is not root but rather a user named enf that does not have sudo privileges (this is unlike GitHub runners). But /var/lib/systemd/coredump files are only readable by root. Since docker is not creating a userns in this case, we can use docker as a cheat to get access to the files without sudo and without addtional modificatons on the ENF runners.

if: failure()
- name: Upload logs from failed tests
uses: actions/upload-artifact@v3
if: failure()
with:
name: ${{matrix.cfg.name}}-np-logs
path: '*-logs.tar.gz'
path: |
*-logs.tar.gz
core*.zst

lr-tests:
name: LR Tests (${{matrix.cfg.name}})
Expand All @@ -201,7 +213,7 @@ jobs:
- cfg: {name: 'ubuntu22repro', base: 'ubuntu22', builddir: 'reproducible'}
runs-on: ["self-hosted", "enf-x86-lowtier"]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Download builddir
uses: actions/download-artifact@v3
with:
Expand All @@ -214,12 +226,17 @@ jobs:
log-tarball-prefix: ${{matrix.cfg.name}}
tests-label: long_running_tests
test-timeout: 1800
- name: Export core dumps
run: docker run --mount type=bind,source=/var/lib/systemd/coredump,target=/cores alpine sh -c 'tar -C /cores/ -c .' | tar x
if: failure()
- name: Upload logs from failed tests
uses: actions/upload-artifact@v3
if: failure()
with:
name: ${{matrix.cfg.name}}-lr-logs
path: '*-logs.tar.gz'
path: |
*-logs.tar.gz
core*.zst

libtester-tests:
name: libtester tests
Expand All @@ -243,7 +260,7 @@ jobs:
# LEAP
- if: ${{ matrix.test != 'deb-install' }}
name: Clone leap
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
- if: ${{ matrix.test != 'deb-install' }}
Expand Down Expand Up @@ -296,7 +313,7 @@ jobs:

# Reference Contracts
- name: checkout eos-system-contracts
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: eosnetworkfoundation/eos-system-contracts
path: eos-system-contracts
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build_base.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
runs-on: ["self-hosted", "enf-x86-beefy"]
container: ${{fromJSON(inputs.platforms)[matrix.platform].image}}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Build
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ph_backward_compatibility.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
image: ${{fromJSON(needs.platform-cache.outputs.platforms)[matrix.platform].image}}
options: --security-opt seccomp=unconfined
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Download builddir
uses: actions/download-artifact@v3
with:
Expand Down