Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upload all test case results in the reportportal plugin #3135

Merged
merged 6 commits into from
Aug 27, 2024
Merged

Upload all test case results in the reportportal plugin #3135

merged 6 commits into from
Aug 27, 2024

Conversation

kkaarreell
Copy link
Collaborator

@kkaarreell kkaarreell commented Aug 6, 2024 •
edited by happz
Loading

  • Upload all test results for a given test case, not just the first one.
  • Store guest name and role as test case attributes.
  • Rename get_rp_api() to rp_api_get()
  • Add and use rp_api_post() and rp_api_put()

Fixes issue #3129

Pull Request Checklist

  • implement the feature
  • write the documentation
  • extend the test coverage
  • update the specification
  • adjust plugin docstring
  • modify the json schema
  • mention the version
  • include a release note

@kkaarreell kkaarreell force-pushed the ks_reportportal_multi branch 6 times, most recently from c97f6e0 to 30cf239 Compare August 7, 2024 07:04
@kkaarreell
Copy link
Collaborator Author

To ease the review pls use
https://app.semanticdiff.com/gh/teemtee/tmt/pull/3135/changes#tmt/steps/report/reportportal.py

@kkaarreell kkaarreell force-pushed the ks_reportportal_multi branch 6 times, most recently from 4e9049f to aaec620 Compare August 7, 2024 11:14
@psss psss added the ci | full test Pull request is ready for the full test execution label Aug 7, 2024
@psss
Copy link
Collaborator

psss commented Aug 7, 2024

/packit test

@psss psss added step | report Stuff related to the report step plugin | reportportal The reportportal report plugin labels Aug 7, 2024
@kkaarreell kkaarreell mentioned this pull request Aug 7, 2024
Closed
@psss psss added this to the 1.36 milestone Aug 8, 2024
@psss psss added the priority | must high priority, must be included in the next release label Aug 13, 2024
@psss psss linked an issue Aug 13, 2024 that may be closed by this pull request
tmt reportportal plugin not reporting multi-host test results properly #3129
Closed
@thrix
Copy link
Collaborator

thrix commented Aug 13, 2024

@kkaarreell seems this change is not tested at all, I am wondering, if you think there could be way to test this?

@kkaarreell
Copy link
Collaborator Author

kkaarreell commented Aug 13, 2024 •
edited
Loading

@kkaarreell seems this change is not tested at all, I am wondering, if you think there could be way to test this?

So what I did was taking a simple single-host plan and add few other hosts in the provision step to make it run on multiple hosts. And then running tmt with all the parameters/variables required for reportportal reporting.

Example:

$ cat plan.fmf 
summary: fake multihost test
provision:
  - role: roleA
    how: container
    image: fedora:latest
  - name: hostB
    how: container
    image: fedora:latest
  - name: hostC
    role: roleC
    how: container
    image: fedora:latest
discover:
  - how: fmf
    url: https://github.com/RedHat-SP-Security/keylime-tests.git
    ref: main
    test:
      - /setup/apply_workarounds
      - /setup/inject_SELinux_AVC_check
execute:
  how: tmt

$ tmt run --all
$ tmt run --last report -h reportportal --url $RP_URL --launch multi-test --project $LOGIN_personal --token $RP_TOKEN

@happz happz added the status | ready for merge The only missing piece is to do the rebase the current 'main' and let the CI finish. label Aug 22, 2024
@happz
Copy link
Collaborator

happz commented Aug 27, 2024

Unrelated failures, merging.

@happz happz merged commit 7c1effa into main Aug 27, 2024
20 of 22 checks passed
@happz happz deleted the ks_reportportal_multi branch August 27, 2024 07:30
@psss psss changed the title reportportal: upload all test case results Upload all test case results in the reportportal plugin Oct 11, 2024
The-Mule pushed a commit to The-Mule/tmt that referenced this pull request Oct 14, 2024
@kkaarreell @The-Mule
reportportal: upload all test case results (teemtee#3135)
1c758b0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@KwisatzHaderach KwisatzHaderach KwisatzHaderach approved these changes

@therazix therazix therazix approved these changes

@4N0body5 4N0body5 4N0body5 approved these changes

@psss psss Awaiting requested review from psss psss is a code owner

@lukaszachy lukaszachy Awaiting requested review from lukaszachy lukaszachy is a code owner

@happz happz Awaiting requested review from happz happz is a code owner

@thrix thrix Awaiting requested review from thrix thrix is a code owner

@janhavlin janhavlin Awaiting requested review from janhavlin janhavlin is a code owner

Assignees
No one assigned
Labels
ci | full test Pull request is ready for the full test execution plugin | reportportal The reportportal report plugin priority | must high priority, must be included in the next release status | ready for merge The only missing piece is to do the rebase the current 'main' and let the CI finish. step | report Stuff related to the report step
Projects
None yet
Milestone
1.36
Development

Successfully merging this pull request may close these issues.

tmt reportportal plugin not reporting multi-host test results properly
7 participants
@kkaarreell @psss @thrix @happz @KwisatzHaderach @therazix @4N0body5