Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Option to print output of passed tests in Reporters #583

Closed
manosnoam opened this issue Jun 25, 2019 · 1 comment
Closed

Option to print output of passed tests in Reporters #583

manosnoam opened this issue Jun 25, 2019 · 1 comment

Comments

@manosnoam
Copy link
Contributor

manosnoam commented Jun 25, 2019 •
edited
Loading

For those who use reporters, such as Teamcity or Junit, for their tests frameworks, it is commonly required to display steps output of passed tests (not only for failed tests, which is the current behavior).

I've implemented an option to use -ginkgo.reportPassed for that purpose,
and the report output, for example in Junit XML, will look as follow:

<?xml version="1.0" encoding="UTF-8"?>
  <testsuite name="E2E suite" tests="1" failures="0" errors="0" time="11.396">
      <testcase name="list existing nodes on the cluster" classname="E2E suite" time="0">
          <skipped></skipped>
      </testcase>
      <testcase name="Pod to Service tests" classname="E2E suite" time="11.395812505">
          <passed>STEP: Creating kubernetes clients
          STEP: Building namespace api objects, basename dataplane-p2s-nd1
          STEP: Creating a namespace e2e-tests-dataplane-p2s-nd1-q5phj to execute the test 
          STEP: Creating a listener pod in cluster B, which will wait for a handshake over TCP
          STEP: Pointing a service ClusterIP to the listerner pod in cluster; Jun 25 15:23:54.494: INFO: Service for listener pod has ClusterIP: 100.96.40.208
          STEP: Creating a connector pod in cluster B, which will attempt the specific UUID handshake over TCP;</passed>
      </testcase>
      <testcase name="create a pod using the provided client" classname="E2E suite" time="0">
      <failure type="Failure">/home/nmanos/go/src/github.com/rancher/submariner/test/e2e/dataplane/tcp_pod_to_service.go:14;Expected;    ;string: 100.96.190.228 (100.96.190.228:1234) open;    ;to contain substring;    ;string: 7277d684-2c95-40fd-9bb5-5bf791a15ac7;/home/nmanos/go/src/github.com/rancher/submariner/test/e2e/dataplane/tcp_pod_to_service.go:46</failure>
      <system-out>STEP: Creating kubernetes clients
          STEP: Building namespace api objects, basename dataplane-p2s-nd
          STEP: Creating a namespace e2e-tests-dataplane-p2s-nd-ttds8 to execute the test</system-out>
      </testcase>
  </testsuite>
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 25, 2019
@manosnoam
Option to print output of passed tests in Junit XML result
800b12d 
This is PR for onsi#583

A new CLI option: -ginkgo.junitPassed.
It will add output for each passed test into the JUnit XML, under
<testcase> <passed>. (The default is only for failed tests output).
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 25, 2019
@manosnoam
Option to print output of passed tests in Junit XML result
d2cf5f6 
This is PR for onsi#583

A new CLI option: -ginkgo.junitPassed.
It will add output for each passed test into the JUnit XML, under
<testcase> <passed>. (The default is only for failed tests output).
@manosnoam manosnoam mentioned this issue Jun 25, 2019
Merged
@manosnoam manosnoam changed the title Option to print steps for passed tests in Junit result Option to print output of passed tests in Junit XML report Jun 25, 2019
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 25, 2019
@manosnoam
Option to print output of passed tests in Junit XML result
26f0456 
This is PR for onsi#583

A new CLI option: -ginkgo.junitPassed.
It will add output for each passed test into the JUnit XML, under
<testcase> <passed>. (The default is only for failed tests output).
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 27, 2019
@manosnoam
Option to print output into report, when tests have passed
ab68009 
This is PR for onsi#583

A new CLI option: -ginkgo.reportPassed  :
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 27, 2019
@manosnoam
Option to print output into report, when tests have passed
814b594 
This is PR for onsi#583

A new CLI option: -ginkgo.reportPassed  :
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 27, 2019
@manosnoam
Option to print output into report, when tests have passed
6be40d9 
This is PR for onsi#583

A new CLI option: -ginkgo.reportPassed  :
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 27, 2019
@manosnoam
Option to print output into report, when tests have passed
9378a22 
This is PR for onsi#583

A new CLI option: -ginkgo.reportPassed  :
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 27, 2019
@manosnoam
Option to print output into report, when tests have passed
07137e3 
This is PR for onsi#583

A new CLI option: -ginkgo.reportPassed  :
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 30, 2019
@manosnoam
Option to print output into report, when tests have passed
29c6c8e 
This is PR for onsi#583

A new CLI option: -ginkgo.reportPassed  :
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jun 30, 2019
@manosnoam
Option to print output into report, when tests have passed
230f436 
This is PR for onsi#583

A new CLI option: -ginkgo.reportPassed  :
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.
@manosnoam
Copy link
Contributor Author

Hi @williammartin,
I've implemented your suggested requirements in #586,
including Tests for:

  • internal/specrunner/spec_runner_test.go
  • reporters/default_reporter_test.go
  • reporters/junit_reporter_test.go
  • reporters/teamcity_reporter_test.go

Travis-CI is now passing.

Can you please review ?

@manosnoam manosnoam changed the title Option to print output of passed tests in Junit XML report Option to print output of passed tests in Reporters Jun 30, 2019
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jul 1, 2019
@manosnoam
Option to print output into report, when tests have passed
f735d68 
This is PR for onsi#583

A new CLI option: -ginkgo.reportPassed  :
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jul 7, 2019
@manosnoam
Option to print output into report, when tests have passed
8108c70 
A new CLI option: -ginkgo.reportPassed
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.

[Fixes onsi#583]
manosnoam added a commit to manosnoam/ginkgo that referenced this issue Jul 7, 2019
@manosnoam
Option to print output into report, when tests have passed
7b45017 
A new CLI option: -ginkgo.reportPassed
It will print output for each passed test in the generated report,
including JUnit, Teamcity, and Default reporters.

For example, in JUnit (XML), the test output will be added under:
`<testcase> <passed>`

The default behavior (without this option), prints test output
only if the test case (spec) has failed.

[Fixes onsi#583]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@manosnoam