Fix Github PR comment formatting #56078

brianseeders · 2020-01-27T22:36:05Z

PR comments with multiple failures are formatted badly, and errors from certain test runners aren't outputting error information because they use fields in the junit report differently than the report I developed with.

brianseeders · 2020-01-27T22:59:47Z

(Post-fix from a local test:)

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: null

Test Failures

Server Mocha Tests.x-pack/legacy/plugins/monitoring/server/es_client/__tests__/instantiate_client·js.Instantiate Client Logging logs that the config was sourced from the production options

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

TypeError: Cannot read property 'monitoring' of undefined
    at getMockServerFromConnectionUrl (x-pack/legacy/plugins/monitoring/server/es_client/__tests__/instantiate_client.js:30:39)
    at Context.it (x-pack/legacy/plugins/monitoring/server/es_client/__tests__/instantiate_client.js:51:22)

Server Mocha Tests.x-pack/legacy/plugins/monitoring/server/es_client/__tests__/instantiate_client·js.Instantiate Client Logging logs that the config was sourced from the monitoring options

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

TypeError: Cannot read property 'monitoring' of undefined
    at getMockServerFromConnectionUrl (x-pack/legacy/plugins/monitoring/server/es_client/__tests__/instantiate_client.js:30:39)
    at Context.it (x-pack/legacy/plugins/monitoring/server/es_client/__tests__/instantiate_client.js:62:22)

Server Mocha Tests.x-pack/legacy/plugins/monitoring/server/es_client/__tests__/instantiate_client·js.Instantiate Client Custom Headers Configuration Does not add xpack.monitoring.elasticsearch.customHeaders if connected to production cluster

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

TypeError: Cannot read property 'monitoring' of undefined
    at getMockServerFromConnectionUrl (x-pack/legacy/plugins/monitoring/server/es_client/__tests__/instantiate_client.js:30:39)
    at Context.it (x-pack/legacy/plugins/monitoring/server/es_client/__tests__/instantiate_client.js:74:22)

and 6 more failures, only showing the first 3.

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

brianseeders · 2020-01-27T23:06:12Z

vars/githubPr.groovy

-${failure.stdOut}
-```
-</details>
+    if (failure.stdOut) {


Looks like different junit reports use these fields differently, so I need to check for all of them

Will probably try to re-factor this stuff this week, the formatting/etc is really hard to follow because of needing to make everything look right in the PR comment

elasticmachine · 2020-01-27T23:11:56Z

Pinging @elastic/kibana-operations (Team:Operations)

tylersmalley

LGTM - will also be able to verify once the build fails as expected.

brianseeders · 2020-01-27T23:47:57Z

From CI, for posterity:

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: cd06d65

Test Failures

Kibana Pipeline / x-pack-intake-agent / X-Pack Jest Tests.x-pack/plugins/canvas/server/collectors.custom_element_collector.handleResponse invalid responses fails

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

Error: expect(received).toEqual(expected) // deep equality

Expected: true
Received: false
    at Object.it.only (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/x-pack/plugins/canvas/server/collectors/custom_element_collector.test.ts:23:21)
    at Object.asyncJestTest (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:102:37)
    at resolve (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/queueRunner.js:43:12)
    at new Promise (<anonymous>)
    at mapper (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/queueRunner.js:26:19)
    at promise.then (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/queueRunner.js:73:41)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Kibana Pipeline / kibana-oss-agent / Chrome UI Functional Tests.test/functional/apps/console/_console·ts.console app console app fails

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: console app
[00:00:00]           └-> "before all" hook
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: console app
[00:00:00]             └-> "before all" hook
[00:00:00]             └-> "before all" hook
[00:00:00]               │ debg navigateTo console
[00:00:00]               │ debg navigating to console url: http://localhost:6111/app/kibana#/dev_tools/console
[00:00:00]               │ debg Navigate to: http://localhost:6111/app/kibana#/dev_tools/console
[00:00:00]               │ debg ... sleep(700) start
[00:00:00]               │ debg browser[INFO] http://localhost:6111/app/kibana?_t=1580167277480#/dev_tools/console 350 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:00]               │
[00:00:00]               │ debg browser[INFO] http://localhost:6111/bundles/app/kibana/bootstrap.js 8:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:00]               │ debg ... sleep(700) end
[00:00:00]               │ debg returned from get, calling refresh
[00:00:01]               │ debg browser[INFO] http://localhost:6111/app/kibana?_t=1580167277480#/dev_tools/console 350 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:01]               │
[00:00:01]               │ debg browser[INFO] http://localhost:6111/bundles/app/kibana/bootstrap.js 8:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:01]               │ debg currentUrl = http://localhost:6111/app/kibana#/dev_tools/console
[00:00:01]               │          appUrl = http://localhost:6111/app/kibana#/dev_tools/console
[00:00:01]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:03]               │ debg browser[INFO] http://localhost:6111/built_assets/dlls/vendors_0.bundle.dll.js 109:138197 "INFO: 2020-01-27T23:21:20Z
[00:00:03]               │        Adding connection to http://localhost:6111/elasticsearch
[00:00:03]               │
[00:00:03]               │      "
[00:00:04]               │ debg TestSubjects.find(kibanaChrome)
[00:00:04]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=10000
[00:00:04]               │ debg ... sleep(501) start
[00:00:04]               │ debg ... sleep(501) end
[00:00:04]               │ debg in navigateTo url = http://localhost:6111/app/kibana#/dev_tools/console
[00:00:04]               │ debg TestSubjects.exists(statusPageContainer)
[00:00:04]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:00:07]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:00:07]             └-> fails
[00:00:07]               └-> "before each" hook: global before each
[00:00:07]               │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/console app console app fails.png"
[00:00:07]               │ info Current URL is: http://localhost:6111/app/kibana#/dev_tools/console
[00:00:07]               │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/console app console app fails.html
[00:00:07]               │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/console app console app fails.png"
[00:00:07]               │ info Current URL is: http://localhost:6111/app/kibana#/dev_tools/console
[00:00:07]               │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/console app console app fails.html
[00:00:07]               └- ✖ fail: "console app console app fails"
[00:00:07]               │

Stack Trace

{ Error: expected true to sort of equal false
    at Assertion.assert (packages/kbn-expect/expect.js:100:11)
    at Assertion.eql (packages/kbn-expect/expect.js:244:8)
    at Context.it.only (test/functional/apps/console/_console.ts:50:23)
    at Object.apply (packages/kbn-test/target/functional_test_runner/lib/mocha/wrap_function.js:98:30) actual: 'true', expected: 'false', showDiff: true }

Kibana Pipeline / kibana-oss-agent / Chrome UI Functional Tests.test/functional/apps/console/_console·ts.console app console app fails

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: console app
[00:00:00]           └-> "before all" hook
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: console app
[00:00:00]             └-> "before all" hook
[00:00:00]             └-> "before all" hook
[00:00:00]               │ debg navigateTo console
[00:00:00]               │ debg navigating to console url: http://localhost:6111/app/kibana#/dev_tools/console
[00:00:00]               │ debg Navigate to: http://localhost:6111/app/kibana#/dev_tools/console
[00:00:00]               │ debg ... sleep(700) start
[00:00:00]               │ debg browser[INFO] http://localhost:6111/app/kibana?_t=1580167197662#/dev_tools/console 350 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:00]               │
[00:00:00]               │ debg browser[INFO] http://localhost:6111/bundles/app/kibana/bootstrap.js 8:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:01]               │ debg ... sleep(700) end
[00:00:01]               │ debg returned from get, calling refresh
[00:00:01]               │ debg browser[INFO] http://localhost:6111/app/kibana?_t=1580167197662#/dev_tools/console 350 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:01]               │
[00:00:01]               │ debg browser[INFO] http://localhost:6111/bundles/app/kibana/bootstrap.js 8:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:01]               │ debg currentUrl = http://localhost:6111/app/kibana#/dev_tools/console
[00:00:01]               │          appUrl = http://localhost:6111/app/kibana#/dev_tools/console
[00:00:01]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:03]               │ debg browser[INFO] http://localhost:6111/built_assets/dlls/vendors_0.bundle.dll.js 109:138197 "INFO: 2020-01-27T23:20:00Z
[00:00:03]               │        Adding connection to http://localhost:6111/elasticsearch
[00:00:03]               │
[00:00:03]               │      "
[00:00:04]               │ debg TestSubjects.find(kibanaChrome)
[00:00:04]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=10000
[00:00:04]               │ debg ... sleep(501) start
[00:00:04]               │ debg ... sleep(501) end
[00:00:04]               │ debg in navigateTo url = http://localhost:6111/app/kibana#/dev_tools/console
[00:00:04]               │ debg TestSubjects.exists(statusPageContainer)
[00:00:04]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:00:07]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:00:07]             └-> fails
[00:00:07]               └-> "before each" hook: global before each
[00:00:07]               │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/console app console app fails.png"
[00:00:07]               │ info Current URL is: http://localhost:6111/app/kibana#/dev_tools/console
[00:00:07]               │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/console app console app fails.html
[00:00:07]               │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/console app console app fails.png"
[00:00:08]               │ info Current URL is: http://localhost:6111/app/kibana#/dev_tools/console
[00:00:08]               │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/console app console app fails.html
[00:00:08]               └- ✖ fail: "console app console app fails"
[00:00:08]               │

Stack Trace

{ Error: expected true to sort of equal false
    at Assertion.assert (packages/kbn-expect/expect.js:100:11)
    at Assertion.eql (packages/kbn-expect/expect.js:244:8)
    at Context.it.only (test/functional/apps/console/_console.ts:50:23)
    at Object.apply (packages/kbn-test/target/functional_test_runner/lib/mocha/wrap_function.js:98:30) actual: 'true', expected: 'false', showDiff: true }

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

This reverts commit 692f4fc.

kibanamachine · 2020-01-28T01:07:21Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 9ee8ef1

History

💔 Build #22883 failed cd06d65

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…ve-out-legacy * 'master' of github.com:elastic/kibana: (187 commits) [ML] Reseting categorization validation if category field is cleared (elastic#56029) [SIEM] Fields browser readable (elastic#56000) [docs] Remove unused callout (elastic#56032) Refactor saved object management registry usage (elastic#54155) [SIEM][Detection Engine] critical blocker, updates the pre-packaged rules, removes dead ones, adds license file (elastic#56090) Fix failing snapshot artifact tests when using env var (elastic#56063) Fix Github PR comment formatting (elastic#56078) [Maps] fix join metric field selection bugs (elastic#56044) Create a new menu for observability links (elastic#54847) [SIEM] [Detection Engine] Fixes histogram intervals (elastic#55969) make test less flaky by retrying if list is re-rendered (elastic#55949) Remove matrix build support (elastic#54202) Add animation to service map layout (elastic#56042) [Canvas] Remove Angular and unnecessary reporting config from Canvas (elastic#54050) [Uptime] Simplify snapshot max to Infinity (elastic#55931) [Uptime] Reintroduce a column for url (elastic#55451) Cleanup action task params objects after successful execution (elastic#55227) [CI] Retry flaky tests (elastic#53961) Expose NP FieldFormats service to server side (elastic#55419) [Endpoint] EMT-65: make endpoint data types common, restructure (elastic#54772) ... # Conflicts: # src/legacy/core_plugins/console/public/np_ready/application/components/split_panel/__snapshots__/split_panel.test.tsx.snap # src/legacy/core_plugins/console/public/np_ready/application/components/split_panel/containers/panel.tsx # src/legacy/core_plugins/console/public/np_ready/application/components/split_panel/context.tsx # src/legacy/core_plugins/console/public/np_ready/application/components/split_panel/index.ts # src/legacy/core_plugins/console/public/np_ready/application/components/split_panel/split_panel.test.tsx # src/legacy/ui/public/vis/editors/default/default_editor.tsx # src/plugins/console/public/application/components/split_panel/__snapshots__/split_panel.test.tsx.snap # src/plugins/console/public/application/components/split_panel/components/resizer.tsx # src/plugins/console/public/application/components/split_panel/containers/panel.tsx # src/plugins/console/public/application/components/split_panel/containers/panel_container.tsx # src/plugins/console/public/application/components/split_panel/context.tsx # src/plugins/console/public/application/components/split_panel/index.ts # src/plugins/console/public/application/components/split_panel/registry.ts # src/plugins/console/public/application/components/split_panel/split_panel.test.tsx # src/plugins/kibana_react/public/split_panel/__snapshots__/split_panel.test.tsx.snap # src/plugins/kibana_react/public/split_panel/containers/panel.tsx # src/plugins/kibana_react/public/split_panel/context.tsx # src/plugins/kibana_react/public/split_panel/index.ts # src/plugins/kibana_react/public/split_panel/split_panel.test.tsx

* master: (77 commits) [ML] Reseting categorization validation if category field is cleared (elastic#56029) [SIEM] Fields browser readable (elastic#56000) [docs] Remove unused callout (elastic#56032) Refactor saved object management registry usage (elastic#54155) [SIEM][Detection Engine] critical blocker, updates the pre-packaged rules, removes dead ones, adds license file (elastic#56090) Fix failing snapshot artifact tests when using env var (elastic#56063) Fix Github PR comment formatting (elastic#56078) [Maps] fix join metric field selection bugs (elastic#56044) Create a new menu for observability links (elastic#54847) [SIEM] [Detection Engine] Fixes histogram intervals (elastic#55969) make test less flaky by retrying if list is re-rendered (elastic#55949) Remove matrix build support (elastic#54202) Add animation to service map layout (elastic#56042) [Canvas] Remove Angular and unnecessary reporting config from Canvas (elastic#54050) [Uptime] Simplify snapshot max to Infinity (elastic#55931) [Uptime] Reintroduce a column for url (elastic#55451) Cleanup action task params objects after successful execution (elastic#55227) [CI] Retry flaky tests (elastic#53961) Expose NP FieldFormats service to server side (elastic#55419) [Endpoint] EMT-65: make endpoint data types common, restructure (elastic#54772) ...

Fix several issues with error reporting in github pr comments

bea08e9

brianseeders added Feature:CI Continuous integration release_note:skip Skip the PR/issue when compiling release notes v6.8.6 v7.6.1 v7.7.0 v8.0.0 labels Jan 27, 2020

Add some failures for testing

692f4fc

brianseeders force-pushed the fix/github-pr-comments branch from 25b4bd1 to 692f4fc Compare January 27, 2020 22:37

A couple more small fixes

cd06d65

brianseeders commented Jan 27, 2020

View reviewed changes

brianseeders changed the title ~~Fix Github PR comments~~ Fix Github PR comment formatting Jan 27, 2020

brianseeders marked this pull request as ready for review January 27, 2020 23:11

brianseeders added the Team:Operations Team label for Operations Team label Jan 27, 2020

tylersmalley approved these changes Jan 27, 2020

View reviewed changes

Revert "Add some failures for testing"

9ee8ef1

This reverts commit 692f4fc.

brianseeders merged commit ba151fe into elastic:master Jan 28, 2020

brianseeders deleted the fix/github-pr-comments branch January 28, 2020 01:45

brianseeders mentioned this pull request Jan 28, 2020

Fix Github PR comment formatting (#56078) #56091

Merged

brianseeders added a commit to brianseeders/kibana that referenced this pull request Jan 28, 2020

Fix Github PR comment formatting (elastic#56078)

e563eed

brianseeders mentioned this pull request Jan 28, 2020

Fix Github PR comment formatting (#56078) #56092

Merged

brianseeders added a commit to brianseeders/kibana that referenced this pull request Jan 28, 2020

Fix Github PR comment formatting (elastic#56078)

2defae0

brianseeders added a commit to brianseeders/kibana that referenced this pull request Jan 28, 2020

Fix Github PR comment formatting (elastic#56078)

089eb41

brianseeders mentioned this pull request Jan 28, 2020

Fix Github PR comment formatting (#56078) #56093

Merged

brianseeders added a commit that referenced this pull request Jan 28, 2020

Fix Github PR comment formatting (#56078) (#56091)

82509cb

brianseeders added a commit that referenced this pull request Jan 28, 2020

Fix Github PR comment formatting (#56078) (#56092)

e992c40

brianseeders added a commit that referenced this pull request Jan 28, 2020

Fix Github PR comment formatting (#56078) (#56093)

bc1058b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Github PR comment formatting #56078

Fix Github PR comment formatting #56078

brianseeders commented Jan 27, 2020 •

edited

Loading

brianseeders commented Jan 27, 2020 •

edited

Loading

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

brianseeders Jan 27, 2020

brianseeders Jan 27, 2020

elasticmachine commented Jan 27, 2020

tylersmalley left a comment

brianseeders commented Jan 27, 2020

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

kibanamachine commented Jan 28, 2020

Fix Github PR comment formatting #56078

Fix Github PR comment formatting #56078

Conversation

brianseeders commented Jan 27, 2020 • edited Loading

brianseeders commented Jan 27, 2020 • edited Loading

💔 Build Failed

Test Failures

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

brianseeders Jan 27, 2020

Choose a reason for hiding this comment

brianseeders Jan 27, 2020

Choose a reason for hiding this comment

elasticmachine commented Jan 27, 2020

tylersmalley left a comment

Choose a reason for hiding this comment

brianseeders commented Jan 27, 2020

💔 Build Failed

Test Failures

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

kibanamachine commented Jan 28, 2020

💚 Build Succeeded

History

brianseeders commented Jan 27, 2020 •

edited

Loading

brianseeders commented Jan 27, 2020 •

edited

Loading