Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nightwatch test-observability integration phase-1 #5

Merged
merged 42 commits into from
May 25, 2023
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
286fc9b
added build start event
Mar 29, 2023
32369f3
stop build event
Mar 30, 2023
fdaaebd
populated test and hook events for test observability
Apr 11, 2023
a7dcf06
added screenshot for failure case
Apr 11, 2023
67a7a00
changes for screenshots and integration object in teststart
Apr 12, 2023
ab028c5
refactored code by removing comments and unncessary code
Apr 12, 2023
f2f6c0d
given preference to env variables
Apr 12, 2023
6133a6a
added argument support for test observability
Apr 12, 2023
f982799
version control file path bug fixed
Apr 13, 2023
34f2c45
observability rerun changes and code refactoring
Apr 18, 2023
a8e58bd
removed before each
Apr 18, 2023
7416ae7
removed vc file path unwanted function
Apr 18, 2023
0240650
sending screenshot in base64 format
Apr 18, 2023
130ffe5
failure stacktrace format changed
Apr 18, 2023
eb8313e
clear interval if empty queue
Apr 18, 2023
0978995
fixed null ptr exception
Apr 18, 2023
0a03b14
check for file read and unlink
Apr 18, 2023
8ddf1d9
added times for benchmarking
Apr 19, 2023
9cd7a0d
removed console logs used for benchmarking
Apr 20, 2023
aff83d3
changes for test level tags
Apr 24, 2023
c39e842
changes for displaying http logs
Apr 24, 2023
5e156d8
made reporter function async
Apr 25, 2023
40eb90b
await for shutdown que handler
Apr 25, 2023
dbfc5fc
bug fixes for custom reporter, rerun build, file name
Apr 27, 2023
7d188e4
functionality added for skipped tests
May 2, 2023
564f67d
skipped tests and crash report added
May 5, 2023
5fc6c31
changes for review comments
May 8, 2023
129d6ff
retry test data fix
May 8, 2023
cfb7ac3
logging changes
May 8, 2023
79bc9ee
removed typo
May 9, 2023
c48343e
changes for review comments
May 10, 2023
bfc7764
added fspromises
May 17, 2023
0c701c4
made request handler singleton
May 17, 2023
173d6ea
added observability sync script
May 18, 2023
9cff9c0
added pref for env vars
May 19, 2023
4783713
null check added
May 22, 2023
eefc057
review comments done
May 23, 2023
4cedf20
added fixes for review comments
May 23, 2023
53c3da6
linting changes and fixes for new review comments
May 24, 2023
5347114
handling of error cases in 401 and 403
May 24, 2023
5148bff
use ES2020
swrdfish May 24, 2023
d0e71d4
Merge branch 'OB_189' of https://github.com/grkpranaykumar/nightwatch…
swrdfish May 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions nightwatch/globals.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,58 @@
const LocalTunnel = require('../src/local-tunnel');
const TestObservability = require('../src/testObservability');
const {CUSTOM_REPORTER_CALLBACK_TIMEOUT} = require('../src/utils/constants');
const CrashReporter = require('../src/utils/crashReporter');
const helper = require('../src/utils/helper');
const Logger = require('../src/utils/logger');

const localTunnel = new LocalTunnel();
const testObservability = new TestObservability();

const nightwatchRerun = process.env.NIGHTWATCH_RERUN_FAILED;
const nightwatchRerunFile = process.env.NIGHTWATCH_RERUN_REPORT_FILE;

module.exports = {

reporter: function(results, done) {
swrdfish marked this conversation as resolved.
Show resolved Hide resolved
if (helper.isTestObservabilitySession()) {
const promises = [];
try {
const modulesWithEnv = results['modulesWithEnv'];
for (const testSetting in modulesWithEnv) {
for (const testFile in modulesWithEnv[testSetting]) {
for (const completedSection in modulesWithEnv[testSetting][testFile].completed) {
if (modulesWithEnv[testSetting][testFile].completed[completedSection]) {
delete modulesWithEnv[testSetting][testFile].completed[completedSection].steps;
delete modulesWithEnv[testSetting][testFile].completed[completedSection].testcases;
}
}
promises.push(testObservability.processTestReportFile(JSON.parse(JSON.stringify(modulesWithEnv[testSetting][testFile]))));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSON.parse and JSON.stringify is unnecessary

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep copy is necessary because the git modules are changing the values in the object while parsing.

}
}

Promise.all(promises).then(() => {
done();
}).catch((err) =>{
Logger.error(`Something went wrong in processing report file for test observability - ${err.message} with stacktrace ${err.stack}`);
CrashReporter.uploadCrashReport(err.message, err.stack);
done();
});

return;
} catch (error) {
CrashReporter.uploadCrashReport(error.message, error.stack);
Logger.error(`Something went wrong in processing report file for test observability - ${error.message} with stacktrace ${error.stack}`);
}
}
done(results);
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this whole code can be refactored like this

reporter: async function(results, done) {
  if (!helper.isTestObservabilitySession()) {
    done(results);
    return;
  }

  try {
    const modulesWithEnv = results['modulesWithEnv'];
    const promises = [];

    for (const testSetting in modulesWithEnv) {
      for (const testFile in modulesWithEnv[testSetting]) {
      const completedSections = modulesWithEnv[testSetting][testFile].completed;

        for (const completedSection in completedSections) {
          if (completedSections[completedSection]) {
            delete completedSections[completedSection].steps;
            delete completedSections[completedSection].testcases;
          }
        }

        // Maybe create a helper method to do `.parse` and `.stringify`
        promises.push(testObservability.processTestReportFile(JSON.parse(JSON.stringify(modulesWithEnv[testSetting][testFile]))));
      }
    }

    await Promise.all(promises);
    done();
  } catch (error) {
    Logger.error(`Something went wrong in processing report file for test observability - ${error.message} with stacktrace ${error.stack}`);
    CrashReporter.uploadCrashReport(error.message, error.stack);
  }
}


onEvent({eventName, hook_type, ...args}) {
if (browser && eventName === 'TestRunStarted') {
browser.execute(`browserstack_executor: {"action": "annotate", "arguments": {"type":"Annotation","data":"ObservabilitySync:${Date.now()}","level": "debug"}}`);
}
},

async before(settings) {
localTunnel.configure(settings);
await localTunnel.start();
Expand All @@ -22,10 +71,42 @@ module.exports = {
settings.desiredCapabilities['bstack:options'].localIdentifier = localTunnel._localOpts.localIdentifier;
}
}

try {
testObservability.configure(settings);
if (helper.isTestObservabilitySession()) {
settings.globals['customReporterCallbackTimeout'] = CUSTOM_REPORTER_CALLBACK_TIMEOUT;
if (testObservability._user && testObservability._key) {
await testObservability.launchTestSession();
}
if (process.env.BROWSERSTACK_RERUN === 'true' && process.env.BROWSERSTACK_RERUN_TESTS && process.env.BROWSERSTACK_RERUN_TESTS!=='null') {
const specs = process.env.BROWSERSTACK_RERUN_TESTS.split(',');
await helper.handleNightwatchRerun(specs);
}
}
} catch (error) {
Logger.error(`Could not configure or launch test observability - ${error}`);
}

},

async after() {
localTunnel.stop();
if (helper.isTestObservabilitySession()) {
try {
await testObservability.stopBuildUpstream();
if (process.env.BS_TESTOPS_BUILD_HASHED_ID) {
Logger.info(`\nVisit https://observability.browserstack.com/builds/${process.env.BS_TESTOPS_BUILD_HASHED_ID} to view build report, insights, and many more debugging information all at one place!\n`);
}
} catch (error) {
Logger.error(`Something went wrong in stopping build session for test observability - ${error}`);
}
process.env.NIGHTWATCH_RERUN_FAILED = nightwatchRerun;
process.env.NIGHTWATCH_RERUN_REPORT_FILE = nightwatchRerunFile;
if (process.env.BROWSERSTACK_RERUN === 'true' && process.env.BROWSERSTACK_RERUN_TESTS) {
await helper.deleteRerunFile();
}
}
},

beforeChildProcess(settings) {
Expand Down
Loading