[Rename] kbn-test in packages directory to osd-test/ #37 [4] (#56)

* [Rename] kbn-test in packages directory to osd-test/functional_test_runner

Signed-off-by: Mihir Soni <mihirsoni.123@gmail.com>

* [Rename] kbn-test in packages directory to osd-test/functional_tests

Signed-off-by: Mihir Soni <mihirsoni.123@gmail.com>

* [Rename] kbn-test/src/failed_tests_reporter in packages directory to osd-test/failed_tests_reporter

Signed-off-by: Mihir Soni <mihirsoni.123@gmail.com>

* [Rename] kbn-test/src/ in packages directory to osd-test/src

Signed-off-by: Mihir Soni <mihirsoni.123@gmail.com>

packages/osd-test/README.md

@@ -0,0 +1,39 @@
+OpenSearch Dashboards Testing Library
+======================
+
+The @osd/test package provides ways to run tests. Currently only functional testing is provided by this library, with unit and other testing possibly added here.
+
+Functional Testing
+-------------------
+
+### Dependencies
+
+Functional testing methods exist in the `src/functional_tests` directory. They depend on the Functional Test Runner, which is found in [`{OPENSEARCH_DASHBOARDS_ROOT}/src/functional_test_runner`](../../src/functional_test_runner). Ideally libraries provided by opensearch-dashboards packages such as this one should not depend on opensearch-dashboards source code that lives in [`{OPENSEARCH_DASHBOARDS_ROOT}/src`](../../src). The goal is to start pulling test and development utilities out into packages so they can be used across OpenSearch Dashboards and plugins. Accordingly the Functional Test Runner itself will be pulled out into a package (or part of a package), and this package's dependence on it will not be an issue.
+
+### Exposed methods
+
+#### runTests(configPaths: Array<string>)
+For each config file specified in configPaths, starts OpenSearch and OpenSearch Dashboards once, runs tests specified in that config file, and shuts down OpenSearch and OpenSearch Dashboards once completed. (Repeats for every config file.)
+
+`configPaths`: array of strings, each an absolute path to a config file that looks like [this](../../test/functional/config.js), following the config schema specified [here](../../src/functional_test_runner/lib/config/schema.js).
+
+Internally the method that starts OpenSearch comes from [osd-opensearch](../../packages/osd-opensearch).
+
+#### startServers(configPath: string)
+Starts OpenSearch and OpenSearch Dashboards servers given a specified config.
+
+`configPath`: absolute path to a config file that looks like [this](../../test/functional/config.js), following the config schema specified [here](../../src/functional_test_runner/lib/config/schema.js).
+
+Allows users to start another process to run just the tests while keeping the servers running with this method. Start servers _and_ run tests using the same config file ([see how](../../scripts/README.md)).
+
+## Rationale
+
+### Single config per setup
+
+We think it makes sense to specify the tests to run along with the particular server configuration for OpenSearch and OpenSearch Dashboards servers, because the tests expect a particular configuration. For example, saml api integration tests expect certain xml files to exist in OpenSearch's config directory, and certain saml specific options to be passed in via the command line (or alternatively via the `.yml` config file) to both OpenSearch and OpenSearch Dashboards. It makes sense to keep all these config options together with the list of test files.
+
+### Multiple configs running in succession
+
+We also think it makes sense to have a test runner intelligently (but simply) start servers, run tests, tear down servers, and repeat for each config, uninterrupted. There's nothing special about each kind of config that specifies running some set of functional tests against some kind of OpenSearch/OpenSearch Dashboards servers. There doesn't need to be a separate job to run each kind of setup/test/teardown. These can all be orchestrated sequentially via the current `runTests` implementation. This is how we envision tests to run on CI.
+
+This inherently means that grouping test files in configs matters, such that a group of test files that depends on a particular server config appears together in that config's `testFiles` list. Given how quickly and easily we can start servers using [@osd/opensearch](../../packages/osd-opensearch), it should not impact performance to logically group tests by domain even if multiple groups of tests share the same server config. We can think about how to group test files together across domains when that time comes.

packages/kbn-test/babel.config.js → packages/osd-test/babel.config.js

@@ -18,6 +18,6 @@
  */
 
 module.exports = {
-  presets: ['@kbn/babel-preset/node_preset'],
+  presets: ['@osd/babel-preset/node_preset'],
   ignore: ['**/*.test.js'],
 };

packages/kbn-test/package.json → packages/osd-test/package.json

@@ -1,22 +1,22 @@
 {
-  "name": "@kbn/test",
+  "name": "@osd/test",
   "version": "1.0.0",
   "private": true,
   "license": "Apache-2.0",
   "main": "./target/index.js",
   "scripts": {
     "build": "babel src --out-dir target --delete-dir-on-start --extensions .ts,.js,.tsx --ignore *.test.js,**/__tests__/** --source-maps=inline",
-    "kbn:bootstrap": "yarn build",
-    "kbn:watch": "yarn build --watch"
+    "osd:bootstrap": "yarn build",
+    "osd:watch": "yarn build --watch"
   },
-  "kibana": {
+  "opensearchDashboards": {
     "devOnly": true
   },
   "devDependencies": {
     "@babel/cli": "^7.10.5",
-    "@kbn/babel-preset": "1.0.0",
-    "@kbn/dev-utils": "1.0.0",
-    "@kbn/utils": "1.0.0",
+    "@osd/babel-preset": "1.0.0",
+    "@osd/dev-utils": "1.0.0",
+    "@osd/utils": "1.0.0",
     "@types/joi": "^13.4.2",
     "@types/lodash": "^4.14.159",
     "@types/parse-link-header": "^1.0.0",

packages/osd-test/src/failed_tests_reporter/index.ts

@@ -17,4 +17,5 @@
  * under the License.
  */
 
-export { runFailedTestsReporterCli } from './run_failed_tests_reporter_cli';
+export { osdTestConfig } from './osd_test_config';
+export { opensearchDashboardsTestUser, opensearchDashboardsServerTestUser, adminTestUser } from './users';

packages/kbn-test/src/index.ts → packages/osd-test/src/index.ts

@@ -31,13 +31,13 @@ export { runTestsCli, processRunTestsCliOptions, startServersCli, processStartSe
 export { runTests, startServers } from './functional_tests/tasks';
 
 // @ts-ignore not typed yet
-export { KIBANA_ROOT } from './functional_tests/lib/paths';
+export { OPENSEARCH_DASHBOARDS_ROOT } from './functional_tests/lib/paths';
 
 // @ts-ignore not typed yet
-export { esTestConfig, createLegacyEsTestCluster } from './legacy_es';
+export { opensearchTestConfig, createLegacyEsTestCluster } from './legacy_opensearch';
 
 // @ts-ignore not typed yet
-export { kbnTestConfig, kibanaServerTestUser, kibanaTestUser, adminTestUser } from './kbn';
+export { osdTestConfig, opensearchDashboardsServerTestUser, opensearchDashboardsTestUser, adminTestUser } from './osd';
 
 // @ts-ignore not typed yet
 export { setupUsers, DEFAULT_SUPERUSER_PASS } from './functional_tests/lib/auth';

packages/kbn-test/src/legacy_es/index.js → packages/osd-test/src/legacy_opensearch/index.js

@@ -17,5 +17,5 @@
  * under the License.
  */
 
-export { createLegacyEsTestCluster } from './legacy_es_test_cluster.js';
-export { esTestConfig } from './es_test_config';
+export { createLegacyOpenSearchTestCluster } from './legacy_opensearch_test_cluster.js';
+export { opensearchTestConfig } from './opensearch_test_config';

packages/kbn-test/src/legacy_es/legacy_es_test_cluster.js → packages/osd-test/src/legacy_opensearch/legacy_opensearch_test_cluster.js

@@ -20,72 +20,72 @@
 import { resolve } from 'path';
 import { format } from 'url';
 import { get, toPath } from 'lodash';
-import { Cluster } from '@kbn/es';
+import { Cluster } from '@osd/opensearch';
 import { CI_PARALLEL_PROCESS_PREFIX } from '../ci_parallel_process_prefix';
-import { esTestConfig } from './es_test_config';
+import { opensearchTestConfig } from './opensearch_test_config';
 
-import { KIBANA_ROOT } from '../';
-import * as legacyElasticsearch from 'elasticsearch';
+import { OPENSEARCH_DASHBOARDS_ROOT } from '../';
+import * as legacyOpenSearch from 'elasticsearch';
 const path = require('path');
 const del = require('del');
 
-export function createLegacyEsTestCluster(options = {}) {
+export function createLegacyOpenSearchTestCluster(options = {}) {
   const {
-    port = esTestConfig.getPort(),
+    port = opensearchTestConfig.getPort(),
     password = 'changeme',
     license = 'oss',
     log,
-    basePath = resolve(KIBANA_ROOT, '.es'),
-    esFrom = esTestConfig.getBuildFrom(),
+    basePath = resolve(OPENSEARCH_DASHBOARDS_ROOT, '.opensearch'),
+    opensearchFrom = opensearchTestConfig.getBuildFrom(),
     dataArchive,
-    esArgs: customEsArgs = [],
-    esEnvVars,
-    clusterName: customClusterName = 'es-test-cluster',
+    opensearchArgs: customOpenSearchArgs = [],
+    opensearchEnvVars,
+    clusterName: customClusterName = 'opensearch-test-cluster',
     ssl,
   } = options;
 
   const clusterName = `${CI_PARALLEL_PROCESS_PREFIX}${customClusterName}`;
 
-  const esArgs = [
+  const opensearchArgs = [
     `cluster.name=${clusterName}`,
     `http.port=${port}`,
     'discovery.type=single-node',
-    `transport.port=${esTestConfig.getTransportPort()}`,
+    `transport.port=${opensearchTestConfig.getTransportPort()}`,
     `indices.id_field_data.enabled=false`,
-    ...customEsArgs,
+    ...customOpenSearchArgs,
   ];
 
   const config = {
-    version: esTestConfig.getVersion(),
+    version: opensearchTestConfig.getVersion(),
     installPath: resolve(basePath, clusterName),
-    sourcePath: resolve(KIBANA_ROOT, '../elasticsearch'),
+    sourcePath: resolve(OPENSEARCH_DASHBOARDS_ROOT, '../opensearch'),
     password,
     license,
     basePath,
-    esArgs,
+    opensearchArgs,
   };
 
   const cluster = new Cluster({ log, ssl });
 
-  return new (class EsTestCluster {
+  return new (class OpenSearchTestCluster {
     getStartTimeout() {
       const second = 1000;
       const minute = second * 60;
 
-      return esFrom === 'snapshot' ? 3 * minute : 6 * minute;
+      return opensearchFrom === 'snapshot' ? 3 * minute : 6 * minute;
     }
 
     async start() {
       let installPath;
 
-      if (esFrom === 'source') {
+      if (opensearchFrom === 'source') {
         installPath = (await cluster.installSource(config)).installPath;
-      } else if (esFrom === 'snapshot') {
+      } else if (opensearchFrom === 'snapshot') {
         installPath = (await cluster.installSnapshot(config)).installPath;
-      } else if (path.isAbsolute(esFrom)) {
-        installPath = esFrom;
+      } else if (path.isAbsolute(opensearchFrom)) {
+        installPath = opensearchFrom;
       } else {
-        throw new Error(`unknown option esFrom "${esFrom}"`);
+        throw new Error(`unknown option opensearchFrom "${opensearchFrom}"`);
       }
 
       if (dataArchive) {
@@ -94,27 +94,27 @@ export function createLegacyEsTestCluster(options = {}) {
 
       await cluster.start(installPath, {
         password: config.password,
-        esArgs,
-        esEnvVars,
+        opensearchArgs,
+        opensearchEnvVars,
       });
     }
 
     async stop() {
       await cluster.stop();
-      log.info('[es] stopped');
+      log.info('[opensearch] stopped');
     }
 
     async cleanup() {
       await this.stop();
       await del(config.installPath, { force: true });
-      log.info('[es] cleanup complete');
+      log.info('[opensearch] cleanup complete');
     }
 
     /**
-     * Returns an ES Client to the configured cluster
+     * Returns an opensearch Client to the configured cluster
      */
     getClient() {
-      return new legacyElasticsearch.Client({
+      return new legacyOpenSearch.Client({
         host: this.getUrl(),
       });
     }
@@ -124,7 +124,7 @@ export function createLegacyEsTestCluster(options = {}) {
     }
 
     getUrl() {
-      const parts = esTestConfig.getUrlParts();
+      const parts = opensearchTestConfig.getUrlParts();
       parts.port = port;
 
       return format(parts);
@@ -136,16 +136,16 @@ export function createLegacyEsTestCluster(options = {}) {
  *  Create a callCluster function that properly executes methods on an
  *  elasticsearch-js client
  *
- *  @param  {elasticsearch.Client} esClient
+ *  @param  {elasticsearch.Client} opensearchClient
  *  @return {Function}
  */
-function createCallCluster(esClient) {
+function createCallCluster(opensearchClient) {
   return function callCluster(method, params) {
     const path = toPath(method);
     const contextPath = path.slice(0, -1);
 
-    const action = get(esClient, path);
-    const context = contextPath.length ? get(esClient, contextPath) : esClient;
+    const action = get(opensearchClient, path);
+    const context = contextPath.length ? get(opensearchClient, contextPath) : opensearchClient;
 
     return action.call(context, params);
   };

packages/kbn-test/src/legacy_es/es_test_config.js → packages/osd-test/src/legacy_opensearch/opensearch_test_config.js

@@ -19,11 +19,11 @@
 
 import url, { format as formatUrl } from 'url';
 import pkg from '../../../../package.json';
-import { adminTestUser } from '../kbn';
+import { adminTestUser } from '../osd';
 
-export const esTestConfig = new (class EsTestConfig {
+export const opensearchTestConfig = new (class OpenSearchTestConfig {
   getVersion() {
-    return process.env.TEST_ES_BRANCH || pkg.version;
+    return process.env.TEST_OPENSEARCH_BRANCH || pkg.version;
   }
 
   getPort() {
@@ -35,37 +35,37 @@ export const esTestConfig = new (class EsTestConfig {
   }
 
   getBuildFrom() {
-    return process.env.TEST_ES_FROM || 'snapshot';
+    return process.env.TEST_OPENSEARCH_FROM || 'snapshot';
   }
 
   getTransportPort() {
-    return process.env.TEST_ES_TRANSPORT_PORT || '9300-9400';
+    return process.env.TEST_OPENSEARCH_TRANSPORT_PORT || '9300-9400';
   }
 
   getUrlParts() {
-    // Allow setting one complete TEST_ES_URL for Es like https://elastic:changeme@myCloudInstance:9200
-    if (process.env.TEST_ES_URL) {
-      const testEsUrl = url.parse(process.env.TEST_ES_URL);
+    // Allow setting one complete TEST_OPENSEARCH_URL for opensearch like https://elastic:changeme@myCloudInstance:9200
+    if (process.env.TEST_OPENSEARCH_URL) {
+      const testOpenSearchUrl = url.parse(process.env.TEST_OPENSEARCH_URL);
       return {
         // have to remove the ":" off protocol
-        protocol: testEsUrl.protocol.slice(0, -1),
-        hostname: testEsUrl.hostname,
-        port: parseInt(testEsUrl.port, 10),
-        username: testEsUrl.auth.split(':')[0],
-        password: testEsUrl.auth.split(':')[1],
-        auth: testEsUrl.auth,
+        protocol: testOpenSearchUrl.protocol.slice(0, -1),
+        hostname: testOpenSearchUrl.hostname,
+        port: parseInt(testOpenSearchUrl.port, 10),
+        username: testOpenSearchUrl.auth.split(':')[0],
+        password: testOpenSearchUrl.auth.split(':')[1],
+        auth: testOpenSearchUrl.auth,
       };
     }
 
-    const username = process.env.TEST_ES_USERNAME || adminTestUser.username;
-    const password = process.env.TEST_ES_PASSWORD || adminTestUser.password;
+    const username = process.env.TEST_OPENSEARCH_USERNAME || adminTestUser.username;
+    const password = process.env.TEST_OPENSEARCH_PASSWORD || adminTestUser.password;
 
     return {
       // Allow setting any individual component(s) of the URL,
-      // or use default values (username and password from ../kbn/users.js)
-      protocol: process.env.TEST_ES_PROTOCOL || 'http',
-      hostname: process.env.TEST_ES_HOSTNAME || 'localhost',
-      port: parseInt(process.env.TEST_ES_PORT, 10) || 9220,
+      // or use default values (username and password from ../osd/users.js)
+      protocol: process.env.TEST_OPENSEARCH_PROTOCOL || 'http',
+      hostname: process.env.TEST_OPENSEARCH_HOSTNAME || 'localhost',
+      port: parseInt(process.env.TEST_OPENSEARCH_PORT, 10) || 9220,
       auth: `${username}:${password}`,
       username: username,
       password: password,

packages/kbn-test/src/kbn/index.ts → packages/osd-test/src/osd/index.ts

@@ -17,5 +17,5 @@
  * under the License.
  */
 
-export { kbnTestConfig } from './kbn_test_config';
-export { kibanaTestUser, kibanaServerTestUser, adminTestUser } from './users';
+export { osdTestConfig } from './osd_test_config';
+export { opensearchDashboardsTestUser, opensearchDashboardsServerTestUser, adminTestUser } from './users';