Flaky start up when running app in local launchpad

[andretrump]

Describe the bug
I have the following project setup: A CAP application providing OData services which are consumed by UI5 frontends. The cds server is used also to serve the static content for the UI5 apps. Additionally I have the cds-launchpad-plugin installed such that I can also run the applications in a local launchpad. Sorry in advance: In the following descriptions I have to censor a lot since the project and code is owned by a customer which I cannot disclose.

When I run the tests on the "standalone" app everything works, but when I set the baseUrl to something like http://localhost:4004/$launchpad#MyEntity-create such that it uses the app running in the local launchpad it only works in one out of four cases, in the others the initialization of the UI bridge seems to fail with the error shown in the output below.

To Reproduce
Steps to reproduce the behavior:

1. Create a CAP project
2. Install the launchpad plugin: npm i -D cds-launchpad-plugin
3. Add a UI5 frontend
4. Execute tests on UI5 app running in the local launchapd.

Expected behavior
I would expect the tests to run as in the standalone app.

Logs/Console Output
Output when the error occurs:

[0-0] RUNNING in firefox - file:///webapp/test/specs/<censored>.test.js
[0-0] 2024-12-10T12:11:12.767Z ERROR webdriver: Request failed with status 500 due to javascript error: Error: Failed to resolve dependencies of 'sap/ui/test/RecordReplay.js'
[0-0]  -> 'sap/ui/test/selectors/_ControlSelectorGenerator.js'
[0-0]   -> 'sap/ui/test/selectors/_selectors.js'
[0-0]    -> 'sap/ui/test/selectors/_DropdownItem.js'
[0-0]     -> 'sap/m/SelectList.js': failed to execute module factory for ''sap/m/SelectList.js'': e is undefined
[0-0] 2024-12-10T12:11:12.768Z ERROR @wdio/utils:shim: javascript error: Error: Failed to resolve dependencies of 'sap/ui/test/RecordReplay.js'
[0-0]  -> 'sap/ui/test/selectors/_ControlSelectorGenerator.js'
[0-0]   -> 'sap/ui/test/selectors/_selectors.js'
[0-0]    -> 'sap/ui/test/selectors/_DropdownItem.js'
[0-0]     -> 'sap/m/SelectList.js': failed to execute module factory for ''sap/m/SelectList.js'': e is undefined
[0-0]     at getErrorFromResponseBody (file:///home/user/projects/<censored>/node_modules/webdriver/build/utils.js:195:12)
[0-0]     at NodeJSRequest._request (file:///home/user/projects/<censored>/node_modules/webdriver/build/request/index.js:193:23)
[0-0]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[0-0]     at async Browser.wrapCommandFn (file:///home/user/projects/<censored>/node_modules/@wdio/utils/build/shim.js:90:29)
[0-0]     at async Browser.wrapCommandFn (file:///home/user/projects/<censored>/node_modules/@wdio/utils/build/shim.js:90:29)
[0-0]     at async clientSide_injectUI5 (/home/user/projects/<censored>/node_modules/wdio-ui5-service/client-side-js/injectUI5.cjs:2:12)
[0-0]     at async injectUI5 (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-bridge.js:122:25)
[0-0]     at async Service.injectUI5 (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/service.js:96:13)
[0-0]     at async Service.before (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/service.js:55:17)
[0-0]     at async Promise.all (index 1)
[0-0] 2024-12-10T12:11:14.045Z ERROR webdriver: Request failed with status 500 due to javascript error: TypeError: window.wdi5.errorHandling is undefined
[0-0] 2024-12-10T12:11:15.375Z ERROR webdriver: Request failed with status 500 due to javascript error: Error: Failed to resolve dependencies of 'sap/ui/test/RecordReplay.js'
[0-0]  -> 'sap/ui/test/selectors/_ControlSelectorGenerator.js'
[0-0]   -> 'sap/ui/test/selectors/_selectors.js'
[0-0]    -> 'sap/ui/test/selectors/_DropdownItem.js'
[0-0]     -> 'sap/m/SelectList.js': failed to execute module factory for ''sap/m/SelectList.js'': e is undefined
[0-0] 2024-12-10T12:11:15.375Z ERROR @wdio/utils:shim: javascript error: Error: Failed to resolve dependencies of 'sap/ui/test/RecordReplay.js'
[0-0]  -> 'sap/ui/test/selectors/_ControlSelectorGenerator.js'
[0-0]   -> 'sap/ui/test/selectors/_selectors.js'
[0-0]    -> 'sap/ui/test/selectors/_DropdownItem.js'
[0-0]     -> 'sap/m/SelectList.js': failed to execute module factory for ''sap/m/SelectList.js'': e is undefined
[0-0]     at getErrorFromResponseBody (file:///home/user/projects/<censored>/node_modules/webdriver/build/utils.js:195:12)
[0-0]     at NodeJSRequest._request (file:///home/user/projects/<censored>/node_modules/webdriver/build/request/index.js:193:23)
[0-0]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[0-0]     at async Browser.wrapCommandFn (file:///home/user/projects/<censored>/node_modules/@wdio/utils/build/shim.js:90:29)
[0-0]     at async Browser.wrapCommandFn (file:///home/user/projects/<censored>/node_modules/@wdio/utils/build/shim.js:90:29)
[0-0]     at async clientSide_injectUI5 (/home/user/projects/<censored>/node_modules/wdio-ui5-service/client-side-js/injectUI5.cjs:2:12)
[0-0]     at async injectUI5 (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-bridge.js:122:25)
[0-0]     at async _waitForUI5 (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-bridge.js:475:13)
[0-0]     at async Browser.<anonymous> (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-bridge.js:299:9)
[0-0]     at async Browser.wrapCommandFn (file:///home/user/projects/<censored>/node_modules/@wdio/utils/build/shim.js:90:29)
[0-0] Error in "AfterTest Hook"
Error: Failed to resolve dependencies of 'sap/ui/test/RecordReplay.js'
 -> 'sap/ui/test/selectors/_ControlSelectorGenerator.js'
  -> 'sap/ui/test/selectors/_selectors.js'
   -> 'sap/ui/test/selectors/_DropdownItem.js'
    -> 'sap/m/SelectList.js': failed to execute module factory for ''sap/m/SelectList.js'': e is undefined
[0-0] javascript error in "<censored>"
javascript error: TypeError: window.wdi5.errorHandling is undefined
    at async clientSide_getControl (/home/user/projects/<censored>/node_modules/wdio-ui5-service/client-side-js/getControl.cjs:3:12)
    at async WDI5Control._getControl (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-control.js:595:26)
    at async WDI5Control.init (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-control.js:59:31)
    at async Browser.<anonymous> (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-bridge.js:224:33)
[0-0] FAILED in firefox - file:///webapp/test/specs/<censored>.test.js

 "spec" Reporter:
------------------------------------------------------------------
[firefox 115.9.1 linux #0-0] Running: firefox (v115.9.1) on linux
[firefox 115.9.1 linux #0-0] Session ID: 58fa9775-c50e-4583-9db5-92f7fceb9150
[firefox 115.9.1 linux #0-0]
[firefox 115.9.1 linux #0-0] » /webapp/test/specs/<censored>.test.js
[firefox 115.9.1 linux #0-0] <censored>
[firefox 115.9.1 linux #0-0]    ✖ <censored>
[firefox 115.9.1 linux #0-0]
[firefox 115.9.1 linux #0-0] 1 failing (3.1s)
[firefox 115.9.1 linux #0-0]
[firefox 115.9.1 linux #0-0] 1) <censored>
[firefox 115.9.1 linux #0-0] TypeError: window.wdi5.errorHandling is undefined
[firefox 115.9.1 linux #0-0] javascript error: TypeError: window.wdi5.errorHandling is undefined
[firefox 115.9.1 linux #0-0]     at async clientSide_getControl (/home/user/projects/<censored>/node_modules/wdio-ui5-service/client-side-js/getControl.cjs:3:12)
[firefox 115.9.1 linux #0-0]     at async WDI5Control._getControl (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-control.js:595:26)
[firefox 115.9.1 linux #0-0]     at async WDI5Control.init (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-control.js:59:31)
[firefox 115.9.1 linux #0-0]     at async Browser.<anonymous> (file:///home/user/projects/<censored>/node_modules/wdio-ui5-service/esm/lib/wdi5-bridge.js:224:33)

wdio.conf.js with applied workaround that seems to fix the issue. (For details see "Additional context")

const outputDir = path.resolve("./webapp/test/__out__");
exports.config = {
    wdi5: {
        screenshotPath: outputDir,
        waitForUI5Timeout: 30000,
        logLevel: "error",
        skipInjectUI5OnStart: true
    },
    specs: ["./**/*.test.js"],
    maxInstances: 10,
    capabilities: [
        {
            acceptInsecureCerts: true,
            browserName: "firefox",
            browserVersion: "0.35.0",
            "moz:firefoxOptions": {
                binary: "/extbin/bin/firefox",
                args: ["-headless"],
                log: { level: "trace" },
                prefs: {
                    "browser.download.dir": outputDir,
                    "browser.download.folderList": 2,
                    "browser.download.manager.showWhenStarting": false,
                    "browser.helperApps.neverAsk.saveToDisk": "*/*",
                    "dom.disable_open_during_load": false
                }
            },
            "wdi5:authentication": {
                provider: "BasicAuth",
                basicAuthUrls: [
                    "http://localhost:4004/odata/v4/my-service/$metadata"
                ]
            }
        }
    ],
    logLevel: "error",
    bail: 0,
    baseUrl: "http://localhost:4004/$launchpad#MyEntity-create",
    waitforTimeout: 10000,
    connectionRetryTimeout: process.argv.indexOf("--debug") > -1 ? 1200000 : 120000,
    connectionRetryCount: 3,
    services: ["ui5"],
    framework: "mocha",
    reporters: [
        "spec",
        ["junit", { outputDir: outputDir }]
    ],
    mochaOpts: {
        ui: "bdd",
        timeout: 600000
    },
    /**
     * Gets executed once before all workers get launched.
     * @param {Object} config wdio configuration object
     * @param {Array.<Object>} capabilities list of capabilities details
     */
    onPrepare: function (config, capabilities) {
        if (!fs.existsSync(outputDir)) {
            fs.mkdirSync(outputDir);
        } else {
            fs.readdirSync(outputDir)
                .forEach(file => fs.rmSync(path.join(outputDir, file), { recursive: true }));
        }
    },
    /**
     * Gets executed before test execution begins. At this point you can access to all global
     * variables like `browser`. It is the perfect place to define custom commands.
     * @param {Array.<Object>} capabilities list of capabilities details
     * @param {Array.<String>} specs        List of spec file paths that are to be run
     * @param {Object}         browser      instance of created browser/device session
     */
    before: async function (capabilities, specs) {
        global.outputDir = outputDir;
        await injectUI5InLaunchpad("MyEntity-create");
    },
    /**
     * Function to be executed after a test (in Mocha/Jasmine only)
     * @param {Object}  test             test object
     * @param {Object}  context          scope object the test was executed with
     * @param {Error}   result.error     error object in case the test fails, otherwise `undefined`
     * @param {Any}     result.result    return object of test function
     * @param {Number}  result.duration  duration of test
     * @param {Boolean} result.passed    true if test has passed, otherwise false
     * @param {Object}  result.retries   informations to spec related retries, e.g. `{ attempts: 0, limit: 0 }`
     */
    afterTest: async function (test, context, { error, result, duration, passed, retries }) {
        if (error) {
            await browser.screenshot();
        }
    },
};

Runtime Env (please complete the following information):

• wdi5/wdio-ui5-service-version: 2.0.10
• UI5 version: 1.120.22
• wdio-version (output of wdio --version): 8.40.6
• node-version (output of node --version): 20.11.0
• OS: Debian GNU/Linux 12 (bookworm)
• Browser + Version: Mozilla Firefox 115.9.1esr

Additional context
The tests are run in firefox in headless mode since I work in the SAP BAS.

I also tired to set btpWorkZoneEnablement: true but this caused some other error, so I think the local launchpad work somehow differently than the launchpad provided by Workzone.

Since it sometimes works, I suspect some sort of timing issue here. Therefore, I implemented the following workaround that makes it run stable:

1. I set skipInjectUI5OnStart: true
2. I handle the injection of UI5 manually in the before hook in my wdio.conf.js

async function injectUI5InLaunchpad(sIntent, iTimeout=20000) {
    // wait for "root div" of app to be loaded
    await browser.waitUntil(async () => {
        return await $(`#application-${sIntent}`).isDisplayedInViewport();
    }, { timeout: iTimeout });
    const oWdi5Service = new Wdi5Service();
    await oWdi5Service.injectUI5();
    // wait for UI5
    await browser.executeAsync((done) => {
        window.bridge
            .waitForUI5()
            .then(() => done(true));
    })
}