Unstable workflow "Test Current"

[gntpet]

Hey @freddydk ,

We need your help again :) This time with "Test Current" workflow. We just configured it recently and noticed very unstable behavior.

For e.g.

• Sometimes it is successful, and executes all the tests
• Sometimes it fails in "Running Tests" phase with an error: Error: Unexpected error when running action. Error Message: ClientSession State is InError (Wait time 18 seconds)
• Sometimes it fails in "Compiling Apps" phase with an error: Unexpected error when running action. Error Message: Out of memory.,
• Sometimes if fails in "Importing Test Data" phase with an error: Unexpected error when running action. Error Message: An internal WebSocket error occurred. Please see the innerException, if present, for more details.
• Sometimes it fails in "Running Tests" phase with an error: Unexpected error when running action. Error Message: Cannot open page 130455. You might need to import the test toolkit to the container and/or remove the folder

An all this happens for the same codebase/branch

As you can see it's totally random. For the CICD part, we don't have these issues. But in CICD part, we are using compileFolder, and do not create container.

Our settings are:

AL-GO-Settings:

{
  "type": "PTE",
  "templateUrl": "https://github.com/microsoft/AL-Go-PTE@main",
  "treatTestFailuresAsWarnings": true,
  "doNotRunBcptTests": true,
  "enableCodeCop": true,
  "rulesetFile": "../cicd.ruleset.json",
  "companyName": "TNM Automated Test",
  "runs-on": "srs-eid",
  "githubRunner": "srs-eid",
  "skipUpgrade": true,
  "environments": [
    "QA"
  ],
  "CICDPushBranches": [
    "main",
    "release/*"
  ],
  "CICDPullRequestBranches": [
    "main",
    "release/*",
    "features/*",
    "bug/*"
  ],
  "DeliverToStorage": {
    "Branches": [
      "main",
      "release/*"
    ]
  },
  "DeployToQA": {
    "EnvironmentType": "SaaS",
    "EnvironmentName": "QA",
    "Projects": "SRS.nl",
    "Branches": [
      "main"
    ],
    "ContinuousDeployment": true,
    "runs-on": "srs-eid"
  },
  "DeployToQA24": {
    "EnvironmentType": "SaaS",
    "EnvironmentName": "QA24",
    "Projects": "SRS.nl",
    "Branches": [
      "feature/bc24-update-EID-3621"
    ],
    "ContinuousDeployment": true,
    "runs-on": "srs-eid"
  },  
  "DeployToUAT": {
    "EnvironmentType": "SaaS",
    "EnvironmentName": "UAT",
    "Projects": "SRS.nl",
    "Branches": [
      "release/*"
    ],
    "ContinuousDeployment": false,
    "runs-on": "srs-eid"
  },
  "DeployToPROD": {
    "EnvironmentType": "SaaS",
    "EnvironmentName": "PROD",
    "Projects": "SRS.nl",
    "Branches": [
      "release/*"
    ],
    "ContinuousDeployment": false,
    "runs-on": "srs-eid"
  },
  "EnableTaskScheduler": true,
  "doNotRunTests": true,
  "alwaysBuildAllProjects": true,
  "PullRequestTrigger": "pull_request",
  "CurrentSchedule": "0 2 * * *",
  "NextMinorSchedule": "0 2 * * 6",
  "NextMajorSchedule": "0 2 * * 0",
  "templateSha": "0af8feb4cf098ebec44278866877fc9aceff0513"
}

Test current

{
    "artifact": "//*//latest",
    "cacheImageName": "",
    "versioningStrategy": 15,
    "doNotRunTests": false,
    "useCompilerFolder": false,
    "doNotPublishApps": false,
    "treatTestFailuresAsWarnings": true,
    "installTestRunner": true,
    "installApps": [
        "https://verySecretUrlToDocumentCaptureDependecies"
    ]
}

settings.json from project folder

{
    "country": "base",
    "artifact": "//*//latest",
    "appFolders": [
        "App"
    ],
    "testFolders": [
        "Test"
    ],
    "useCompilerFolder": true,
    "doNotPublishApps": true,
    "generateDependencyArtifact": false
}

custom ImportTestDataInBcContainer.ps1 script

Param([Hashtable]$parameters) 
New-CompanyInBcContainer  -containerName $parameters.containerName -tenant $parameters.tenant -companyName "TNM Automated Test"
Invoke-NavContainerCodeunit -containerName $parameters.containerName -tenant $parameters.tenant -CompanyName "TNM Automated Test" -Codeunitid 2 6> $null    

We do use larger github runner for running our pipelines. Specs are:
16 | 64 GB | 600 GB | Windows

Any advise?

Best Regards,
Gintautas

[gntpet]

One more error to the collection. This time it's from step "Importing test data"

Error: Unexpected error when running action. Error Message: Cannot bind argument to parameter 'WindowsAccount' because it is null., StackTrace: at Invoke-ScriptInBcContainer, C:\ProgramData\BcContainerHelper\6.0.17\BcContainerHelper\ContainerHandling\Invoke-ScriptInNavContainer.ps1: line 112 <- at Invoke-NavContainerCodeunit, C:\ProgramData\BcContainerHelper\6.0.17\BcContainerHelper\ObjectHandling\Invoke-NavContainerCodeunit.ps1: line 49 <- at <ScriptBlock>, C:\a\srs-bc\srs-bc\SRS.base\.AL-Go\ImportTestDataInBcContainer.ps1: line 3 <- at <ScriptBlock>, C:\ProgramData\BcContainerHelper\6.0.17\BcContainerHelper\AppHandling\Run-AlPipeline.ps1: line 2315 <- at <ScriptBlock>, C:\ProgramData\BcContainerHelper\6.0.17\BcContainerHelper\AppHandling\Run-AlPipeline.ps1: line 947 <- at Run-AlPipeline, C:\ProgramData\BcContainerHelper\6.0.17\BcContainerHelper\AppHandling\Run-AlPipeline.ps1: line 909 <- at <ScriptBlock>, C:\a\_actions\microsoft\AL-Go-Actions\v5.1\RunPipeline\RunPipeline.ps1: line 361 <- at <ScriptBlock>, C:\a\_temp\ac4616ac-5fe8-44a0-abca-b1ee51138d11.ps1: line 4 <- at <ScriptBlock>, <No file>: line 1

[freddydk]

Could you share your ImportTestDataInBcContainer with us?

[freddydk]

Could you share the full log of a failing Test Current?

[gntpet]

Could you share your ImportTestDataInBcContainer with us?

It's already shared. See at the bottom of the post.

Could you share the full log of a failing Test Current?

Will send file via Yammer

[freddydk]

Looking at your log, the problem is probably Invoke-NavContainerCodeUnit, which I myself stopped using earlier due to instabilities.

Reason is that the Invoke-NavCodeUnit cmdlet only works with Windows Authentication. For this reason - BcContainerHelper creates a tmp windows user and uses this to invoke the codeunit.
Apparently - that doesn't always work:-(

I modified my apps to include an API page and then I used Invoke-BcContainerAPI to invoke the code I needed - that always works and is something I can use in online environments as well.
Another mechanism is to have an install codeunit in the app, which runs the code needed.