Publish-NavApp in a container hangs running PS7 (after a prev. publish failure)

[MODUSCarstenScholling]

Publish-NavApp in a container hangs running PS7 (after a prev. publish failure)

Publish-NavApp in a container hangs running PS7. But this is dependent on other commands. I have added a script below to repro. It has to do with a failing publish of one app and a failing publish of another app depending on the previous failing app. But it is required to have the runtime creation. I have tried to reduce, but I failed.

The errors for the last app are never shown in the console. instead the command hangs for hours if not canceled and the error is pushed to the event log.

Due to the fact that we have 150 apps, not every combination work together. This is not known while we create runtime packages for every app and every supported platform. So there can be compiler errors due to obsoletions or similar stuff.

The result is shown in the table below. The process hangs always when usePwshForBc24 is false. But due to the other case with other issues #3575 and PowerShell/PowerShell#23982, we can not turn it on.

Isolation	PowerShell	usePwshForBc24	Hangs
process	7.4.1	true	false
process	7.4.1	false	true
process	5.1	true	false
process	5.1	false	true
hyperv	7.4.1	true	false
hyperv	7.4.1	false	true
hyperv	5.1	true	false
hyperv	5.1	false	true

I have added the transcripts for every combination and the script to repro. If you need, I can provide

Scripts used to create container and cause the issue

The error or hang occurs when processing publishing of Tour Planning, after Packing Unit Mgt. failed already. The list of apps depends on each other.

Import-Module -Name 'BCContainerHelper';

$containerName = 'CrashMe';
$isolation = 'process';
$usePwshForBc24 = $false;

$bcContainerHelperConfig.usePwshForBc24 = $usePwshForBc24;

try
{
    $rootPath = $PSScriptRoot;
    Start-Transcript -Path (Join-Path -Path $rootPath -ChildPath "transcript-$($PSVersionTable.PSVersion)-$($isolation)-usePwsh_$($usePwshForBc24).txt") -Force -IncludeInvocationHeader;

    $bcContainerHelperConfig;

    if (!(Get-BcContainers -ErrorAction SilentlyContinue | Where-Object { $_ -ieq $containerName }))
    {
        $artifactUrl = Get-BCArtifactUrl -type OnPrem -country 'de' -version '24' -select Latest -accept_insiderEula;

        "New BC Container"; (Measure-Command {
            New-BcContainer -containerName $containerName -artifactUrl $artifactUrl -accept_eula -accept_insiderEULA -accept_outdated -auth NavUserPassword `
	            -Credential (New-Object System.Management.Automation.PSCredential -argumentList 'admin', (ConvertTo-SecureString -String "admin" -AsPlainText -Force)) `
	            -memoryLimit "8G" -updateHosts -useBestContainerOS -isolation $isolation -shortcuts None;
        }).TotalSeconds;
    }
    else
    {
        #"Restart BC Container"; (Measure-Command {
        #    Restart-BcContainer -containerName $containerName;
        #}).TotalSeconds;
    }

    $runtimeIndex = 1;

    $apps = @(
        (Join-Path -Path $rootPath -ChildPath "MODUS Consult GmbH_MODUS M365 Foundation_23.3.24070401.0_app.app"),
        (Join-Path -Path $rootPath -ChildPath "MODUS Consult GmbH_MODUS M365 Foundation_23.3.24040302.24040302_app.app"),
        (Join-Path -Path $rootPath -ChildPath "MODUS Consult GmbH_MODUS M365 Base_21.2.24030601.24031801_app.app"),
        (Join-Path -Path $rootPath -ChildPath "MODUS Consult GmbH_MODUS M365 Charge_22.3.24022601.24022601_app.app"),
        (Join-Path -Path $rootPath -ChildPath "MODUS Consult GmbH_MODUS M365 Packing Unit Management_22.3.24010901.24010901_app.app"),
        (Join-Path -Path $rootPath -ChildPath "MODUS Consult GmbH_MODUS M365 Tour Planning_23.1.24020901.24020901_app.app")
    );

    "Get App Info"; (Measure-Command {
        $unpublishApps = Get-BcContainerAppInfo -containerName $containerName -tenant default -tenantSpecificProperties -sort DependenciesLast | Where-Object { $_.Name.StartsWith('MODUS') };
    }).TotalSeconds;


    "Unpublish"; (Measure-Command {
        $unpublishApps | ForEach-Object {
            UnPublish-BcContainerApp -containerName $containerName -tenant default -name $_.Name -version $_.Version -doNotSaveData -doNotSaveSchema -force;
        }
    }).TotalSeconds;


    "Create Folder"; (Measure-Command {
        Invoke-ScriptInBcContainer `
	        -containerName $containerName `
	        -scriptblock {
                Remove-Item -Path 'C:\CrashMe\' -Recurse -Force -ErrorAction SilentlyContinue;
                New-Item -Path 'C:\CrashMe' -ItemType Directory -Force;
            } -Verbose;
    }).TotalSeconds;


    "Copy to Container"; (Measure-Command {
        $containerApps = @();

        $apps | ForEach-Object {
            $app = $_;

            $containerApp = Join-Path -Path 'C:\CrashMe' -ChildPath (Split-Path -Path $app -Leaf);
            Copy-FileToBcContainer -containerName $containerName -localPath $app -containerPath $containerApp;

            $containerApps += $containerApp;
        };
    }).TotalSeconds;


    "Crash the party"; (Measure-Command {
        $index = 0;

        $containerApps | ForEach-Object {
            $index++;
            $app = $_;

	        Invoke-ScriptInBcContainer `
		        -containerName $containerName `
                -argumentList $app, ($index -le $runtimeIndex) `
		        -scriptblock {
                    param([string]$app, $isRuntime)
            
                    try
                    {
                        Write-Host "Publishing app $($app)";
                        Publish-NavApp -ServerInstance $serverInstance -path $app -skipVerification -packageType Extension -Verbose -ErrorAction Stop;
                        Write-Host "Finished publishing app";
                    }
                    catch
                    {
                        Write-Host "Error publishing app $($app): $($_)";
                    }

                    if ($isRuntime)
                    {
                        try
                        {
					        $info = Get-NAVAppInfo -Path $app;
					        $appFileName = "$($info.Publisher)_$($info.Name)_$($info.Version).runtime.app".Split([System.IO.Path]::GetInvalidFileNameChars()) -join '';
					        $appFilePath = Join-Path -Path 'C:\CrashMe' -ChildPath $appFileName;

					        Write-Host "Creating Runtime Package $([System.IO.Path]::GetFileName($appFileName)).";
					        Get-NavAppRuntimePackage -ServerInstance $serverInstance -appName $info.Name -appPublisher $info.Publisher -appVersion $info.Version -Path $appFilePath;
					        Write-Host "Finished creating Runtime Package";
                        }
                        catch
                        {
                            Write-Host "Error creating runtime for app $($app): $($_)";
                        }
                    }
	            } -Verbose;
        };
    }).TotalSeconds;
}

finally
{
    Stop-Transcript;
}

Full output of scripts

CrashMe-Transcripts.zip

[MODUSCarstenScholling]

@freddydk I cannot proof this is a BCContainerHelper issue, but it took me hours already to explain this to myself. I cannot imagine how to explain this to anyone else not familiar with BCCH, PS, and BC.

[freddydk]

Currently on vacation - but you can try a few things.
Remove-BcContainerSession before entering the loop with publishing apps - this removes prior "state" of stuff.
Note that you can also specify -usepwsh on a specific Invoke-Script call - or you can try -useSession false.

I will look closer at the issue when I am back

[MODUSCarstenScholling]

Have some great days off!

Removing the session did not work. I was aware of -usePwsh $true, using this for now as a workaround.

[epernst]

I'm experiencing this too, including when using UnPublish-BcContainerApp.

[MODUSCarstenScholling]

@epernst Welcome to the club, even if you didn't ask for it.

Every second or third day, I see another issue with Invoke* or PS7 usage. The problem is, that different issues can be solved with different settings for UsePwsh. Scripts, peppered with workarounds for various problems, sometimes like this, sometimes like that.

Different behaviors on different systems with containers for the same version, the patches, isolation and other stuff is identical. I am lost with all this, loosing hours and hours with troubleshooting for problems I cannot solve using a single way...

[epernst]

@MODUSCarstenScholling I've just upgraded our onprem BC deployment Powershell scripts for v24. That was not too much fun. I thought I had to use PS7 for that, but a lot of the MS cmdlets failed, if I used PS7. Some, like all those from the web administration module, could not load using PS7 (depending on a module that cannot be used in PS7), while others just gave strange errors when called from PS7. Calling the same functions from PS5, worked fine, even if PS5 actually calls the PS7 version of the module. And as you write, I have also spent hours and hours debugging this.
No guide lines or other documentation from MS, on when to use what. So, I'm quite disappointed with this "move" to from PS5 to PS7.

[MODUSCarstenScholling]

@freddydk Is there anything you can say about this issue?