Run-ALCops (6.0.19-preview1213) stops working with missing file in BC24, BC23 works #3575
Comments
|
Could you try to insert these lines inside the Invoke-ScriptInBcContainer I have seen cases before, where a copied file isn't available immediately inside the container in either process or hyperv isolation (cannot remember which one) |
|
Well, of course. I've added the following Write-Host "STEP 04.3: appTmpFile = -$($appTmpFile)- => EXISTS LEAF $(Test-Path -Path $appTmpFile -PathType Leaf)";
# Wait for file to be accessible in container
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $appTmpFile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
Write-Host "STEP 04.4: appTmpFile = -$($appTmpFile)- => EXISTS: $(Test-Path -Path $appTmpFile)";And then I stored 12 files to process. This is the output:
This is a delay of more than a minute in the process, an average of 5+ seconds per file. This is far to much. And it only applies to BC24 containers. |
|
And an example of the original process with 12 apps on BC 25 using PS Core (above was Windows PS and BC 24): First example output: Times:
3 minutes waiting time. Not acceptable... UPDATE: The times do not correspond to the size of the app files (added above). The sizes are different. Interesting is, that the 2.x second entries apply to our editor app and it's test app. But the latter is 3 times smaller than the editor itself. |
Tell that to docker :-) Obviously, we can find a different way to copy the file into the container - but files should really be available for docker when they are copied to a share. Maybe antivirus is playing a role here? or process/hyperv? |
Really? What does docker differently between BC 23 and BC 24+? |
|
BC24 runs PowerShell 7 inside the container - BC23 runs PowerShell 5.1 You can set $bcContainerHelperConfig.usePwshForBC24 = $false to see if this changes things. I assume that it has something to do with that - but again, I copy the file to a share, then I run a script in the container, which cannot see the file - it likely isn't because the file is version 24 - it likely is because of PowerShell 7 in docker - but we have a similar waiting for a file inside the Compile-AppInBcContainer, so this is something we have seen before. |
|
Ok, thanks. I am now again using PS5 with $bcContainerHelperConfig.usePwshForBC24 = $false. I'm not that experienced with all that docker stuff and sessions. But what I find strange is that the app files are copied directly into the host helper folder (root) as guid.app. Was that the original intention? |
|
Reason is that this folder is shared with the container in order to check the signature inside the container. |
|
There has been a ton of problems after the PS7 stuff - needless to say that I didn't like that change. |
Yes, I know. The question was >why are the apps copied directly to "C:\ProgramData\BCContainerHelper\guid.app" instead of copying them to a subfolder.< |
I will try that in the next days. But I cannot promise, because I had a lot of issues with process in the past. usePwSh btw. reduces the wairing time to 0 (or 0.004 seconds). |
|
What OS are you running on the host? |
|
Windows Server 2019 and Windows 11. First is an Agent, second my Personal Maschine. |
|
And I assume it fails on both? |
|
Tried this returns Can you repro the problem with a script like this? |
|
That is interesting: Still with isolation hyperv and $bcContainerHelperConfig.usePwshForBC24 = $true.
$lessAppNames = @($appNames | Where-Object { ($_ -imatch 'Foundation') });#-or ($_ -imatch 'M365 Base')
$alcOutput = Run-AlCops `
-containerName $containerName `
-credential $containerCredential `
-apps $lessAppNames `
-Verbose;
1..10 | ForEach-Object {
$destname = Join-Path $bcContainerHelperConfig.hostHelperFolder "$([guid]::NewGuid().ToString()).app"
$out = new-object byte[] 104857600; (new-object Random).NextBytes($out)
[IO.File]::WriteAllBytes($destname, $out)
Invoke-ScriptInBcContainer -containerName $containerName -scriptblock { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
} -argumentList (Get-BcContainerPath -containerName $containerName -path $destname)
Remove-Item $destname -force
}
Write-Host "NO REPRO";
1..3 | ForEach-Object {
$destname = Join-Path $bcContainerHelperConfig.hostHelperFolder "$([guid]::NewGuid().ToString()).app"; $destname | Out-File -FilePath $destname -Force;
Invoke-ScriptInBcContainer -containerName $containerName -scriptblock { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
$content = Get-Content -Path $tmpfile; Write-Host "Content: $($content)";
} -argumentList (Get-BcContainerPath -containerName $containerName -path $destname)
Remove-Item -Path $destname -Force;
}
Write-Host "DOES REPRO";
1..3 | ForEach-Object {
$destname = Join-Path $bcContainerHelperConfig.hostHelperFolder "$([guid]::NewGuid().ToString()).app"; $destname | Out-File -FilePath $destname -Force;
Invoke-ScriptInBcContainer -containerName $containerName -scriptblock { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
$content = Get-Content -Path $tmpfile; Write-Host "Content: $($content)";
Remove-Item -Path $tmpfile -Force;
} -argumentList (Get-BcContainerPath -containerName $containerName -path $destname)
}Output: THE FINALS
$containerName = 'RunAlCops241';
$Global:sessionCache = @{};
function MyInvokeInABox
{
param
(
[string]$containerName,
[scriptblock]$scriptblock,
[object[]]$argumentList,
[switch]$useSessionCache
)
if ($Global:sessionCache.ContainsKey($containerName) -and $useSessionCache.IsPresent)
{
$session = $Global:sessionCache[$containerName];
}
else
{
$containerId = (docker ps --format "{{.Names}}:{{.ID}}" -a --no-trunc | Where-Object { $_.StartsWith("$($containerName):") }).Split(':')[1];
$session = New-PSSession -ContainerId $containerId -RunAsAdministrator -ConfigurationName 'PowerShell.7';
if ($Global:sessionCache.ContainsKey($containerName))
{
$Global:sessionCache.Remove($containerName);
}
$Global:sessionCache += @{ $containerName = $session; };
}
Invoke-Command -Session $session -ScriptBlock $scriptblock -ArgumentList $argumentList;
}
Write-Host "NO REPRO (SESSION CACHE, NO FILE OPERATION INSIDE CONTAINER)";
1..3 | ForEach-Object {
$destname = Join-Path $bcContainerHelperConfig.hostHelperFolder "$([guid]::NewGuid().ToString()).app"; $destname | Out-File -FilePath $destname -Force;
MyInvokeInABox -containerName $containerName -useSessionCache -scriptblock { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
$content = Get-Content -Path $tmpfile; Write-Host "Content: $($content)";
} -argumentList (Get-BcContainerPath -containerName $containerName -path $destname);
Remove-Item -Path $destname -Force;
}
Write-Host "AGAIN NO REPRO (NO SESSION CACHE, WITH FILE OPERATION INSIDE CONTAINER)";
1..3 | ForEach-Object {
$destname = Join-Path $bcContainerHelperConfig.hostHelperFolder "$([guid]::NewGuid().ToString()).app"; $destname | Out-File -FilePath $destname -Force;
MyInvokeInABox -containerName $containerName -scriptblock { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
$content = Get-Content -Path $tmpfile; Write-Host "Content: $($content)";
Remove-Item -Path $tmpfile -Force;
} -argumentList (Get-BcContainerPath -containerName $containerName -path $destname);
}
Write-Host "DOES REPRO (WITH SESSION CACHE, WITH FILE OPERATION INSIDE CONTAINER)";
1..3 | ForEach-Object {
$destname = Join-Path $bcContainerHelperConfig.hostHelperFolder "$([guid]::NewGuid().ToString()).app"; $destname | Out-File -FilePath $destname -Force;
MyInvokeInABox -containerName $containerName -useSessionCache -scriptblock { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
$content = Get-Content -Path $tmpfile; Write-Host "Content: $($content)";
Remove-Item -Path $tmpfile -Force;
} -argumentList (Get-BcContainerPath -containerName $containerName -path $destname);
}RESULT with isolation = hyperv Are you able to provide a fix, workaround, helpful comment? Is there something wrong with the PS7 session? Or the file operation? Or .NET? Do you have contact to someone (PS team, .NET team, ...) who is able to point something out with tne above information? |
|
Will have a look and see if I can understand what is happening here. |
|
So, it really is this simple... $containerName = 'bc24';
$scriptblock = { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
$content = Get-Content -Path $tmpfile; Write-Host "Content: $($content)";
Remove-Item -Path $tmpfile -Force;
}
$destname = Join-Path $bcContainerHelperConfig.hostHelperFolder "$([guid]::NewGuid().ToString()).app"; $destname | Out-File -FilePath $destname -Force;
Invoke-ScriptInBcContainer -containerName $containerName -scriptblock $scriptblock -argumentList (Get-BcContainerPath -containerName $containerName -path $destname);
$destname = Join-Path $bcContainerHelperConfig.hostHelperFolder "$([guid]::NewGuid().ToString()).app"; $destname | Out-File -FilePath $destname -Force;
Invoke-ScriptInBcContainer -containerName $containerName -scriptblock $scriptblock -argumentList (Get-BcContainerPath -containerName $containerName -path $destname);
Start-Sleep -Seconds 20
$destname = Join-Path $bcContainerHelperConfig.hostHelperFolder "$([guid]::NewGuid().ToString()).app"; $destname | Out-File -FilePath $destname -Force;
Invoke-ScriptInBcContainer -containerName $containerName -scriptblock $scriptblock -argumentList (Get-BcContainerPath -containerName $containerName -path $destname);First is fast. It seems like there is some kind of cache after performing a file operation inside a hyperv session, which flushes after 20 seconds. If I cannot figure out myself what this is, maybe I know somebody who can. |
|
It becomes even more crazy - just tried a longshot $scriptblock = { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
$content = Get-Content -Path $tmpfile
Write-Host "Content: $($content)"
Remove-Item -Path $tmpfile -Force
Set-Content -Path 'c:\programdata\bccontainerhelper\$($tmpfile.GetHashCode()).txt' -Value $content
}removes the problem... It's like creating a file in the folder where the file was removed flushes some kind of cache. This: $scriptblock = { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
$content = Get-Content -Path $tmpfile
Write-Host "Content: $($content)"
powershell "Remove-Item -Path '$tmpfile' -Force"
}Also removes the problem (using powershell 5 to delete the file) $scriptblock = { Param($tmpfile)
Write-Host "WAITED $((Measure-Command { while (!(Test-Path -Path $tmpfile)) { Start-Sleep -Milliseconds 100 } }).TotalSeconds) SECONDS FOR FILE";
$content = Get-Content -Path $tmpfile
Write-Host "Content: $($content)"
[System.IO.File]::Delete($tmpfile)
}also removes the problem But... - Run-AlCops doesn't really remove any files though - so there are probably other things that makes this fail. Will discuss this with some people who might know what is going on. |
|
I will have to create a repro without any kind of BcContainerHelper - a clean WindowsServerCore container - install powershell 7, repro the issue and file a bug here: https://github.com/PowerShell/PowerShell |
|
Thank you very much for that much help and assistance. A really special issue. |
Hi Freddy, it's me again :)
Describe the issue
Run-ALCops (6.0.19-preview1213) stops working with missing file in BC24, BC23 works.
When running Run-ALCops on BC24 (tested .1 and .2) on Windows PS or on PowerShell 7.4.2, reading of the authenticode signature fails for the second or third app. When debugging, the behavior might be different and might work.
I can provide our apps per PM if needed.
Updated Run-ALCops.ps1 with debugging
Script to get the error with BC24.1 (BC23.4 works), Remove-BCContainer has been commented out
Full output of script RunAlCops234
Full output of script RunAlCops241
Additional context
If the error occurs, the file exists before Invoke-ScriptInBcContainer but not in the container/scriptblock. Get-AuthenticodeSignature fails with file not found.
The text was updated successfully, but these errors were encountered: