Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Windows] Removing $env:SystemRoot\Installer\* causes installer and uninstaller issues #11119

Closed
3 of 15 tasks
AjkayAlan opened this issue Dec 6, 2024 · 9 comments
Closed
3 of 15 tasks

Comments

@AjkayAlan
Copy link

Description

Windows 20241201 images introduced more aggressive cleanup behavior as part of the build process. One of the additions to the logic was to delete $env:SystemRoot\Installer\* per (https://github.com/actions/runner-images/blob/main/images/windows/scripts/build/Invoke-Cleanup.ps1#L18)

This causes issues for anyone working with upgrading or uninstalling the existing installed packages on the image that are installed as MSI's - as now existing packages installed as MSI's have broke installers/uninstallers and throw nasty errors when trying to interact with them.

For instance, if I try to clean up NodeJS installations before installing nvm-windows (as nvm-windows gets somewhat upset if any NodeJS installs exist when installing) via a command like the following in PowerShell: Get-Package | Where Name -Match "^Node" | Uninstall-Package, I now get an InvalidOperation error:

image

Microsoft themselves say its not a good idea to delete files in this directory per https://blogs.technet.microsoft.com/joscon/2012/01/18/can-you-safely-delete-files-in-the-windirinstaller-directory/. My expectation is that others using Windows runners may have started encountering issues with their tools if they did similar things, such as #11095

Can this specific change be removed?

Platforms affected

  • Azure DevOps
  • GitHub Actions - Standard Runners
  • GitHub Actions - Larger Runners

Runner images affected

  • Ubuntu 20.04
  • Ubuntu 22.04
  • Ubuntu 24.04
  • macOS 12
  • macOS 13
  • macOS 13 Arm64
  • macOS 14
  • macOS 14 Arm64
  • macOS 15
  • macOS 15 Arm64
  • Windows Server 2019
  • Windows Server 2022

Image version and build link

Image: windows-2022
Version: 20241201.2.0
Included Software: https://github.com/actions/runner-images/blob/win22/20241201.2/images/windows/Windows2022-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/win22%2F20241201.2

Is it regression?

Yes. windows-2022 version 20241125.1.0 worked as expected

Expected behavior

Software installed as MSI's should be able to be uninstalled/updated without error.

Actual behavior

When attempting to modify or uninstall software on the image that was installed as an MSI, errors are thrown.

Repro steps

  1. Create a GitHub Action using windows-2022, version 20241201.2.0
  2. Add a step with the following code: Get-Package | Where Name -Match "^Node" | Uninstall-Package

Here is an example github action:

name: Example Workflow
on:
  pull_request:
    branches:
      - main
  workflow_dispatch:

permissions:
  contents: read

jobs:
  example_job:
    name: Example Job
    runs-on: windows-latest
    defaults:
      run:
        shell: powershell # This defaults to PowerShell 5. `pwsh` goes to 7.

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Cleanup Node
        run: |
          Get-Package | Where Name -Match "^Node" | Uninstall-Package
@kishorekumar-anchala
Copy link
Contributor

Hi @AjkayAlan , Working on your issue , we will update you ASAP. thanks for your support !

@kishorekumar-anchala
Copy link
Contributor

Hi @AjkayAlan ,

I request you please try to use pwsh to install/ uninstall packages , it will fix the problem . thank you !

defaults:
      run:
        shell: pwsh

@AjkayAlan
Copy link
Author

AjkayAlan commented Dec 10, 2024

Hi @kishorekumar-anchala - Unfortunately I don't believe that is working for the example I provided. Instead, it simply isn't doing anything.

Here is an example workflow using pwsh:

name: Example Workflow
on:
  pull_request:
    branches:
      - main
  workflow_dispatch:

permissions:
  contents: read

jobs:
  example_job:
    name: Example Job
    runs-on: windows-latest
    defaults:
      run:
        shell: pwsh

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Get Packages Before Uninstall
        run: |
          Get-Package | Where Name -Match "^Node"

      - name: Cleanup Node
        run: |
          Get-Package | Where Name -Match "^Node" | Uninstall-Package

      - name: Get Packages After Uninstall
        run: |
          Get-Package | Where Name -Match "^Node"

When running that workflow, it "passes" but doesn't actually do anything. Here is a raw log output excerpt - you can see that the Get-Package commands aren't even returning anything.

2024-12-10T14:36:07.7997985Z ##[group]Run Get-Package | Where Name -Match "^Node"
2024-12-10T14:36:07.7998446Z �[36;1mGet-Package | Where Name -Match "^Node"�[0m
2024-12-10T14:36:07.8034129Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
2024-12-10T14:36:07.8034471Z ##[endgroup]
2024-12-10T14:36:31.2234927Z ##[group]Run Get-Package | Where Name -Match "^Node" | Uninstall-Package
2024-12-10T14:36:31.2235850Z �[36;1mGet-Package | Where Name -Match "^Node" | Uninstall-Package�[0m
2024-12-10T14:36:31.2278328Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
2024-12-10T14:36:31.2278650Z ##[endgroup]
2024-12-10T14:36:33.1394968Z ##[group]Run Get-Package | Where Name -Match "^Node"
2024-12-10T14:36:33.1395404Z �[36;1mGet-Package | Where Name -Match "^Node"�[0m
2024-12-10T14:36:33.1426452Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
2024-12-10T14:36:33.1426780Z ##[endgroup]
2024-12-10T14:36:34.9833445Z Post job cleanup.

If I switch to PowerShell 5, my Get-Package does output things correctly, but fails due to the root cause of this ticket (which I believe was the change to remove all entries in $env:SystemRoot\Installer\* as identified in the original report):

2024-12-10T14:38:27.7667736Z ##[group]Run Get-Package | Where Name -Match "^Node"
2024-12-10T14:38:27.7668246Z �[36;1mGet-Package | Where Name -Match "^Node"�[0m
2024-12-10T14:38:27.7716869Z shell: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.EXE -command ". '{0}'"
2024-12-10T14:38:27.7717327Z ##[endgroup]
2024-12-10T14:38:49.5906079Z 
2024-12-10T14:38:49.5907881Z Name                           Version          Source                           ProviderName                          
2024-12-10T14:38:49.5908856Z ----                           -------          ------                           ------------                          
2024-12-10T14:38:49.5910165Z Node.js                        18.20.5                                           msi                                   
2024-12-10T14:38:49.5944315Z 
2024-12-10T14:38:49.5944560Z 
2024-12-10T14:38:49.6252473Z ##[group]Run Get-Package | Where Name -Match "^Node" | Uninstall-Package
2024-12-10T14:38:49.6253036Z �[36;1mGet-Package | Where Name -Match "^Node" | Uninstall-Package�[0m
2024-12-10T14:38:49.6280614Z shell: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.EXE -command ". '{0}'"
2024-12-10T14:38:49.6280998Z ##[endgroup]
2024-12-10T14:38:53.3588815Z Uninstall-Package : Uninstallation operation failed. Please look at 
2024-12-10T14:38:53.3590806Z 'C:\Users\runneradmin\AppData\Local\Temp\033ea20a-3ae2-4a7d-a735-cbaa88be5e5b\msi.log' file for details.
2024-12-10T14:38:53.3591976Z At D:\a\_temp\c1ed9a28-b5a7-48d0-91ca-dc55a50dafe1.ps1:2 char:43
2024-12-10T14:38:53.3592804Z + Get-Package | Where Name -Match "^Node" | Uninstall-Package
2024-12-10T14:38:53.3593842Z +                                           ~~~~~~~~~~~~~~~~~
2024-12-10T14:38:53.3594685Z     + CategoryInfo          : InvalidOperation: ({CD45BC04-FEE6-41E9-A979-1632B8F82758}:String) [Uninstall-Package], E 
2024-12-10T14:38:53.3597186Z    xception
2024-12-10T14:38:53.3597883Z     + FullyQualifiedErrorId : Uninstallation operation failed. Please look at '{0}' file for details.,Microsoft.PowerS 
2024-12-10T14:38:53.3598828Z    hell.PackageManagement.Cmdlets.UninstallPackage
2024-12-10T14:38:53.3599314Z  
2024-12-10T14:38:53.3808840Z ##[error]Process completed with exit code 1.
2024-12-10T14:38:53.3977167Z Post job cleanup.

So - to summarize, switching to PowerShell 7 (pwsh) does NOT solve the issue.

@Alexey-Ayupov
Copy link
Contributor

Hi @AjkayAlan, Sorry for introducing such a breaking change. We are going to revert it in windows-2019 and windows-2022 images, but for now we will leave it for windows-2025 image.

This was a necessary step to make the windows-2025 image less than 150GB. We assumed that no one was deleting packages during execution. We'll look at how to keep enough free disk space without deleting the contents of the $env:SystemRoot\Installer directory.

@AjkayAlan
Copy link
Author

@Alexey-Ayupov - No worries, and I totally get it! Thank you for reverting that change for the existing runner images (assuming that is the root cause). IMO making that a breaking change for the Windows 2025 images makes sense, and gives people a chance to "opt in" to that change so they can figure out better ways to do things.

I will agree, we are definitely stretching the use cases of how we use the runner images, in part that for us its so easy to inherit all the goodness and hard work that you all have done with the images, and then just change our small pieces as necessary.

Thanks again for being open to reverting that change - and if there is anything I can do to help, please let me know.

@kishorekumar-anchala
Copy link
Contributor

kishorekumar-anchala commented Dec 11, 2024

HI @AjkayAlan ,
Could you please check and confirm if the issue still persist . Thank you @Alexey-Ayupov .

@AjkayAlan
Copy link
Author

Hi @kishorekumar-anchala! Unfortunately I will need to wait until there is a tag cut and the image is updated on GitHub Actions (as I am using the github hosted runners running these images). That, or is there another way to get the updated image in GitHub Actions for a beta test?

@kishorekumar-anchala kishorekumar-anchala added the awaiting-deployment Code complete; awaiting deployment and/or deployment in progress label Dec 12, 2024
@AjkayAlan
Copy link
Author

@kishorekumar-anchala looks like the deployment completed and I can confirm that the issue is now fixed with:

Image: windows-2022
Version: 20241211.1.0

Thank you again! This can be marked as resolved!

@kishorekumar-anchala
Copy link
Contributor

thankyou @AjkayAlan for your confirmation.

@kishorekumar-anchala kishorekumar-anchala removed the awaiting-deployment Code complete; awaiting deployment and/or deployment in progress label Dec 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants
@AjkayAlan @Alexey-Ayupov @kishorekumar-anchala and others