MSYS2 scripted Install, GHA, etc

[MSP-Greg]

I'm not sure where to post this. The issue revolves around scripting MSYS2 installation and update.

Some other recent issues (msys2/MSYS2-packages#1962, msys2/MSYS2-packages#1965) have touched on the issue.

I assume most of the breakage will be caused by commits to https://github.com/msys2/MSYS2-packages.

GitHub Actions (GHA) now has MSYS2 installed on their Windows images. Both 64 & 32 bit clang cmake llvm toolchain ragel packages/groups are installed. Custom Actions have also been developed to install/update MSYS2. Of concern is that the script GHA is using will reliably update MSYS2. Currently, the images are updated frequently, typically every 10 days or better.

Also mentioned is moving some CI to GHA. I think there are several people using MSYS2 that would be willing to help. GHA supports jobs that conditionally run based on file locations, so specific jobs could be ran if, for instance, gpg, msys2-runtime or pacman files were changed.

I have mostly worked with MinGW build scripts (and not a lot), so I'm not familiar with the MSYS2 code, but I'd certainly be willing to help with GHA workflows if they will help with CI.

Also, GHA isn't the only CI, so a 'recommended' script would be helpful.

This is a very general issue, but making MSYS2 robust as far as scripted install/update is important.

[MSP-Greg]

Thinking back on this, the last time I recall issues with MSYS2, and the issues listed above -

Package A and package B get updated. Doing a normal update (via -Syu) installs A first, then B. But, due to whatever, B must be installed first, then A, as A requires the new B?

Not sure if that describes one possible type of problem, but if it does, that might be messy to check with CI?

[lazka]

So, we need

• to use GHA in our own CI (in addition to appveyor, maybe as a replacement for azure)
• a recommended way on how to handle setup/update/install in CI environments, somewhere on our website

[eine]

@lazka, I've been enhancing and maintaining eine/setup-msys2 for the last months. It's a GitHub Action for users to easily setup MSYS2 in their workflows.

Since GitHub added a default MSYS2 installation to windows-latest virtual environments some days ago, Action setup-msys2 allows to either reuse that, or to retrieve and extract a clean one from https://github.com/msys2/msys2-installer/releases.

The repository includes a workflow to test multiple (+15) setups: https://github.com/eine/setup-msys2/blob/master/.github/workflows/action.yml. For examples of other external projects using it (ref msys2/setup-msys2#20):

• https://github.com/leanprover/lean4/blob/master/.github/workflows/ci.yml#L54-L58
• https://github.com/substratic/crash-the-stack/blob/master/.github/workflows/ci.yml#L19-L24
• https://github.com/umarcor/ghdl/blob/master/.github/workflows/push.yml#L71-L83

---

Hence, I'd like to propose forking setup-msys2 or moving it to this organization:

• Use it for building/testing packages in GitHub Actions.
  • Note there is a branch named msys-nightly.
  • Hopefully deploy them automatically after having them tested.
• Hopefully improve or replace msys2.cmd.
• Hopefully extend it to handle other use cases. For example:
  • Move/copy the install script for Windows 10 from actions/virtual-environments.
  • Generalize it or add other similar scripts for GitLab, Travis, AppVeyor, etc.
• Hopefully improve it so that updating doesn't take 5-10min.

[lazka]

Hence, I'd like to propose forking setup-msys2 or moving it to this organization:

Sounds good to me (without knowing the differences between the repos). @Ecco any thoughts on this?

[Ecco]

Hi @lazka ! We'd be glad to transfer ownership of our numworks/setup-msys2 repo to you guys 😄

[lazka]

Hi @lazka ! We'd be glad to transfer ownership of our numworks/setup-msys2 repo to you guys 😄

Any thoughts on using @eine 's fork instead? -> https://github.com/eine/setup-msys2/blob/master/CHANGELOG.md

[eine]

@lazka, as a matter of fact, >65% of changes and 50% of the actual code in numworks/setup-msys2 were contributed from eine/setup-msys2 through PRs. The reason for eine/setup-msys2 to be 26 commits ahead of numworks' is that the latter has been kind of stalled for the last 5 months.

Anyway, I'm ok with having numworks/setup-msys2 transferred and then proposing PRs to bring it up to date. Specially, because numworks/setup-msys2 contains more issues/PRs.

[eine]

@lazka, I think that the "automatic cleanup" of intermediate updates is not working as expected (ref msys2/MSYS2-packages#1962 (comment)). As you can see in https://github.com/eine/setup-msys2/runs/743849758?check_suite_focus=true#step:4:182, :: To complete this update all MSYS2 processes including this terminal will be closed. Confirm to proceed [Y/n] warning: terminating MSYS2 processes failed is shown and the return code is !=0. I worked around it with msys2/setup-msys2@258a810, but I thought that you'd want to be aware.

EDIT

Maybe it should be already fixed by msys2/MSYS2-packages#2002?

[lazka]

Maybe it should be already fixed by msys2/MSYS2-packages#2002?

could be, but unclear. I'll watch out if I see it after this.

[eine]

@lazka, @Ecco, shall we migrate numworks/setup-msys2 or eine/setup-msys2?

[lazka]

@eine feel free to transfer eine/setup-msys2 to the msys2 org. I'll make you both admins there in case that isn't already the case then.

[eine]

@lazka, I'm afraid I cannot, because I'm not allowed to create repos in this org. I invited you to the repo, so you can hopefully trigger the transfer.

[lazka]

hm, right, I'm not allowed either. Can I get admin rights maybe?

[eine]

It seems not possible to give admin rights in "user repos". All I could do was add you as a collaborator, which seems not enough: https://help.github.com/en/github/setting-up-and-managing-your-github-user-account/permission-levels-for-a-user-account-repository

So, I "requested" the transfer to your personal account. Hopefully, once it's lazka/setup-msys2, you will be allowed to transfer it to the org.

[lazka]

thanks, done

[eine]

Nice! I updated the readme to replace all the references from 'eine/setup-msys2' to 'msys2/setup-msys2' (README.md).

[lazka]

I've contacted github to remove the "forked from" status (and opened a few PRs ;) )

[eine]

Oh, I was wondering why it was gone :D. All merged now. Thanks.

[lazka]

So the only thing left imo is that we document what we do in msys2/setup-msys2 on the website.

[eine]

Should it be a section in https://github.com/msys2/msys2.github.io/blob/source/docs/wiki/MSYS2-installation.md?

[lazka]

I was more thinking about a dedicated "CI" section with

• GHA
• Appveyor
• Generic instructions for all other systems

[lazka]

Here is a first draft: https://www.msys2.org/docs/ci/

[lazka]

Is there anything left to do here? I'd suggest to open separate actionable issues if there is.

[eine]

According to the OP's comment above:

Also, GHA isn't the only CI, so a 'recommended' script would be helpful.

This is a very general issue, but making MSYS2 robust as far as scripted install/update is important.

Most of the current setup-msys2 Action can be rewritten from JavaScript to Powershell. That would allow to use the same script for the GitHub virtual-environment, the GitHub Action, or for users to run it on other services. @MSP-Greg are you up to integrating https://github.com/actions/virtual-environments/blob/master/images/win/scripts/Installers/Install-Msys2.ps1 into setup-msys2?

[MSP-Greg]

@eine

So the goal is a PowerShell script?

Re https://www.msys2.org/docs/ci/, when MSYS2 is pre-installed, I tend to prefer updating only what is needed, as opposed to a full update. Especially with Action, which is updating frequently, unlike other CI services that often went months between updates...

[eine]

The main entrypoint of a GitHub Action must be a JavaScript script. However, it is possible to use it for executing a powershell script only; i.e. as an intermediate argument parsing layer only. For example, currently we use powershell to compute the checksum of the installer: https://github.com/msys2/setup-msys2/blob/master/main.js#L51

Hence, yes, the goal would be to migrate part of the logic that is currently written in JS to powershell, so it is usable outside of GitHub Actions.

Re msys2.org/docs/ci, when MSYS2 is pre-installed, I tend to prefer updating only what is needed, as opposed to a full update. Especially with Action, which is updating frequently, unlike other CI services that often went months between updates...

With setup-msys2, updating any package (including the base set of packages) is optional. Any proposed script should be feature-equivalent to the current setup-msys2.

[lazka]

Re msys2.org/docs/ci, when MSYS2 is pre-installed, I tend to prefer updating only what is needed, as opposed to a full update.

Partial updates are not supported, same as with Arch Linux.

[lazka]

I think there is nothing left to do here. I I missed something maybe copy and create a new issue.

[MSP-Greg]

@lazka

I apologize for leaving this open. Thanks.

[lazka]

I don't mind open issues :)