[Bug] Windows - Install-Msys2.ps1 - remove msys2 p7zip / 7z package #916
Conversation
|
/azp run windows2016, windows2019 |
|
Azure Pipelines successfully started running 2 pipeline(s). |
Due to conflicts with two installed versions
|
Rebased (duh) and added two minor changes. Worked locally... |
|
Reminder Once MSYS2 has been rolled out, the current Windows Rubies can be replaced with standard installs. The 'vendored' code in RubyInstaller2 Rubies will find the MSYS2 install, so there won't be an issues. JFYI, the code looks for an embedded MSYS2 first, then checks C:/msys64. In the most recent release: The current download, rubyinstaller-devkit-2.7.1-1-x64.exe, is 131 MB, while the standard install, rubyinstaller-2.7.1-1-x64.7z, is 11 MB. Given that there are four Rubies installed (2.4 thru 2.7), that should free up a lot of disk space, and remove a lot of duplicated files. I'd do a PR for it, but I can't find the code... EDIT: Thanks to everyone helping with adding mSYS2. 0200517.1 is now available, and a few Ruby repos are working fine with ruby/setup-ruby and MSP-Greg/setup-ruby-pkgs |
|
@MSP-Greg , I think we don't need to remove p7zip to unblock only one particular customer case. Let's figure out how many apps can raise the same issue in the We can easily reproduce the issue for example with
Normal behavior:
As I know you provided the request to change |
|
Everything you said is true. I thought an exception for p7zip was warranted because:
With Windows CI, there's always going to be issues with GNU dev/build tools. Or, I think 7z is in a different category than things like gcc, make, openssl, etc... My POV is also biased from helping macOS/nix types get Windows CI working, which probably shows in how ruby/setup-ruby works with Windows. Long story short, if you want to close, that's ok by me... |
|
The What causes that? Why does IIRC it seems most software already has to care to use Since GitHub Actions is adding a MSYS2 installation, I would assume adding it to PATH should work reasonably well. If basic tools like |
|
The problem is using the bash shell, which is from Git, it's not the MSYS2 bash shell. See actions/runner#497 |
|
Because the Git Bash does some initialization that MSYS2 binaries don't like, or conversely those binaries need some setup that MSYS2 bash does? Why don't we see this issue with the RubyInstaller DevKit (+ Git bash) but we see them with GitHub Actions MSYS2? Simply because it has more packages? |
|
Do you know of any repos using Puma, Ruby, RubyGems, & ruby-loco are all working fine? I had to update setup-ruby-pkgs because of a GPG key issue (ruby 2.4 OpenSSL 1.0.2), and it builds the OpenSSL gem as a test, also ok... |
|
BTW:
|
Note that MSYS2 provides three different shells: MSYS, MINGW64 and MINGW32. Replacing
Ref #908. MSYS2 provides a built-in mechanism to either inherit the PATH or set a new "clean" env for the shell. For reference, |
|
@al-cheb, see also actions/toolkit#318. |
|
I made changes to the script and ran it on Actions. Tried to bullet-proof it, changed the output a bit, including the logging of packages installed. I can:
|
|
/azp run windows2016, windows2019 |
|
Azure Pipelines successfully started running 2 pipeline(s). |
| pacman --noconfirm -Sy pacman | ||
| pacman --noconfirm -Su | ||
| Write-Host "pacman -Sy --noconfirm --needed pacman" | ||
| pacman -Sy --noconfirm --needed pacman |
There was a problem hiding this comment.
@miketimofeev, @al-cheb, as discussed in #906 (comment), pacman should NOT be updated first.
There was a problem hiding this comment.
@eine, Thank you for providing details. I think we can include these changes to the PR. Or if we don't want to mix changes. I can prepare a separate PR.
I've got changes that I can push as a separate commit here, the changes I'm referring to were in the CI I mentioned above, see See Actions step here. The file is at https://github.com/MSP-Greg/actions-msys2-pkgs/blob/master/Install-Msys2.ps1#L50-L55, highlighted are the changes /additions, along with multiple Note that additional changes were made to logging, especially the package lists. In general, the main issue here is that MSYS2 got to a state where 'standard update' code did not work (which is fixed in 20200517)). I've been using MSYS2 for several years, and I think this is only the second time that has happened. So, I'm not sure we can be certain that whatever code we use can be guaranteed to install/update MSYS2 at every moment in time in the future. I haven't looked to see if MSYS2 has code checking this, if not, I'll suggest/contribute there. @eine, any recollection of past issues? @al-cheb - Thanks, can you have a look at the links? |
|
@MSP-Greg, Thank you for updates. I have run the Instal-Msys2.ps1 script on a local machine. It works as expected without pre-installed 7z.
7-zip is not pre-installed: |
|
Should I move the changes here with another commit? Are you ok with the other changes (logging, taskkill, etc)? It's Friday, be nice for all of us if we can resolve this today... |
|
@MSP-Greg , LGTM. I think we can include all changes in this PR. @maxim-lobanov, What do you think? |
@MSP-Greg, overall, I believe that whoever that will take care of maintaining MSYS2 support in this virtual environment, should watch the main repos of the project and should have close contact with the maintainers (through issues, chat or private channels). As you said, updating MSYS2 does NOT fail often, but it is a rolling distribution, and it is expected to have issues more frequently than frozen distributions. This is specially so because the (your) decision was to auto-update the environment by default and to include additional packages, as opposed to providing a simple and stable solution. I already explained in our previous discussions that such an approach would imply complications in the maintenance. To be precise, @lazka is working hard these last weeks, and the only mechanism to be aware of it is to get partially involved: msys2/MSYS2-packages#1962 (comment) |
|
I haven't pushed another commit, but below is patch file: diff --git a/images/win/scripts/Installers/Install-Msys2.ps1 b/images/win/scripts/Installers/Install-Msys2.ps1
index 8c5f859..90971b4 100644
--- a/images/win/scripts/Installers/Install-Msys2.ps1
+++ b/images/win/scripts/Installers/Install-Msys2.ps1
@@ -1,12 +1,14 @@
################################################################################
## File: Install-Msys2.ps1
-## Desc: Install Msys2 and 64-bit gcc, cmake, & llvm (32-bit commented out)
+## Desc: Install Msys2 and 64 & 32 bit gcc, cmake, & llvm
################################################################################
# References
# https://github.com/msys2/MINGW-packages/blob/master/azure-pipelines.yml
# https://packages.msys2.org/group/
+$dash = "—" * 40
+
$origPath = $env:PATH
$gitPath = "$env:ProgramFiles\Git"
@@ -17,7 +19,7 @@ $msys2Uri = ((Invoke-RestMethod $msys2_release).assets | Where-Object {
$msys2File = "$env:TEMP\msys2.tar.xz"
-# Download the latest msys2 x86_64
+# Download the latest msys2 x86_64, filename includes release date
Write-Host "Starting msys2 download using $($msys2Uri.split('/')[-1])"
(New-Object System.Net.WebClient).DownloadFile($msys2Uri, $msys2File)
Write-Host "Finished download"
@@ -31,7 +33,7 @@ $env:PATH = "$gitPath\usr\bin;$gitPath\mingw64\bin;$origPath"
$tar = "$gitPath\usr\bin\tar.exe"
# extract tar.xz to C:\
-Write-Host "Starting msys2 extraction from $msys2FileU"
+Write-Host "Starting msys2 extraction"
&$tar -xJf $msys2FileU -C /c/
Remove-Item $msys2File
Write-Host "Finished extraction"
@@ -39,55 +41,52 @@ Write-Host "Finished extraction"
# Add msys2 bin tools folders to PATH temporary
$env:PATH = "C:\msys64\mingw64\bin;C:\msys64\usr\bin;$origPath"
-Write-Host "bash pacman-key --init"
+Write-Host "`n$dash bash pacman-key --init"
bash.exe -c "pacman-key --init 2>&1"
Write-Host "bash pacman-key --populate msys2"
bash.exe -c "pacman-key --populate msys2 2>&1"
-Write-Host "pacman -Sy --noconfirm --needed pacman"
-pacman -Sy --noconfirm --needed pacman
-pacman -Su --noconfirm
-
-# Force stop gpg-agent to continue installation
-Get-Process gpg-agent -ErrorAction SilentlyContinue | Stop-Process -Force
-
-Write-Host "pacman --noconfirm -Syyuu"
+Write-Host "`n$dash pacman --noconfirm -Syyuu"
pacman.exe -Syyuu --noconfirm
-pacman.exe -Syuu --noconfirm
+taskkill /f /fi "MODULES eq msys-2.0.dll"
+Write-Host "`n$dash pacman --noconfirm -Syuu (2nd pass)"
+pacman.exe -Syuu --noconfirm
+taskkill /f /fi "MODULES eq msys-2.0.dll"
-Write-Host "Install msys2 packages"
+Write-Host "`n$dash Install msys2 packages"
pacman.exe -S --noconfirm --needed --noprogressbar base-devel compression
+taskkill /f /fi "MODULES eq msys-2.0.dll"
-Write-Host "Remove p7zip/7z package due to conflicts"
+Write-Host "`n$dash Remove p7zip/7z package due to conflicts"
pacman.exe -R --noconfirm --noprogressbar p7zip
# mingw package list
$tools = "___clang ___cmake ___llvm ___toolchain ___ragel"
# install mingw64 packages
-Write-Host "Install mingw64 packages"
+Write-Host "`n$dash Install mingw64 packages"
$pre = "mingw-w64-x86_64-"
pacman.exe -S --noconfirm --needed --noprogressbar $tools.replace('___', $pre).split(' ')
# install mingw32 packages
-Write-Host "Install mingw32 packages"
+Write-Host "`n$dash Install mingw32 packages"
$pre = "mingw-w64-i686-"
pacman.exe -S --noconfirm --needed --noprogressbar $tools.replace('___', $pre).split(' ')
# clean all packages to decrease image size
-Write-Host "Clean packages"
+Write-Host "`n$dash Clean packages"
pacman.exe -Scc --noconfirm
-Write-Host "Installed mingw64 packages"
-pacman.exe -Qs --noconfirm mingw-w64-x86_64-
+Write-Host "`n$dash Installed mingw64 packages"
+pacman.exe -Q | grep ^mingw-w64-x86_64-
-Write-Host "Installed mingw32 packages"
-pacman.exe -Qs --noconfirm mingw-w64-i686-
+Write-Host "`n$dash Installed mingw32 packages"
+pacman.exe -Q | grep ^mingw-w64-i686-
-Write-Host "Installed msys2 packages"
-pacman.exe -Qs --noconfirm
+Write-Host "`n$dash Installed msys2 packages"
+pacman.exe -Q | grep -v ^mingw-w64-
-Write-Host "MSYS2 installation completed"
+Write-Host "`nMSYS2 installation completed"
-exit 0
\ No newline at end of file
+exit 0
|
|
Re rolling vs release, etc, etc -- I work a lot with Ruby master. Given that, there is an interest in always using the most recent gcc available. The most recent packages are zst files. Without a full MSYS2 update, those can't even be installed. One example, but there are also security updates (openssl, etc) and other examples. Also, as I've mentioned before, the home page of msys.org shows updating as part of the normal user install process. We disagree on this, but we can certainly work together and benefit. Ok? |
|
@MSP-Greg let's merge this PR for now to include it to next week deployment. |
|
@MSP-Greg, sure! I believe that tight integration between upstream MSYS2, this repo and setup-msys2 (or any similar Action) is required in order to provide a pleasant UX. Since we are users, we'd better work together to benefit all of us! Let's move the discussion about higher-level strategies to msys2/msys2-installer#5. |
|
Just to clarify:
What, me worry? Sorry. No, progress is good, and thanks.
So the patch file above I'll open a new PR? Not sure if it will conflict with this, but I can wait until this is accepted... |
|
Thanks. See #928 |
|
Ref #355 |
Description
New tool, Bug fixing, or Improvement? Bug
Due to conflicts with two installed versions
See #905