Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Editing asset cache output when using x-script #1541

Merged
merged 5 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
throw "Script download error"
22 changes: 22 additions & 0 deletions azure-pipelines/e2e-assets/asset-caching/success-script.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
param (
[string]$Url,
[string]$ExpectedSHA512,
[string]$Destination
)

# Ensure the destination folder exists
$downloadFolder = Split-Path -Path $Destination -Parent
if (-not (Test-Path $downloadFolder)) {
New-Item -Path $downloadFolder -ItemType Directory | Out-Null
}

# Download the file directly to the specified destination path
Invoke-WebRequest -Uri $Url -OutFile $Destination
JavierMatosD marked this conversation as resolved.
Show resolved Hide resolved

# Confirm that the file exists and is accessible
if (-not (Test-Path $Destination)) {
Throw "Download failed: File not found at $Destination"
exit 1
}

exit 0
28 changes: 28 additions & 0 deletions azure-pipelines/end-to-end-tests-dir/asset-caching.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -197,3 +197,31 @@ $actual = Run-VcpkgAndCaptureOutput -TestArgs ($commonArgs + @("x-download", "$d
if (-not ($actual.Contains("Asset cache hit for example3.html; downloaded from: file://$AssetCache"))) {
throw "Failure: azurl (yes), x-block-origin (yes), asset-cache (hit), download (n/a)"
}

# Testing x-download failure with asset cache (x-script) and x-block-origin settings
$env:X_VCPKG_ASSET_SOURCES = "clear;x-script,pwsh $PSScriptRoot/../e2e-assets/asset-caching/failing-script.ps1 {url} {sha512} {dst};x-block-origin"
$actual = Run-VcpkgAndCaptureOutput -TestArgs ($commonArgs + @("x-download", "$downloadsRoot/example3.html", "--url", "https://example.com", "--sha512", "d06b93c883f8126a04589937a884032df031b05518eed9d433efb6447834df2596aebd500d69b8283e5702d988ed49655ae654c1683c7a4ae58bfa6b92f2b73a"))
if (-not ($actual.Contains("The script download failed.") -and
$actual.Contains("error: <mirror-script> failed with exit code: (1).") -and
$actual.Contains("error: Missing example3.html and downloads are blocked by x-block-origin."))) {
throw "Failure: x-script downloads failure messaging"
}

# Testing x-download success with asset cache (x-script) and x-block-origin settings
Refresh-TestRoot
$env:X_VCPKG_ASSET_SOURCES = "clear;x-script,pwsh $PSScriptRoot/../e2e-assets/asset-caching/success-script.ps1 {url} {sha512} {dst};x-block-origin"
$actual = Run-VcpkgAndCaptureOutput -TestArgs ($commonArgs + @("x-download", "$downloadsRoot/example3.html", "--url", "https://example.com", "--sha512", "d06b93c883f8126a04589937a884032df031b05518eed9d433efb6447834df2596aebd500d69b8283e5702d988ed49655ae654c1683c7a4ae58bfa6b92f2b73a"))
if (-not ($actual.Contains("Successfully downloaded example3.html."))) {
throw "Failure: x-script download success message"
}

# Testing x-download failure with mismatched sha
Refresh-TestRoot
$env:X_VCPKG_ASSET_SOURCES = "clear;x-script,pwsh $PSScriptRoot/../e2e-assets/asset-caching/success-script.ps1 {url} {sha512} {dst};x-block-origin"
$actual = Run-VcpkgAndCaptureOutput -TestArgs ($commonArgs + @("x-download", "$downloadsRoot/example3.html", "--url", "https://example.com", "--sha512", "d06b93c883f8126a04589937a884032df031b05518eed9d433efb6447834df2596aebd500d69b8283e5702d988ed49655ae654c1683c7a4ae58bfa6b92f2b73b"))
if (-not ($actual.Contains("error: File does not have the expected hash:") -and
$actual.Contains("url: <mirror-script>") -and
$actual.Contains("Expected hash: d06b93c883f8126a04589937a884032df031b05518eed9d433efb6447834df2596aebd500d69b8283e5702d988ed49655ae654c1683c7a4ae58bfa6b92f2b73b") -and
$actual.Contains("Actual hash: d06b93c883f8126a04589937a884032df031b05518eed9d433efb6447834df2596aebd500d69b8283e5702d988ed49655ae654c1683c7a4ae58bfa6b92f2b73a"))) {
throw "Failure: x-script download failure message - sha mismatch"
}
1 change: 1 addition & 0 deletions include/vcpkg/base/message-data.inc.h
Original file line number Diff line number Diff line change
Expand Up @@ -1144,6 +1144,7 @@ DECLARE_MESSAGE(ErrorRequirePackagesList,
(),
"",
"`vcpkg install` requires a list of packages to install in classic mode.")
DECLARE_MESSAGE(ErrorScriptDownloadFailed, (msg::error_msg), "", "The script download failed.\n{error_msg}.")
DECLARE_MESSAGE(ErrorInvalidExtractOption,
(msg::option, msg::value),
"The keyword 'AUTO' should not be localized",
Expand Down
2 changes: 2 additions & 0 deletions locales/messages.json
Original file line number Diff line number Diff line change
Expand Up @@ -663,6 +663,8 @@
"_ErrorParsingBinaryParagraph.comment": "An example of {spec} is zlib:x64-windows.",
"ErrorRequireBaseline": "this vcpkg instance requires a manifest with a specified baseline in order to interact with ports. Please add 'builtin-baseline' to the manifest or add a 'vcpkg-configuration.json' that redefines the default registry.",
"ErrorRequirePackagesList": "`vcpkg install` requires a list of packages to install in classic mode.",
"ErrorScriptDownloadFailed": "The script download failed.\n{error_msg}.",
JavierMatosD marked this conversation as resolved.
Show resolved Hide resolved
"_ErrorScriptDownloadFailed.comment": "An example of {error_msg} is File Not Found.",
"ErrorUnableToDetectCompilerInfo": "vcpkg was unable to detect the active compiler's information. See above for the CMake failure output.",
"_ErrorUnableToDetectCompilerInfo.comment": "failure output will be displayed at the top of this",
"ErrorVcvarsUnsupported": "in triplet {triplet}: Use of Visual Studio's Developer Prompt is unsupported on non-Windows hosts.\nDefine 'VCPKG_CMAKE_SYSTEM_NAME' or 'VCPKG_CHAINLOAD_TOOLCHAIN_FILE' in the triplet file.",
Expand Down
7 changes: 4 additions & 3 deletions src/vcpkg/base/downloads.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -974,6 +974,7 @@ namespace vcpkg
RedirectedProcessLaunchSettings settings;
settings.environment = get_clean_environment();
settings.echo_in_debug = EchoInDebug::Show;

auto maybe_res = flatten(cmd_execute_and_capture_output(cmd, settings), "<mirror-script>");
if (maybe_res)
{
Expand All @@ -982,14 +983,14 @@ namespace vcpkg
if (maybe_success)
{
fs.rename(download_path_part_path, download_path, VCPKG_LINE_INFO);
msg::println(msgDownloadSuccesful, msg::path = download_path.filename());
return urls[0];
}

errors.push_back(std::move(maybe_success).error());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is being fairly careful to put the errors in errors rather than emitting them here directly; can you explain why this case is different than the others?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this error occurs when there is a sha mismatch. I'm printing rather than putting in errors so that the order of the failure messages makes sense and is consistent with the download failure messages below.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I would really like to see at least a comment here explaining what the criteria for 'errors' vs being printed directly is, since changing which is which on and ad-hoc basis like this seems likely to create as many bugs as solved every time it is touched.

However, this function is already awful about that so not going to block over this.

msg::println_error(maybe_success.error());
}
else
{
errors.push_back(std::move(maybe_res).error());
msg::println(msgErrorScriptDownloadFailed, msg::error_msg = maybe_res.error());
}
}
}
Expand Down
Loading