Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grpc: Retry CMake configuration if it fails #23153

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

js-nano
Copy link

@js-nano js-nano commented Mar 18, 2024

The gRPC CMake configuration step can fail to due to a failure when downloading third party archives. These failures are expected, and the user is instructed to retry. So, if configuration fails, retry it

Also set the gRPC_DOWNLOAD_ARCHIVES CMake flag to False, to avoid downloading these unnecessary archives in the first place, in versions of gRPC that understand the flag

Specify library name and version: grpc/all

This fixes #23094


@conan-center-bot

This comment has been minimized.

The gRPC CMake configuration step can fail to due to a failure when downloading third party archives.  These failures are expected, and the user is instructed to retry.  So, if configuration fails, retry it

Also set the `gRPC_DOWNLOAD_ARCHIVES` CMake flag to `False`, to avoid downloading these unnecessary archives in the first place, in versions of gRPC that understand the flag
@conan-center-bot

This comment has been minimized.

@conan-center-bot conan-center-bot added Failed Missing dependencies Build failed due missing dependencies in Conan Center and removed Service Under Maintenance labels Jun 12, 2024
@conan-center-bot
Copy link
Collaborator

Conan v1 pipeline ✔️

All green in build 4 (0ac619c39a7e119efdfa3a75546aefee41f4113c):

  • grpc/1.50.0:
    All packages built successfully! (All logs)

  • grpc/1.54.3:
    All packages built successfully! (All logs)

  • grpc/1.50.1:
    All packages built successfully! (All logs)

  • grpc/1.48.4:
    All packages built successfully! (All logs)


Conan v2 pipeline ❌

Note: Conan v2 builds are now mandatory. Please read our discussion about it.

The v2 pipeline failed. Please, review the errors and note this is required for pull requests to be merged. In case this recipe is still not ported to Conan 2.x, please, ping @conan-io/barbarians on the PR and we will help you.

See details:

Failure in build 5 (0ac619c39a7e119efdfa3a75546aefee41f4113c):

  • grpc/1.50.1:
    Didn't run or was cancelled before finishing

  • grpc/1.50.0:
    Didn't run or was cancelled before finishing

  • grpc/1.54.3:
    CI failed to create some packages (All logs)

    Logs for packageID 2cf586938382402f8da34d74e46850ff4e56c618:
    [settings]
    arch=x86_64
    build_type=Release
    compiler=gcc
    compiler.cppstd=17
    compiler.libcxx=libstdc++11
    compiler.version=11
    os=Linux
    [options]
    */*:shared=True
    
    [...]
        protobuf/3.21.12#2070373d5f1a2606a4491faef9315a73:10670e90b03eb1607902ed190ae2fecdc52d69bb#eff9a8c0282afb8c3fb04152953185c8 - Download (conan-center)
    Skipped binaries
        abseil/20230125.3, autoconf/2.71, automake/1.16.5, bzip2/1.0.8, c-ares/1.19.1, flex/2.6.4, gnu-config/cci.20210814, gperf/3.1, libcap/2.69, libmount/2.39.2, libselinux/3.6, libsystemd/255, libtool/2.4.7, libxcrypt/4.4.36, lz4/1.9.4, m4/1.4.19, meson/1.2.2, meson/1.4.0, ninja/1.11.1, openssl/3.2.2, pcre2/10.42, pkgconf/2.1.0, pkgconf/2.2.0, re2/20230301, xz_utils/5.4.5, zlib/1.3.1, zstd/1.5.5
    ERROR: Missing binary: grpc/1.54.3:8cc2949c5b31e1cbd41f8dcef63fdab0744e62fa
    
    grpc/1.54.3: WARN: Can't find a 'grpc/1.54.3' package binary '8cc2949c5b31e1cbd41f8dcef63fdab0744e62fa' for the configuration:
    [settings]
    arch=x86_64
    build_type=Release
    compiler=gcc
    compiler.cppstd=17
    compiler.libcxx=libstdc++11
    compiler.version=11
    os=Linux
    [options]
    codegen=True
    cpp_plugin=True
    csharp_ext=False
    csharp_plugin=True
    fPIC=True
    node_plugin=True
    objective_c_plugin=True
    php_plugin=True
    python_plugin=True
    ruby_plugin=True
    shared=False
    [requires]
    abseil/20230125.3.Z
    bzip2/1.0.Z
    c-ares/1.19.Z
    libcap/2.69.Z
    libmount/2.39.Z
    libselinux/3.6.Z
    libsystemd/255.0.Z
    libxcrypt/4.4.Z
    lz4/1.9.Z
    openssl/3.2.Z
    pcre2/10.42.Z
    protobuf/3.21.Z
    re2/20230301.0.Z
    xz_utils/5.4.Z
    zlib/1.3.Z
    zstd/1.5.Z
    
    ERROR: Missing prebuilt package for 'grpc/1.54.3'. You can try:
        - List all available packages using 'conan list grpc/1.54.3:* -r=remote'
        - Explain missing binaries: replace 'conan install ...' with 'conan graph explain ...'
        - This is a **test_package** missing binary. You can use --build (for all dependencies) or --build-test (exclusive for 'test_package' dependencies) to define what can be built from sources
    
    More Info at 'https://docs.conan.io/2/knowledge/faq.html#error-missing-prebuilt-package'
    
  • grpc/1.48.4:
    Didn't run or was cancelled before finishing


Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

@@ -174,6 +174,11 @@ def generate(self):
tc.cache_variables["gRPC_BUILD_GRPC_PYTHON_PLUGIN"] = self.options.python_plugin
tc.cache_variables["gRPC_BUILD_GRPC_RUBY_PLUGIN"] = self.options.ruby_plugin

# Never download unnecessary archives
# (supported in gRPC >= 1.62.0)
tc.cache_variables["gRPC_DOWNLOAD_ARCHIVES"] = False
Copy link
Contributor

@samuel-emrys samuel-emrys Jun 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not particularly familiar with gRPC - so to confirm, these archives aren't required? I agree that a conan recipe should never attempt to download external files, but if they are required then they should be added to conandata.yml and downloaded/extracted in the source() method.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the issue where the flag was introduced, it seems that they're not required for anything directly related to building or testing gRPC
grpc/grpc#34587 (comment)

In addition, for all but one of the archives, no attempt is made to download the archives -- the CMake script checks for the presence of directories that are (in most cases) already present in the Conan source checkout.

@samuel-emrys
Copy link
Contributor

@RubenRBS looks like the v2 pipeline is failing on a weird error - it can't find itself as a dependency?

@samuel-emrys
Copy link
Contributor

@js-nano from @RubenRBS on slack, to get the conan 2 pipeline passing you'll need to simplify the test_package so that self.tested_reference_str is only requested as a requires(), and not a tool_requires(), i.e.

    def build_requirements(self):
        if not self._is_legacy_one_profile:
            # For the grpc-cpp-plugin executable at build time
-           self.tool_requires(self.tested_reference_str)
            self.tool_requires("protobuf/<host_version>")

This is due to a limitation in the existing CI infrastructure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Failed Missing dependencies Build failed due missing dependencies in Conan Center
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[package] grpc/all: configuration is designed to possibly fail, but succeed on a retry
4 participants