Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cuttlefish: add arm64 build #51956

Merged
merged 6 commits into from
Nov 6, 2024
Merged

cuttlefish: add arm64 build #51956

merged 6 commits into from
Nov 6, 2024

Conversation

mencian
Copy link
Contributor

@mencian mencian commented Nov 6, 2024

Describe your pull request here


Please read the guidelines for Bioconda recipes before opening a pull request (PR).

General instructions

  • If this PR adds or updates a recipe, use "Add" or "Update" appropriately as the first word in its title.
  • New recipes not directly relevant to the biological sciences need to be submitted to the conda-forge channel instead of Bioconda.
  • PRs require reviews prior to being merged. Once your PR is passing tests and ready to be merged, please issue the @BiocondaBot please add label command.
  • Please post questions on Gitter or ping @bioconda/core in a comment.

Instructions for avoiding API, ABI, and CLI breakage issues

Conda is able to record and lock (a.k.a. pin) dependency versions used at build time of other recipes.
This way, one can avoid that expectations of a downstream recipe with regards to API, ABI, or CLI are violated by later changes in the recipe.
If not already present in the meta.yaml, make sure to specify run_exports (see here for the rationale and comprehensive explanation).
Add a run_exports section like this:

build:
  run_exports:
    - ...

with ... being one of:

Case run_exports statement
semantic versioning {{ pin_subpackage("myrecipe", max_pin="x") }}
semantic versioning (0.x.x) {{ pin_subpackage("myrecipe", max_pin="x.x") }}
known breakage in minor versions {{ pin_subpackage("myrecipe", max_pin="x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
known breakage in patch versions {{ pin_subpackage("myrecipe", max_pin="x.x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
calendar versioning {{ pin_subpackage("myrecipe", max_pin=None) }}

while replacing "myrecipe" with either name if a name|lower variable is defined in your recipe or with the lowercase name of the package in quotes.

Bot commands for PR management

Please use the following BiocondaBot commands:

Everyone has access to the following BiocondaBot commands, which can be given in a comment:

@BiocondaBot please update Merge the master branch into a PR.
@BiocondaBot please add label Add the please review & merge label.
@BiocondaBot please fetch artifacts Post links to CI-built packages/containers.
You can use this to test packages locally.

Note that the @BiocondaBot please merge command is now depreciated. Please just squash and merge instead.

Also, the bot watches for comments from non-members that include @bioconda/<team> and will automatically re-post them to notify the addressed <team>.

Copy link
Contributor

coderabbitai bot commented Nov 6, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The pull request introduces several changes to the Cuttlefish recipe files, focusing on the build configuration and metadata. The build.sh script has been significantly modified to utilize CMake's -S and -B options for specifying source and build directories. New environment variables for compilation, such as INCLUDES, LIBPATH, LDFLAGS, and CXXFLAGS, have been added, and the script's syntax for checking the operating system has been updated to [[ ... ]]. The CFLAGS variable has been removed from the macOS section, while a new variable, CONFIG_ARGS, has been introduced for conditional CMake configuration options. The meta.yaml file has been updated to increment the build number, add a run_exports section, and remove certain build requirements. Additionally, the about section has been enhanced with a new license_family field, and various fields have been reformatted to include quotes. The file build_failure.osx-64.yaml, which documented a previous build failure, has been deleted.

Possibly related PRs

  • infernal: add osx-arm64 build #50997: The changes in build.sh for the infernal package involve modifications to environment variables and build commands, which may relate to similar changes in the build.sh of the main PR that also focuses on build configurations and environment variables.
  • Update python to >=3.8 in k8 recipe #51054: The k8 recipe's build.sh changes include updates to environment variables and build commands, paralleling the modifications made in the main PR's build.sh for similar purposes.
  • bambamc: add aarch64/arm64 builds #51067: The bambamc recipe's build.sh includes enhancements for ARM architecture, which aligns with the changes made in the cuttlefish build script for improved compatibility.
  • gappa: add arm64 build #51098: The gappa recipe's build.sh updates include permission settings for binaries, which is a similar concern addressed in the cuttlefish build process.
  • gxf2bed: add aarch64/arm64 builds #51120: The gxf2bed recipe's build.sh changes include environment variable handling, which is relevant to the modifications in the cuttlefish build script.
  • lighter: add aarch64/arm64 builds #51136: The lighter recipe's build.sh includes architecture-specific flags, which relate to the changes made in the cuttlefish build script for better compatibility.
  • mashmap: add aarch64 build #51161: The mashmap recipe's build.sh updates include enhancements for macOS compatibility, which is relevant to the changes made in the cuttlefish build script.
  • r-cobrar: add aarch64/arm64 builds #51203: The r-cobrar recipe's meta.yaml updates include platform support, which aligns with the changes in the cuttlefish recipe.
  • Rebuild breseq #51221: The breseq recipe's updates include build number increments and dependency management, which are similar to the changes in the cuttlefish recipe.
  • kmercamel: add aarch64/arm64 builds #51267: The kmercamel recipe's updates include additional platform support, which is relevant to the changes made in the cuttlefish recipe.
  • vcf-validator: add linux-aarch64 build #51425: The vcf-validator recipe's updates include additional platform support, which aligns with the changes in the cuttlefish recipe.
  • genbank: add aarch64/arm64 builds #51448: The genbank recipe's updates include additional platform support, which is relevant to the changes made in the cuttlefish recipe.
  • Sentieon add ARM support #51837: The sentieon recipe's updates include additional platform support, which aligns with the changes in the cuttlefish recipe.
  • mutyper: add aarch64/arm64 builds #51905: The mutyper recipe's updates include additional platform support, which is relevant to the changes made in the cuttlefish recipe.

Suggested labels

please review & merge, aarch64


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@mencian mencian added aarch64 Related to adding linux-aarch64 support osx-arm64 Related to adding osx-arm64 support labels Nov 6, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (4)
recipes/cuttlefish/meta.yaml (1)

37-41: LGTM! Metadata improvements enhance package quality.

Good improvements:

  • Added license family for better license categorization
  • Version-specific documentation URL ensures users get correct documentation
  • Proper quoting of URLs

Consider adding a brief description of platform support in the summary:

-  summary: "Construction of the compacted de Bruijn graph efficiently."
+  summary: "Construction of the compacted de Bruijn graph efficiently. Supports x86_64 and ARM architectures."
recipes/cuttlefish/build.sh (3)

3-4: Remove unused variables INCLUDES and LIBPATH.

The variables INCLUDES and LIBPATH are exported but not used in the script. Removing them will clean up the code.

Apply this diff to remove the unused variables:

-export INCLUDES="-I${PREFIX}/include"
-export LIBPATH="-L${PREFIX}/lib"

8-8: Use $(uname) instead of backticks for command substitution.

Using $(uname) enhances readability and is the recommended modern syntax.

Apply this diff to update the command substitution:

-if [[ `uname` == 'Darwin' ]]; then
+if [[ $(uname) == 'Darwin' ]]; then

13-13: Rephrase the comment to maintain professionalism.

The comment contains informal language. Rewriting it will maintain a professional tone.

Apply this diff to rephrase the comment:

-# It's dumb and absurd that the KMC build can't find the bzip2 header <bzlib.h>
+# Note: KMC build requires explicit paths to locate the bzip2 header <bzlib.h>
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 4c12ca9 and 8cb61a0.

📒 Files selected for processing (3)
  • recipes/cuttlefish/build.sh (1 hunks)
  • recipes/cuttlefish/build_failure.osx-64.yaml (0 hunks)
  • recipes/cuttlefish/meta.yaml (2 hunks)
💤 Files with no reviewable changes (1)
  • recipes/cuttlefish/build_failure.osx-64.yaml
🔇 Additional comments (4)
recipes/cuttlefish/meta.yaml (3)

20-29: LGTM! Requirements are architecture-agnostic.

The build and host requirements are properly structured for cross-platform compilation, with no architecture-specific dependencies that would block aarch64/arm64 builds.


44-46: LGTM! Platform support and identifiers properly declared.

The additional-platforms section correctly declares ARM64 support, and the biotools identifier enhances package discoverability.

Let's verify the build system support for these architectures:

#!/bin/bash
# Description: Verify build system support for ARM architectures
# Expected: Build configuration for ARM architectures in conda-forge or bioconda

# Check for ARM architecture support in conda configuration
rg -l "linux-aarch64|osx-arm64" --type yaml

Also applies to: 53-53


9-11: LGTM! Appropriate version pinning strategy.

The run_exports section with max_pin="x" ensures ABI compatibility by constraining updates to the same major version, which is suitable for C++ libraries.

Let's verify if other bioconda C++ packages follow similar pinning strategy:

recipes/cuttlefish/build.sh (1)

19-25: Build configuration updates look good.

The changes to the cmake command streamline the build process and support the addition of aarch64/arm64 builds.

@mencian mencian removed the aarch64 Related to adding linux-aarch64 support label Nov 6, 2024
@mencian mencian changed the title cuttlefish: add aarch64/arm64 builds cuttlefish: add arm64 builds Nov 6, 2024
@mencian mencian changed the title cuttlefish: add arm64 builds cuttlefish: add arm64 build Nov 6, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 5b842be and e57fd98.

📒 Files selected for processing (1)
  • recipes/cuttlefish/build.sh (1 hunks)
🔇 Additional comments (2)
recipes/cuttlefish/build.sh (2)

13-15: Address underlying build system issues with KMC

The comment suggests underlying issues with the KMC build system. Instead of working around it with environment variables, consider fixing the root cause.

Consider:

  1. Using pkg-config to locate bzip2
  2. Adding proper find_package() calls in CMake
  3. Setting up proper target_include_directories()

19-25: 🛠️ Refactor suggestion

Enhance CMake configuration for conda and ARM64 builds

The CMake configuration needs additional flags for proper conda integration and ARM64 support.

 cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
     -DCMAKE_CXX_COMPILER="${CXX}" \
     -DCMAKE_CXX_FLAGS="${CXXFLAGS}" \
-    -DINSTANCE_COUNT=64 \
+    -DINSTANCE_COUNT=$([[ "${target_platform}" == *"arm64"* ]] && echo "32" || echo "64") \
     -DCMAKE_INSTALL_PREFIX="${PREFIX}" \
+    -DCMAKE_INSTALL_LIBDIR=lib \
+    -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
+    -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON \
+    -DCMAKE_INSTALL_RPATH="${PREFIX}/lib" \
     "${CONFIG_ARGS}"

The changes:

  1. Adjust INSTANCE_COUNT based on architecture (ARM typically has fewer cores)
  2. Add proper RPATH handling for conda environments
  3. Ensure libraries are installed to the correct location

Comment on lines +3 to +6
export INCLUDES="-I${PREFIX}/include"
export LIBPATH="-L${PREFIX}/lib"
export LDFLAGS="${LDFLAGS} -L${PREFIX}/lib"
export CXXFLAGS="${CXXFLAGS} -O3 -fcommon -I${PREFIX}/include"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Optimize environment variable configuration

Several suggestions to improve the build configuration:

  1. INCLUDES and LIBPATH appear redundant as they're already covered by CXXFLAGS and LDFLAGS
  2. The -fcommon flag suggests potential symbol collision issues. Consider fixing the multiple symbol definitions instead
  3. Consider adding architecture-specific optimizations for ARM64 builds
-export INCLUDES="-I${PREFIX}/include"
-export LIBPATH="-L${PREFIX}/lib"
 export LDFLAGS="${LDFLAGS} -L${PREFIX}/lib"
-export CXXFLAGS="${CXXFLAGS} -O3 -fcommon -I${PREFIX}/include"
+export CXXFLAGS="${CXXFLAGS} -O3 -I${PREFIX}/include"
+if [[ "${target_platform}" == "osx-arm64" ]]; then
+    export CXXFLAGS="${CXXFLAGS} -mcpu=apple-m1"
+fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
export INCLUDES="-I${PREFIX}/include"
export LIBPATH="-L${PREFIX}/lib"
export LDFLAGS="${LDFLAGS} -L${PREFIX}/lib"
export CXXFLAGS="${CXXFLAGS} -O3 -fcommon -I${PREFIX}/include"
export LDFLAGS="${LDFLAGS} -L${PREFIX}/lib"
export CXXFLAGS="${CXXFLAGS} -O3 -I${PREFIX}/include"
if [[ "${target_platform}" == "osx-arm64" ]]; then
export CXXFLAGS="${CXXFLAGS} -mcpu=apple-m1"
fi

Comment on lines +8 to +11
if [[ `uname` == 'Darwin' ]]; then
export MACOSX_DEPLOYMENT_TARGET=10.11
export CXXFLAGS="${CXXFLAGS} -D_LIBCPP_DISABLE_AVAILABILITY"
export CONFIG_ARGS="-DCMAKE_FIND_FRAMEWORK=NEVER -DCMAKE_FIND_APPBUNDLE=NEVER"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Review macOS deployment target for ARM64 compatibility

Setting MACOSX_DEPLOYMENT_TARGET=10.11 might be too old for ARM64 support, as Apple Silicon requires at least macOS 11.0 (Big Sur).

 if [[ `uname` == 'Darwin' ]]; then
-    export MACOSX_DEPLOYMENT_TARGET=10.11
+    if [[ "${target_platform}" == "osx-arm64" ]]; then
+        export MACOSX_DEPLOYMENT_TARGET=11.0
+    else
+        export MACOSX_DEPLOYMENT_TARGET=10.11
+    fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if [[ `uname` == 'Darwin' ]]; then
export MACOSX_DEPLOYMENT_TARGET=10.11
export CXXFLAGS="${CXXFLAGS} -D_LIBCPP_DISABLE_AVAILABILITY"
export CONFIG_ARGS="-DCMAKE_FIND_FRAMEWORK=NEVER -DCMAKE_FIND_APPBUNDLE=NEVER"
if [[ `uname` == 'Darwin' ]]; then
if [[ "${target_platform}" == "osx-arm64" ]]; then
export MACOSX_DEPLOYMENT_TARGET=11.0
else
export MACOSX_DEPLOYMENT_TARGET=10.11
fi
export CXXFLAGS="${CXXFLAGS} -D_LIBCPP_DISABLE_AVAILABILITY"
export CONFIG_ARGS="-DCMAKE_FIND_FRAMEWORK=NEVER -DCMAKE_FIND_APPBUNDLE=NEVER"

@mencian mencian merged commit da8ab83 into master Nov 6, 2024
6 checks passed
@mencian mencian deleted the cuttlefish branch November 6, 2024 05:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
osx-arm64 Related to adding osx-arm64 support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant