Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve dependency install and build #10920

Closed
wants to merge 7 commits into from

Conversation

majetideepak
Copy link
Collaborator

@majetideepak majetideepak commented Sep 3, 2024

https://github.com/facebookincubator/velox/pull/10422/files added additional packages to be installed via brew.
When the package versions are updated via brew, it can cause a build failure.
See #10886
We must install or bundle fixed/supported versions to avoid this.

Allow installed package headers to be picked up before brew/system package headers
Add support for DEPENDENCY_DIR and INSTALL_PREFIX.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 3, 2024
Copy link

netlify bot commented Sep 3, 2024

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 50f9065
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/66e9f845b91abe0008fccd1f

@majetideepak majetideepak marked this pull request as draft September 4, 2024 03:23
@majetideepak majetideepak marked this pull request as ready for review September 4, 2024 14:19
@majetideepak majetideepak changed the title Remove library install from brew Fix dependency install and build Sep 4, 2024
@@ -144,7 +144,8 @@ function install_hdfs_deps {
wget_and_untar https://archive.apache.org/dist/hadoop/common/hadoop-2.10.1/hadoop-2.10.1.tar.gz hadoop
cp -a hadoop /usr/local/
fi
cmake_install $libhdfs3_dir
cd $libhdfs3_dir
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't leave this dir and it is not in () would we not be in the wrong dir at the end of the function?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Good catch!

facebook-github-bot pushed a commit that referenced this pull request Sep 5, 2024
Summary:
Brew fmt11 is not supported.
ccache brings in fmt11 from brew.
The brew fmt11 headers are taking precedence over bundled headers.
Bundle fmt and download prebuilt ccache that is statically linked.
We will fix the header include issue in #10920 and add ccache back.
Resolves #10936

Pull Request resolved: #10933

Reviewed By: amitkdutta

Differential Revision: D62256989

Pulled By: kgpai

fbshipit-source-id: 84aa17af17d564f291c416b479493bd9e5ea0715
@majetideepak majetideepak changed the title Fix dependency install and build Improve dependency install and build Sep 5, 2024
if [ -d "${BINARY_DIR}" ] && prompt "Do you want to rebuild ${NAME}?"; then
${SUDO} rm -rf "${BINARY_DIR}"
if [ -d "${BINARY_DIR}" ]; then
if prompt "Do you want to rebuild ${NAME}?"; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you are updating cmake_install, please fix ${NAME} per #10908, and then I will close my PR. Thanks!

Sth like

local DIR="${DEPENDENCY_DIR}/$1"
local NAME=${basename "${DIR}"}
pushd "${DIR}"

Copy link
Collaborator Author

@majetideepak majetideepak Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My PR fixes this by moving pushd $DIR above NAME.
Your change is not needed.

README.md Outdated
$ export DEPENDENCY_DIR=/path/to/your/dependencies
```
location to download and build packages. This defaults to `deps-download` in the current
working directory. Use `INSTALL_PREFIX` to set the install directory of the packages.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could we start INSTALL_PREFIX env in a new paragraph for clarity?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

.gitignore Show resolved Hide resolved
scripts/setup-macos.sh Outdated Show resolved Hide resolved
.gitignore Show resolved Hide resolved
@majetideepak majetideepak added ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall labels Sep 18, 2024
@facebook-github-bot
Copy link
Contributor

@Yuhta has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@Yuhta merged this pull request in 66b5b0d.

Copy link

Conbench analyzed the 1 benchmark run on commit 66b5b0d3.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

facebook-github-bot pushed a commit that referenced this pull request Sep 19, 2024
Summary:
Minio download broke in this PR: #10920

Resolves: #11035

Pull Request resolved: #11038

Reviewed By: Yuhta

Differential Revision: D63030598

Pulled By: pedroerp

fbshipit-source-id: 1e884813e4a88e5d33196dd4b0bb172558d979a2
@majetideepak majetideepak deleted the fix-fmt branch September 30, 2024 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants