Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add env for arm64 and xfail geospatial tests #8371

Merged
merged 3 commits into from
Feb 16, 2024

Conversation

ncclementi
Copy link
Contributor

@ncclementi ncclementi commented Feb 15, 2024

Add environment-arm64.yml to build dev env on mac m1 and m2.

While doing this, a shapely/geos bug fix was exposed (see issue shapely/shapely#1992). Including the corresponding fixes, when shapely is upgraded, the xfails should start xpass.

This PR: needs a follow up for documentation. To point out:

  • that apache-flink install will downgrade some packages.
  • We are not including go-shfmt the pacakge is not available for amr64 and it's not on pip. I don't see this package on our current requirement-dev.txt so I'll ignore this.
  • quarto >=1.4 is not available in conda-forge it'll have to be installed following https://quarto.org/docs/get-started/index.html

Add environment-arm64.yml to build dev env on mac m1 and m2.
While doing this, a shapely/geos bug fix was exposed (see
shapely issue 1992). Including the corresponding fixes, when
shapely is upgraded, the xfails should start xpass.
environment-arm64.yml Outdated Show resolved Hide resolved
@ncclementi
Copy link
Contributor Author

ncclementi commented Feb 16, 2024

I should take care of this one #8375 here too. Not sure what's the best way. Maybe pinning dask?

edit: maybe add to ibis/pyproject.toml: added this. ✅

@ncclementi
Copy link
Contributor Author

ncclementi commented Feb 16, 2024

With this environment, I tried

  • just flink up
  • pytest ibis/backends/flink/tests

and pretty much everything fails with

E               RuntimeError: Java gateway process exited before sending its port number.
E               Stderr:
E               The operation couldn’t be completed. Unable to locate a Java Runtime.
E               Please visit http://www.java.com for information on installing Java.

for the record I do have installed apache-flink

Screenshot 2024-02-15 at 8 24 10 PM

cc: @chloeh13q @deepyaman any idea what's happening here?

EDIT: turns out I don't have java install (why would I 😛) . I'm trying to brew install and it goes through but it's asking me to do some things, but I don't know what's the prefer/recommended option here.
If anyone that runs flink stuffs can provide some insights it would be great : ) .

I did brew install java11

For the system Java wrappers to find this JDK, symlink it with
  sudo ln -sfn /opt/homebrew/opt/openjdk@11/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-11.jdk

openjdk@11 is keg-only, which means it was not symlinked into /opt/homebrew,
because this is an alternate version of another formula.

If you need to have openjdk@11 first in your PATH, run:
  echo 'export PATH="/opt/homebrew/opt/openjdk@11/bin:$PATH"' >> ~/.zshrc

For compilers to find openjdk@11 you may need to set:
  export CPPFLAGS="-I/opt/homebrew/opt/openjdk@11/include"
==> Summary

@chloeh13q
Copy link
Contributor

@ncclementi I use sdkman to manage java because I have multiple versions of java running on my computer. Installing java 11 using sdkman is pretty straightforward, you just need to run

sdk install java 11.0.19-tem

and there is no extra step involved.

I don't remember having to symlink when I installed java11 via brew install; maybe try running the tests first to see whether it works?

Copy link
Member

@cpcloud cpcloud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit, otherwise LGTM.

Thanks for powering through this 😬

ibis/backends/postgres/tests/test_geospatial.py Outdated Show resolved Hide resolved
@cpcloud cpcloud added dependencies Issues or PRs related to dependencies developer-tools Tools related to ibis development labels Feb 16, 2024
@cpcloud cpcloud added this to the 9.0 milestone Feb 16, 2024
@cpcloud
Copy link
Member

cpcloud commented Feb 16, 2024

FWIW go-shfmt isn't a Python package so it won't show up on PyPI or in requirements-dev.txt.

@cpcloud cpcloud enabled auto-merge (squash) February 16, 2024 10:34
@cpcloud cpcloud merged commit 08e9d1e into ibis-project:main Feb 16, 2024
75 checks passed
ncclementi added a commit to ncclementi/ibis that referenced this pull request Feb 21, 2024
Co-authored-by: Phillip Cloud <417981+cpcloud@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Issues or PRs related to dependencies developer-tools Tools related to ibis development
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants