Not Found "embree2/rtcore.h"

[sunjc0306]

Hi, I need to use this lib for my research. There is a fatal error to say“ ‘embree2/rtcore.h’ file was not found”? How do I install this lib?

[scopatz]

Can you please provide more information on how you tried to install pyembree?

[sunjc0306]

When I downloaded your code, I executed python setup.py install command on Ubuntu system

[sunjc0306]

Chinese meaning means files and folders can not be found

[scopatz]

That means you need to download and install embree, which is a dependency of embree.

[sunjc0306]

Is a dependency of embree downloaded from https://www.embree.org/downloads.html? In additional, C language is used for this dependency . What do I need to pay attention to when installing this dependency? Is there a detailed installation procedure?
Best regard!

[scopatz]

The short answer is yes, you will need a C compiler. Sorry I don't have the time right now to properly answer your question in full. If you would like to use a precompiled version, I recommend that you use conda and install the pyembree package from conda-forge

[akaszynski]

Running into this issue as well. Turns out the current release at https://github.com/embree/embree/releases/ is version 3, and pyembree is reliant on version 2. Should you choose to install this from source, you'll have to build the older version. Would be super cool if pyembree was on pip, but it's great that it's at least on conda.

[akaszynski]

Specifically, you'll want to install https://github.com/embree/embree/releases/tag/v2.17.7

[scopatz]

Yeah, I really disagree with the PyPI model, but if someone wanted to PR wheel building to the rever.xsh script, I would merge it.

[akaszynski]

Strangely, I've been able to build pyembree as a wheel and install it on another system that doesn't have embree installed. I'm testing this package out for a basic PR and thought to contribute here as well since it I generally just install packages through PyPi and not anaconda.

I'm attaching the wheel here (zipped due to github), but I'd like to know if it's actually functioning as expected. @Pursur, could you take a look at this?
pyembree.zip

[sunjc0306]

Thanks vary much, I hava install this library by directly copying the folder named embree2 in Embree v2 to the pyembree-master folder . However, I don't know if it's the right thing to do. Your method is also successfully achieved in my environment that the operation system is Ubuntu18.04 ,python version is 3.7

[akaszynski]

Your method is also successfully achieved in my environment that the operation system is Ubuntu18.04 ,python version is 3.7

Thanks @Pursur. I'm curious why the wheel works without having the embree libraries installed. From looking at the sizes of the rtcore...so files, I'm guessing much of the library is statically linked, which means that building this via a manylinux docker script will work, as would any MacOS or Windows build.

Yeah, I really disagree with the PyPI model, but if someone wanted to PR wheel building to the rever.xsh script, I would merge it.

While I appreciate your sentiment regarding PyPI, I like the flexibility of being able to use system python without resorting to anaconda for multi-platform Python development. Though, admittedly part of that comes with being somewhat unfamiliar with conda packages.

I'd actually recommend building via GitHub Actions or using Azure pipelines as this allows for multi-platform automated development. It takes a bit to get setup, but once it's done automating package creation and deployment is quite straightforward.

[scopatz]

Yeah, to be fair, I don't use anaconda either. I use conda and conda-forge.

[adamgranthendry]

@scopatz and @akaszynski I'm also struggling to install embree. I prefer poetry and have submitted questions here and here. Any help would be much appreciated!

[adamgranthendry]

Installation Instructions

Tested on:

OS: Windows 10 x64 Professional, Build 1909
Python: 3.8.5 x64
Date: 31-JULY-2021

Steps

1. Install Microsoft Visual Studio 2017 Build Tools and the Windows 10 SDK. These are required for building cython code.

(NOTE: FYI, Microsoft Visual Studio version numbers are not the same as Microsoft Visual C++ Compiler versions. MSVCv14X x64/x86 build tools are required for Python 3.8. Installing the Visual Studio 2017 Build Tools with the Windows 10 SDK. If you wish to install Visual Studio 2017, select the Desktop development with C++ Workload and make sure the required Windows 10 SDK build tool is selected under Individual Components. Refer to the Python Wiki for the right build tool versions to install).

2. Install vcpkg in C:\vcpkg and add the path to your System Environment Variables. Be sure to bootstrap and integrate vcpkg:

cd C:\vcpkg
git clone https://github.com/Microsoft/vcpkg.git
bootstrap-vcpkg.bat
vcpkg integrate install

Alternatively, you can download and install the embree-2.17.7.x64.msi Installer on Embree's GitHub page (to learn more about Embree, visit their website).

3. Install embree2 64-bit. If using vcpkg, this is done from the command-line as:

vcpkg install embree2:x64-windows

NOTE: To date, pyembree relies on Embree 2 and has not been updated version 3. The above command will install version 2.17.7, which is the required version.

4. Create your project folder and initialize a virtual environment with venv. In this example, Python 3.8.5 x64-bit is chosen, simply as it is the Python version used in Miniconda py38_4.9.2. (It may work for Python 3.6 and 3.7, but at the time of this writing, one should not go to higher than 3.8)

5. Install the following packages:

py -m pip install numpy cython wheel setuptools pyvista pykdtree rtree trimesh

6. Navigate to <virtual environment folder>\Lib\site-packages and clone the pyembree repo:

git clone https://github.com/scopatz/pyembree.git

and remove the underlying .git subfolder.

7. Move into pyembree folder and modify the setup.py file to:

import os
from setuptools import find_packages, setup

import numpy as np
from Cython.Build import cythonize
from Cython.Distutils import build_ext


include_path = [
    np.get_include(),
]

ext_modules = cythonize("pyembree/*.pyx", language_level=3, include_path=include_path)

for ext in ext_modules:
    ext.include_dirs = include_path
    ext.libraries = [
        "pyembree/embree2/lib/embree",
        "pyembree/embree2/lib/tbb",
        "pyembree/embree2/lib/tbbmalloc",
    ]

setup(
    name="pyembree",
    version="0.1.6",
    cmdclass={"build_ext": build_ext},
    ext_modules=ext_modules,
    zip_safe=False,
    packages=find_packages(),
    include_package_data=True,
    package_data={"pyembree": ["*.cpp", "*.dll"]},
)

8. Move into the pyembree subfolder. Copy the folder C:\vcpkg\installed\x64-windows\include\embree2 into this folder (if you installed from the .msi, this will be the folder C:\Program Files\Intel\Embree v2.17.7 x64\include\embree2). In addition, copy the libraries from C:\vcpkg\installed\x64-windows\lib into this embree2 folder (if you installed from the .msi, this will be the folder C:\Program Files\Intel\Embree v2.17.7 x64\lib\). You only need the (static) libraries

embree.lib
tbb.lib
tbbmalloc.lib

and dynamic libraries

embree.dll
tbb.dll
tbbmalloc.dll

9. Add the following line to the top of every *.pyx and *.pxd file in pyembree:

# distutils: language=c++

and change every relative import of an rtcore module to an absolute import in each file. For example, change:

cimport rtcore as rtc

to

cimport pyembree.rtcore as rtc

10. Create the following MANIFEST.in file in the top-level pyembree folder:

include *.h *.pxd *.dll
recursive-include embree2 *.h *.isph
recursive-include embree2/lib *.lib

11. At the end, your folder should look like this:

<virtual environment folder>\Lib\site-packages\pyembree
│   .authors.yml
│   .gitignore
│   .mailmap
│   .travis-install.sh
│   .travis.yml
│   AUTHORS
│   CHANGELOG.rst
│   LICENSE
│   MANIFEST.in
│   README.rst
│   rever.xsh
│   setup.py
│   
├───examples
│       attenuate.py
│       intersection.py
│       
├───news
│       TEMPLATE.rst
│       
├───pyembree
│   │   embree.dll
│   │   mesh_construction.h
│   │   mesh_construction.pyx
│   │   rtcore.pxd
│   │   rtcore.pyx
│   │   rtcore_geometry.pxd
│   │   rtcore_geometry_user.pxd
│   │   rtcore_ray.pxd
│   │   rtcore_scene.pxd
│   │   rtcore_scene.pyx
│   │   tbb.dll
│   │   tbbmalloc.dll
│   │   triangles.pyx
│   │   __init__.pxd
│   │   __init__.py
│   │
│   └───embree2
│       │   rtcore.h
│       │   rtcore.isph
│       │   rtcore_builder.h
│       │   rtcore_geometry.h
│       │   rtcore_geometry.isph
│       │   rtcore_geometry_user.h
│       │   rtcore_geometry_user.isph
│       │   rtcore_ray.h
│       │   rtcore_ray.isph
│       │   rtcore_scene.h
│       │   rtcore_scene.isph
│       │   rtcore_version.h
│       │
│       └───lib
│               embree.lib
│               tbb.lib
│               tbbmalloc.lib
│
├───recipes
│   └───pyembree
│           build.sh
│           meta.yaml
│
└───tests
        test_intersection.py

12. Build pyembree by running the following from the top-level pyembree folder (TIP: To help debug errors, pipe stderr to a local file errors.txt that you can look at):

py setup.py build_ext -i 2> errors.txt

13. Test that everything is working correctly by opening a python terminal in your virtual environment and running

>>> import pyembree
>>> from pyembree import rtcore_scene
>>>

If you get no errors, then pyembree has been installed correctly. You can also create a .egg and install the library with

py setup.py install

The setup.py we created will copy the dlls and cpp files to the correct folder.

14. As an last optional step, create a main.py in your virtual environment and copy and paste the code from Project to Finite Plane. If it runs without errors in 1-2 seconds, then pyembree is working properly with trimesh for vectorized ray tracing (NOTE: trimesh will work without pyembree, but it will be very slow. It should not take a long time to run this example code if pyembree is properly installed. trimesh resorts back to trimesh.ray.ray_triangle.RayMeshIntersector if it cannot import pyembree (see pyembree ray casting acceleration #875), so you won't see any errors; just slow running code):

import numpy as np
from pykdtree.kdtree import KDTree

import pyvista as pv
from pyvista import examples

# Load data
data = examples.load_random_hills()
data.translate((10, 10, 10))

# Create triangular plane (vertices [10, 0, 0], [0, 10, 0], [0, 0, 10])
size = 10
vertices = np.array([[size, 0, 0], [0, size, 0], [0, 0, size]])
face = np.array([3, 0, 1, 2])

planes = pv.PolyData(vertices, face)

# Subdivide plane so we have multiple points to project to
planes = planes.subdivide(8)

# Get origins and normals
origins = planes.cell_centers().points
normals = planes.compute_normals(cell_normals=True, point_normals=False)["Normals"]

# Vectorized Ray trace
points, pt_inds, cell_inds = data.multi_ray_trace(
    origins, normals
)  # Must have rtree, trimesh, and pyembree installed

# Filter based on distance threshold, if desired (mimics VTK ray_trace behavior)
# threshold = 10  # Some threshold distance
# distances = np.linalg.norm(origins[inds] - points, ord=2, axis=1)
# inds = inds[distances <= threshold]

tree = KDTree(data.points.astype(np.double))
_, data_inds = tree.query(points)

elevations = data.point_arrays["Elevation"][data_inds]

# Mask points on planes
planes.cell_arrays["Elevation"] = np.zeros((planes.n_cells,))
planes.cell_arrays["Elevation"][pt_inds] = elevations
planes.set_active_scalars("Elevation")  # Probably not necessary, but just in case

# Create axes
axis_length = 20
tip_length = 0.25 / axis_length * 3
tip_radius = 0.1 / axis_length * 3
shaft_radius = 0.05 / axis_length * 3
x_axis = pv.Arrow(
    direction=(axis_length, 0, 0),
    tip_length=tip_length,
    tip_radius=tip_radius,
    shaft_radius=shaft_radius,
    scale="auto",
)
y_axis = pv.Arrow(
    direction=(0, axis_length, 0),
    tip_length=tip_length,
    tip_radius=tip_radius,
    shaft_radius=shaft_radius,
    scale="auto",
)
z_axis = pv.Arrow(
    direction=(0, 0, axis_length),
    tip_length=tip_length,
    tip_radius=tip_radius,
    shaft_radius=shaft_radius,
    scale="auto",
)
x_label = pv.PolyData([axis_length, 0, 0])
y_label = pv.PolyData([0, axis_length, 0])
z_label = pv.PolyData([0, 0, axis_length])
x_label.point_arrays["label"] = [
    "x",
]
y_label.point_arrays["label"] = [
    "y",
]
z_label.point_arrays["label"] = [
    "z",
]

# Plot results
p = pv.Plotter()
p.add_mesh(x_axis, color="r")
p.add_point_labels(x_label, "label", show_points=False, font_size=24)
p.add_mesh(y_axis, color="r")
p.add_point_labels(y_label, "label", show_points=False, font_size=24)
p.add_mesh(z_axis, color="r")
p.add_point_labels(z_label, "label", show_points=False, font_size=24)
p.add_mesh(data)
p.add_mesh(planes)
p.show()

[adamgranthendry]

@scopatz Why do you disagree with the PyPI model? I'm curious and would like to learn.

[adam-grant-hendry]

@akaszynski

1. pyembree will install without embree (though I'm not sure why). If embree is unavailable, trimesh resorts back to trimesh.ray.ray_triangle.RayMeshIntersector per pyembree ray casting acceleration #875, so you won't see an error in that instance, but it will be extremely slow.

2. Did you build your wheel using CMake?