Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add color for sampling from mesh #6842

Merged
merged 7 commits into from
Aug 21, 2024
Merged

Add color for sampling from mesh #6842

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
21c6dec
First version, colors are not sampled correctly
nikste Jun 24, 2024
d6b1970
Fixed uv lookup
nikste Jun 28, 2024
2bef72c
Apply style
nikste Jun 28, 2024
6ece35e
Add poissant sampling
nikste Jun 28, 2024
a69d392
Add changelog entry
nikste Jun 28, 2024
cb5a95e
Add triangle_material_id
nikste Aug 8, 2024
f8b0492
apply style
nikste Aug 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions cpp/open3d/geometry/TriangleMesh.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
#include <numeric>
#include <queue>
#include <tuple>
#include <iostream>

#include "open3d/geometry/BoundingVolume.h"
#include "open3d/geometry/IntersectionTest.h"
Expand Down Expand Up @@ -440,7 +441,7 @@ std::shared_ptr<PointCloud> TriangleMesh::SamplePointsUniformlyImpl(

// sample point cloud
bool has_vert_normal = HasVertexNormals();
bool has_vert_color = HasVertexColors();
// bool has_vert_color = HasVertexColors();
utility::random::UniformRealGenerator<double> uniform_generator(0.0, 1.0);
auto pcd = std::make_shared<PointCloud>();
pcd->points_.resize(number_of_points);
Expand All @@ -450,9 +451,9 @@ std::shared_ptr<PointCloud> TriangleMesh::SamplePointsUniformlyImpl(
if (use_triangle_normal && !HasTriangleNormals()) {
ComputeTriangleNormals(true);
}
if (has_vert_color) {
// if (has_vert_color) {
pcd->colors_.resize(number_of_points);
}
// }

for (size_t point_idx = 0; point_idx < number_of_points; ++point_idx) {
double r1 = uniform_generator();
Expand All @@ -478,6 +479,18 @@ std::shared_ptr<PointCloud> TriangleMesh::SamplePointsUniformlyImpl(
b * vertex_colors_[triangle(1)] +
c * vertex_colors_[triangle(2)];
}
// if (has_triangle_uvs && has_textures) {
Eigen::Vector2d uv = a * triangle_uvs_[triangle(0)] +
b * triangle_uvs_[triangle(1)] +
c * triangle_uvs_[triangle(2)];

int w = textures_[0].width_;
int h = textures_[0].height_;
pcd->colors_[point_idx] = Eigen::Vector3d(
(double) *textures_[0].PointerAt<uint8_t>(uv(0) * w, uv(1) * h, 0) / 255 ,
(double) *textures_[0].PointerAt<uint8_t>(uv(0) * w, uv(1) * h, 1) / 255 ,
(double) *textures_[0].PointerAt<uint8_t>(uv(0) * w, uv(1) * h, 2) / 255);
Copy link
Contributor Author

@nikste nikste Jun 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benjaminum
I'm not an Computer graphics expert. It seems the barycentric coordinates you were referring to (#2483 (comment)) are a sort of mixture weights for triangles (in this case it looks like a,b,c ?). Unfortunately this code seems to samples from the correct colors but from the wrong position in the texture map.

Am I making a mistake here with the uv computation or am I using the PointerAt function wrong ?
Maybe you see a mistake immediately?

picture:
result of current sampling code:
test_wrong
right:
test_right

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you try (1-uv(1))*h?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also does not look quite right unfortunately:
test_1_minus

Copy link
Contributor Author

@nikste nikste Jun 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could it be that triangle(0), triangle(1), triangle(2) in triangle_uvs_don't select adjacent triangle vertices in the texture?
I'm trying to run it on a cube with different surface colors. This is the texture map:
cube_texture
sampled vertices were:
cube_texture_sampled
if we take a weighted average of these sampled points, we might end up in the black area (no valid texture).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose the triangle_index_generator does not take this into account, or something is unexpected in triangle_uvs_

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nikste triangle stores the indices of the vertices but we need the index to the right uv.
triangle_uvs_ if I recall correctly should have the size 3*#triangles. Can you try to get the uvs with [3tidx, 3tidx+1, 3*tidx+2]?

// }
}

return pcd;
Expand Down