Implement 3d picking of shapes #73
Comments
|
I would like to tackle this, I'll start this weekend. In a modern version of GL, or any of the low overhead APIs, this would be best accomplished using occlusion queries and writing ids as color, but implementing it that way feels pointless without framebuffer support (we would stall the pipeline waiting for the query to finish because we don't have another framebuffer, and so can't do the main render while we wait), so for now I will write it using the select buffer APIs. |
|
Excellent. Do keep in mind that we are currently using OpenGL 3 on Windows/Linux, but OpenGL 2 on MacOS. |
|
If krma gets used, your opengl programming will be in vain. There is a
"cherno" video on youtube demonstrating how to do this, it may have used
vulkan, but it writes ids as color in a separate image and uses the mouse
coordinates to extract the object id. In any case, you may be better
served to use vulkan.
I solved this a different way for REGEN. I need to be able to select ALL
the objects under the mouse pointer, as more than one "entity" could occupy
the same coordinates, and the user needs to be able to pick which one. So
I convert the mouse coordinates to a world space ray using the inverse
camera matrix, and I use axis aligned bounding boxes and
ray-intersects-entity functions to compute hits. For complex stuff like
NURBS, I am using compute shaders to map ray-intersects-point onto the
curve and then I reduce on the CPU to see if an entity has a hit.
…On Thu, Oct 27, 2022 at 7:49 PM Kaveh Kardan ***@***.***> wrote:
Excellent. Do keep in mind that we are currently using OpenGL 3 on
Windows/Linux, but OpenGL 2 on MacOS.
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABGMMNDIIUJKKYZHBYPCSTWFMPJ3ANCNFSM5776Y2EQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
I haven't examined Kaveh's code deeply yet, but if he is using triangular
meshes for polyhedrons, it would be fairly straightforward to write a
compute shader that takes index and vertex buffers, maps
ray-intersects-triangle on them, and then reduces to detect a hit. If you
wanted to do it with a ray.
…On Thu, Oct 27, 2022 at 8:12 PM Andrew Wolven ***@***.***> wrote:
If krma gets used, your opengl programming will be in vain. There is a
"cherno" video on youtube demonstrating how to do this, it may have used
vulkan, but it writes ids as color in a separate image and uses the mouse
coordinates to extract the object id. In any case, you may be better
served to use vulkan.
I solved this a different way for REGEN. I need to be able to select ALL
the objects under the mouse pointer, as more than one "entity" could occupy
the same coordinates, and the user needs to be able to pick which one. So
I convert the mouse coordinates to a world space ray using the inverse
camera matrix, and I use axis aligned bounding boxes and
ray-intersects-entity functions to compute hits. For complex stuff like
NURBS, I am using compute shaders to map ray-intersects-point onto the
curve and then I reduce on the CPU to see if an entity has a hit.
On Thu, Oct 27, 2022 at 7:49 PM Kaveh Kardan ***@***.***>
wrote:
> Excellent. Do keep in mind that we are currently using OpenGL 3 on
> Windows/Linux, but OpenGL 2 on MacOS.
>
> —
> Reply to this email directly, view it on GitHub
> <#73 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AABGMMNDIIUJKKYZHBYPCSTWFMPJ3ANCNFSM5776Y2EQ>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>
|
|
Currently the |
|
If I implement faces as triangle fans, then it will be the same for mouse
picking purposes.
I don't know how it would actually look though. Do you provide vertex
normals for the vertices in the faces?
…On Thu, Oct 27, 2022 at 8:57 PM Kaveh Kardan ***@***.***> wrote:
Currently the polyhedron class supports arbitrary faces. There is a very
naive triangulation method, only guaranteed to work with convex faces.
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABGMMLA6SG6NNT474HXYKDWFMXIVANCNFSM5776Y2EQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Yes, I do like the object id based solutions as I mentioned. That's more or less what an occlusion query based solution would do as well; the difference is that you can toggle the depth test when you do an occlusion query to count anything in the scissor/viewport as a hit, which means you can support both selecting occluded objects and box selection (you just change the scissor rect to match the box instead of the cursor) almost trivially. Another nice thing with both of these methods over a raymarch is that, since they operate on the results of the pipeline rather than AABBs, they will always be pixel perfect, and all of this without needing to do anything complex in software/compute.
I agree that Vulkan (and Metal, since I presume we don't want to rely on MoltenVk for MacOS) would be better for many things, including this. My main concern is that it seems not great to have Vulkan/Metal and GL contexts coexisting in the same application, especially because the current code is using the fixed function pipeline and so would have to duplicate vertex data between three APIs (no GL buffer exposed to share with extensions or anything like that). Even if we did do it in Vulkan right now, the code would have to change drastically to fit within a new backend anyway. To me, this indicates that we should either do it in GL for now and replace it with OQs later, or punt it until the rendering backend has settled. |
|
I have had some discussions with Kaveh about graphics languages. Maybe he
could chime in. The current state of affairs is that Kaveh is on Macos and
doesn't want to neglect macos, but the current code is mostly OpenGL 1.1
with some 2.1 stuff. The absolute bare minimum requirement for moving
forward is OpenGL 3.3. The reasoning behind that is another discussion.
We could open that discussion if you like, it's basically about performance
and support. However, although macOS does indeed support OpenGL 3.3
through 4.5, but unlike Windows and Linux, Macos can't run OpenGL 1.1/2.1
code in a context version >= 3.3. So a jump needs to take place. We
wouldn't be mixing GL 1.1 with GL 3.3 or GL 1.1 with vulkan, because that
is a technical issue really not worth solving. Kons-9 is not that big yet,
and we could port the GL 1.1 stuff in a fairly short amount of time. I
volunteered (or am being allowed) to write a graphics engine which would
ultimately be platform independent. I had a choice between OpenGL 4.5,
which is actually deprecated on macos and could be removed in any future
release of macos, or Vulkan. I already had the pipeline code written in
Vulkan, since my application REGEN uses that, so I am deferring the OpenGL
for later or possibly never. MoltenVK will be an ok crutch for the time
being. It doesn't support many extensions, like line width, and it doesn't
support compute shaders. We are not using compute shaders yet, so the
macOS version of kons-9 can get by with MoltenVK which should buy me some
time to write the Metal backend to the graphics engine [hopefully object
selection could be written for MoltenVK, which it should support]. I'm
getting very close to actually starting the port of kons-9 to this graphics
engine, I basically just have to make the graphics engine behave like
opengl polygon mode, with edges and vertices and then it is ready to slip
under. If you're still considering writing the object selection code in
OpenGL, you would either have to write it in GL 2.1, in which case it would
be replaced by vulkan/metal in short order, or if you wrote in in >= GL 3.3
it would force me to write an entire OpenGL 3.3 backend for the graphics
engine. I think the time would be better spent making a Metal backend for
the graphics engine rather than building an OpenGL backend which is
deprecated on macOS anyway. So imho MoltenVK is the way forward on macOS
until the Metal backend is finished. The object selection code would have
to be rewritten from GL 2.1 to something else anyway. Why not make it
Vulkan, which should work on MacOS for the short term, and would be used by
Windows and Linux in the long term, and the necessary rewrite would be from
Vulkan to Metal.
…On Fri, Oct 28, 2022 at 9:12 AM Nick Smoker ***@***.***> wrote:
If krma gets used, your opengl programming will be in vain. There is a
"cherno" video on youtube demonstrating how to do this, it may have used
vulkan, but it writes ids as color in a separate image and uses the mouse
coordinates to extract the object id.
Yes, I do like the object id based solutions as I mentioned. That's more
or less what an occlusion query based solution would do as well; the
difference is that you can toggle the depth test when you do an occlusion
query to count anything in the scissor/viewport as a hit, which means you
can support both selecting occluded objects and box selection (you just
change the scissor rect to match the box instead of the cursor) almost
trivially. Another nice thing with both of these methods over a raymarch is
that, since they operate on the results of the pipeline rather than AABBs,
they will always be pixel perfect, and all of this without needing to do
anything complex in software/compute.
In any case, you may be better served to use vulkan.
I agree that Vulkan (and Metal, since I presume we don't want to rely on
MoltenVk for MacOS) would be better for many things, including this. My
main concern is that it seems not great to have Vulkan/Metal and GL
contexts coexisting in the same application, especially because the current
code is using the fixed function pipeline and so would have to duplicate
vertex data between three APIs (no GL buffer exposed to share with
extensions or anything like that). Even if we did do it in Vulkan right
now, the code would have to change drastically to fit within a new backend
anyway. To me, this indicates that we should either do it in GL for now and
replace it with OQs later, or punt it until the rendering backend has
settled.
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABGMMPT2UCXEFKFAJ4QQ7LWFPNLHANCNFSM5776Y2EQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
I completely agree that no GL 3.3 backend should be written, you can dump GL with better APIs on all platforms these days, the only reason to keep it is legacy compat. If the Vulkan backend is coming very quickly as you say, then I would be in support of putting this in a branch until the rendering backend is solidified; the reason I wanted to do it in GL is because I was under the impression that the Vulkan backend was a long ways off. If the VK backend is indeed coming soon, I would be happy to write it in Vulkan and keep the object selection in a branch, or in my fork, until the move to Vulkan is complete. |
|
You can actually experiment with the vulkan-based graphics engine now. I
could write instructions on how to load it, and how to use the API. You
have to install the VulkanSDK and run a script to compile the shaders, as I
am not including the spirv in the repository.
…On Fri, Oct 28, 2022 at 2:29 PM Nick Smoker ***@***.***> wrote:
I completely agree that no GL 3.3 backend should be written, you can dump
GL with better APIs on all platforms these days, the only reason to keep it
is legacy compat. If the Vulkan backend is coming very quickly as you say,
then I would be in support of punting this until the rendering backend is
solidified; the reason I wanted to do it in GL is because I was under the
impression that the Vulkan backend was a long ways off.
If the VK backend is indeed coming soon, I would be happy to keep the
object selection in a branch, or in my fork, until the move to Vulkan is
complete.
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABGMMKBYQW7WV64ADC7P33WFQSSNANCNFSM5776Y2EQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Good discussion, guys. Let's take the long-term view. We are in an enviable position where we don't have any short-term deadlines or requirements. As Andrew mentioned, I am on MacOS and I do want the system to run on MacOS/Linux/Windows, so Vukan and Metal seem to be the way to go. As a high-level view of where I imagine the interactive graphics going, I want to be on the same level as Maya/Houdini/Blender. Though it is early days for kons-9, I want it to be a contender in both GUI features and performance. Currently, our OpenGL code is in |
|
I did read the Open Inventor docs recently, and liked much of what I saw. I can see us having various picking abstractions for manipulators, surface representations, and volumetrics. And probably more that I'm not aware of. |
My initial intent was a fairly simple abstraction over GPU queries and query pools, similar to Blender's object selection API, but with fewer layers of indirection since we do not need to fit the API into as complex of a system as blender. |
|
Sounds like a plan. I hereby delegate the design to you. :) |
|
Cool. I will post an update on the design here once I have a demo running in Vulkan. |
|
should I assume the cpu code will be common lisp?
…On Fri, Oct 28, 2022 at 3:45 PM Nick Smoker ***@***.***> wrote:
Cool. I will post an update on the design here once I have a demo running
in Vulkan.
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABGMMIRMYRZR6RRGAFVRFLWFQ3PPANCNFSM5776Y2EQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Yeah, I plan on using krma as a base for this. If I have trouble with setup, can I come to you with questions? For actual usage, I can probably make do reading the code. |
|
Yes, I would actually appreciate some feedback on it.
Technically speaking, I'm an amatuer graphics programmer.
…On Fri, Oct 28, 2022 at 5:24 PM Nick Smoker ***@***.***> wrote:
Yeah, I plan on using krma as a base for this. If I have trouble with
setup, can I come to you with questions? For actual usage, I can probably
make do reading the code.
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABGMMM6SFP7YROK42NYPS3WFRHC3ANCNFSM5776Y2EQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
@nsmoker Have you had a chance to look into this? No rush, just wondering. |
[Need someone for this.]
Initially by clicking on just shapes in the 3D view.
Requires some knowledge of OpenGL.
Later, it can be extended to mesh parts (face, edge, vertex) of poly-mesh at some point.
The text was updated successfully, but these errors were encountered: