Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Solves issue #7059 #7113

Merged
merged 18 commits into from
Nov 2, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions src/core/environment.js
Original file line number Diff line number Diff line change
Expand Up @@ -1137,6 +1137,102 @@ function exitFullscreen() {
}
}


/**
* Converts 3D world coordinates to 2D screen coordinates.
*
* This function takes a 3D vector and converts its coordinates
* from the world space to screen space. This is useful for placing
* 2D elements in a 3D scene or for determining the screen position
* of 3D objects.
*
* @method worldToScreen
* @param {p5.Vector} worldPosition The 3D coordinates in the world space.
* @return {p5.Vector} A vector containing the 2D screen coordinates.
* @example
* <div>
* <code>
* // Example 2: Convert 2D world coordinates of a rotating square to screen coordinates
* function setup() {
* createCanvas(400, 400);
* let vertices = [
* createVector(-10, -10),
* createVector(10, -10),
* createVector(10, 10),
* createVector(-10, 10)
* ];
*
* push();
* translate(200, 200);
* rotate(PI / 4);
* let screenPos = vertices.map(v => worldToScreen(v));
* pop();
*
* background(200);
* screenPos.forEach((pos, i) => {
* text(`(${pos.x.toFixed(1)}, ${pos.y.toFixed(1)})`, pos.x, pos.y);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

* });
* }
* </code>
* </div>
* @example
* <div>
* <code>
* // Example 1: Convert 3D world coordinates of a rotating cube to 2D screen coordinates
* function setup() {
* createCanvas(400, 400, WEBGL);
* let vertices = [
* createVector(-50, -50, -50),
* createVector(50, -50, -50),
* createVector(50, 50, -50),
* createVector(-50, 50, -50),
* createVector(-50, -50, 50),
* createVector(50, -50, 50),
* createVector(50, 50, 50),
* createVector(-50, 50, 50)
* ];
*
* push();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you feel about moving this to draw() and making the rotation change slowly over time?

Copy link
Member Author

@Garima3110 Garima3110 Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A great idea , will do this in a while.

* translate(0, 0, 0);
* rotateX(PI / 4);
* rotateY(PI / 4);
* let screenPos = vertices.map(v => worldToScreen(v));
* pop();
*
* background(200);
* screenPos.forEach((pos, i) => {
* text(`(${pos.x.toFixed(1)}, ${pos.y.toFixed(1)})`, pos.x, pos.y);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the screen coordinates relative to the top left? if so we may need to subtract width/2 and height/2 to keep these centered when drawing them in WebGL mode

* });
* }
* </code>
* </div>
*
*/
p5.prototype.worldToScreen = function(worldPosition) {
const renderer = this._renderer;
if (renderer.drawingContext instanceof CanvasRenderingContext2D) {
// Handle 2D context
const transformMatrix = new DOMMatrix()
.scale(1 / pixelDensity())
.multiply(renderer.drawingContext.getTransform());
const screenCoordinates = transformMatrix.transformPoint(
new DOMPoint(worldPosition.x, worldPosition.y)
);
return createVector(screenCoordinates.x, screenCoordinates.y);
} else {
// Handle WebGL context
const cameraCoordinates = renderer.uMVMatrix.multiplyPoint(worldPosition);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, maybe it's this -- we split up uMVMatrix into uModelMatrix and uViewMatrix, and only update uMVMatrix right before we draw a shape here:

_setMatrixUniforms() {
const modelMatrix = this._renderer.uModelMatrix;
const viewMatrix = this._renderer.uViewMatrix;
const projectionMatrix = this._renderer.uPMatrix;
const modelViewMatrix = (modelMatrix.copy()).mult(viewMatrix);
this._renderer.uMVMatrix = modelViewMatrix;

Maybe we can refactor the bit that sets uMVMatrix into a function on the renderer, like calculateCombinedMatrix(), which gets called in _setMatrixUniforms above, and then you can also call it here right before you make cameraCoordinates?

const normalizedDeviceCoordinates =
renderer.uPMatrix.multiplyAndNormalizePoint(cameraCoordinates);
const screenX = (0.5 + 0.5 * normalizedDeviceCoordinates.x) * this.width;
const screenY = (0.5 - 0.5 * normalizedDeviceCoordinates.y) * this.height;
const screenZ = 0.5 + 0.5 * normalizedDeviceCoordinates.z;
return createVector(screenX, screenY, screenZ);
}
};



/**
* Returns the sketch's current
* <a href="https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Web_mechanics/What_is_a_URL" target="_blank">URL</a>
Expand Down
48 changes: 48 additions & 0 deletions test/unit/core/environment.js
Original file line number Diff line number Diff line change
Expand Up @@ -236,4 +236,52 @@ suite('Environment', function() {
assert.isNumber(myp5.displayDensity(), pd);
});
});

suite('2D context test', function() {
beforeEach(function() {
myp5.createCanvas(100, 100);
});

test('worldToScreen for 2D context', function() {
let worldPos = myp5.createVector(50, 50);
let screenPos = myp5.worldToScreen(worldPos);
assert.closeTo(screenPos.x, 50, 0.1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is great! Can we add a test of rotation in 2D too?

assert.closeTo(screenPos.y, 50, 0.1);
});

test('worldToScreen with rotation in 2D', function() {
myp5.push();
myp5.rotate(myp5.PI / 2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think since we translate before we rotate, the rotation doesn't end up actually affecting the position. If we rotate first, we should see an effect (I would expect (x, y) to become (y, -x) I think?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, actually, ignore this -- it's only true if you're using (0,0) as your local coordinate. I see the offset by 10 does actually switch axes here!

myp5.translate(50, 50);
let worldPos = myp5.createVector(10, 0);
let screenPos = myp5.worldToScreen(worldPos);
myp5.pop();
assert.closeTo(screenPos.x, 50, 0.1);
assert.closeTo(screenPos.y, 60, 0.1);
});
});

suite('3D context test', function() {
beforeEach(function() {
myp5.createCanvas(100, 100, myp5.WEBGL);
});

test('worldToScreen for 3D context', function() {
let worldPos = myp5.createVector(0, 0, 0);
let screenPos = myp5.worldToScreen(worldPos);
assert.closeTo(screenPos.x, 50, 0.1);
assert.closeTo(screenPos.y, 50, 0.1);
});

test('worldToScreen with rotation in 3D', function() {
myp5.push();
myp5.rotateY(myp5.PI / 2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar comment here: just rotating won't affect the coordinates, so could we do a translation after this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, ignore that, as mentioned above, it should be ok as long as you're converting a nonzero coordinate. But it feels odd that the resulting coordinate would still be 50,50 since the untransformed example above also ends up at that. should this value be something else?

Copy link
Member Author

@Garima3110 Garima3110 Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, ignore that, as mentioned above, it should be ok as long as you're converting a nonzero coordinate. But it feels odd that the resulting coordinate would still be 50,50 since the untransformed example above also ends up at that. should this value be something else?

Maybe it should be 200,200
The default camera position in p5.js is at (0, 0, 800), looking towards the origin (0, 0, 0) along the negative Z-axis. The camera's view is centered on the Z-axis, so any point on this axis is projected to the center of the screen.
rotateY(myp5.PI / 2) rotates the point (50, 0, 0) 90 degrees around the Y-axis.
This rotation transforms the point (50, 0, 0) into (0, 0, -50). After rotation, the point is 50 units in front of the origin along the negative Z-axis. After the rotation, the point (0, 0, -50) is at a distance of 850 units from the camera (800 - (-50)). The screen coordinates are calculated based on the projection of this point into the 2D view of the camera.
Since the rotated point lies directly on the Z-axis, its X and Y screen coordinates should map to the center of the canvas, so --> 200,200

What are your thoughts on this @davepagurek please let me know?!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I definitely didn't catch that this was a rotateY -- I read it as just rotate, which would have been around the z axis. That now explains why the local offset doesn't update the resulting screen x and y values. The 50,50 result now makes sense, since I think the canvas is only 100x100:

myp5.createCanvas(100, 100);

So I think this result does actually make sense, and maybe we should just add one more test of rotation about the Z axis so that we can confirm that an example like the one you use in the 2D tests also works in WebGL.

myp5.translate(50, 0, 0);
let worldPos = myp5.createVector(50, 0, 0);
let screenPos = myp5.worldToScreen(worldPos);
myp5.pop();
assert.closeTo(screenPos.x, 100, 0.1);
assert.closeTo(screenPos.y, 50, 0.1);
});
});
});
Loading