Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pose convertion, resize and camera principal point #45

Open
bring728 opened this issue Jan 27, 2022 · 0 comments
Open

pose convertion, resize and camera principal point #45

bring728 opened this issue Jan 27, 2022 · 0 comments

Comments

@bring728
Copy link

bring728 commented Jan 27, 2022

  1. I was curious about the nerf_pose_to_ours function, and I read the article below. But there are still some things I don't understand. Pose convention of nerf_pose_to_ours. #13

As I understand the Pose value is the Camera to world matrix, and each column represents the x-axis, y-axis, z-axis, and location of the camera in the world coordinate system.

If I want to change the coordinate axis of the camera from the opengl coordinate system (right, up, backward) to the opencv coordinate system (right, down, forward), the pose values [r1, r2, r3, t] are set to [r1, -r2, -r3, t], isn't it? Why does Translation change? Isn't the world coordinate system fixed?

The poses_bounds.npy file stores the camera coordinate axes as (down, right, backward). When you change this from NeRF to OpenGL coordinate system (right, up, backward), don't you do it like the following? Why is the method of changing the coordinate axes in the nerf_to_our_pose function different from the method of changing the coordinate axes below?

nex-code/utils/load_llff.py

Lines 245 to 254 in eeff38c

def load_llff_data(basedir, factor=8, recenter=True, bd_factor=.75, spherify=False, path_zflat=False, split_train_val = 0, render_style=''):
# poses, bds, imgs = _load_data(basedir, factor=factor) # factor=8 downsamples original imgs by 8x
poses, bds, intrinsic = _load_data(basedir, factor=factor, load_imgs=False) # factor=8 downsamples original imgs by 8x
print('Loaded', basedir, bds.min(), bds.max())
# Correct rotation matrix ordering and move variable dim to axis 0
#poses [R | T] [3, 4, images]
poses = np.concatenate([poses[:, 1:2, :], -poses[:, 0:1, :], poses[:, 2:, :]], 1)

Doesn't the world coordinate system matter whether it is opencv convention or opengl convention? Isn't the world coordinate system determined independently? Maybe it's because the nerf_to_our_pose function is located after recenter..? I'm confused.

  1. It makes sense to multiply the focal length by the scale factor when resizing the image. By the way, why add 0.5 to the principal point, multiply, and subtract 0.5 again? Can't we just multiply by sw? I'm curious about the hidden meaning here.

nex-code/utils/sfm_utils.py

Lines 188 to 191 in eeff38c

cam['fx'] = ocam['fx'] * sw
cam['fy'] = ocam['fy'] * sh
cam['px'] = (ocam['px']+0.5) * sw - 0.5
cam['py'] = (ocam['py']+0.5) * sh - 0.5

@bring728 bring728 changed the title about opengl -> opencv pose convertion pose convertion, resize and camera principal point Jan 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant