geo2raw.get_img_coords_dict sometimes returns wrong coordinates

[Seo-4d696b75]

Hi. I'm now trying to convert geo coordinates of trees into pixel coordinated on each Pix4d raw image (images token by a drone). The coordinates of trees are given as point cloud, and all the points are transformed by function geo2raw.get_img_coords_dict . Then, rectangle bounding boxes, which include the transformed points inside, are calculated as below:

from EasyIDP.easyric.objects import Pix4D
from EasyIDP.easyric.calculate import geo2raw
import numpy as np

p4d = Pix4D(..params..)

# load point cloud from csv file (x,y,z)
points = np.loadtxt(file_src, delimiter=",")
# convert all the points to each img pixel-coordinate
pix2d_dict = geo2raw.get_img_coords_dict(p4d, points, method='pmat')
# calc rect box
for img_name, pixels in pix2d_dict.items():
    pixel_x = pixels[:, 0]
    pixel_y = pixels[:, 1]
    min_x = pixel_x.min()
    min_y = pixel_y.min()
    max_x = pixel_x.max()
    max_y = pixel_y.max()
    box = np.array([min_x, min_y, max_x, max_y])
    np.savetxt(file_dst, box, delimiter=",")

When I visualized those transformed boxes on each drone image, I found some boxes are projected to pixels coordinates far from original trees. Here is an example.

Many of boxes seems to be transformed to right pixel coordinates and red rectangles are just overlapped on trees in image. But boxes with id:172 and id:144 are located on coordinates where no trees are found.

When I check those point clouds with id:172 and 144 by CloudCompare, they are located far from the region of the above image (clouds colored red are id:172 and 144).

Does anyone has any idea for this problem?

[HowcanoeWang]

It is an existing bug with pix4d pmatrix calculation, it sometimes tends to project plot on another unrelated images very far away.

The solution I currently used to deal with it, is adding geo2raw.filter_cloest_img() function to judge whether the camera geo position is just above plot geo position.

pix2d_dict = geo2raw.get_img_coords_dict(p4d, points, method='pmat')
pix2d_dict_filter = geo2raw.filter_cloest_img(p4d, pix2d_dict, points, num=3, dist_thresh=5)

You can either use num or dist_thresh to filter wrong images

• num: Only keep 3 closest images. (If not given, using the following distance filter)
• dist_thresh: filter image-plot geo distance < 5m

I guess in your case, you prefer not missing any tree labels in each image, so the num filter is not applicable. But even using the distance filter, please be very careful about your flight height and camera FOV, the dist_thresh (geo_dist) need changes on different conditions.

(Please check the document Package Usage | Transformer and use plot.draw_polygon_on_imgs to plot image & determine the proper numbers or distances for your case)

PS: This function has been posted into the latest EasyIDP version, you need to download again. Or
you can simply copy and paste the following code directly
(please help me debug if any, I just modify this in mind and not tests on your dateset yet, too large to download)

def filter_closest_img(p4d, img_dict, plot_geo, dist_thresh=None, num=None):
    """[summary]

    Parameters
    ----------
    img_dict : dict
        The outputs dict of geo2raw.get_img_coords_dict()
    plot_geo : nx3 ndarray
        The plot boundary polygon vertex coordinates
    num : None or int
        Keep the closest {x} images
    dist_thresh : None or float
        If given, filter the images smaller than this distance first

    Returns
    -------
    dict
        the same structure as output of geo2raw.get_img_coords_dict()
    """
    dist_geo = []
    dist_name = []
    
    img_dict_sort = {}
    for img_name, img_coord in img_dict.items():

        xmin_geo, ymin_geo = plot_geo[:,0:2].min(axis=0)
        xmax_geo, ymax_geo = plot_geo[:,0:2].max(axis=0)
        xctr_geo = (xmax_geo + xmin_geo) / 2
        yctr_geo = (ymax_geo + ymin_geo) / 2

        ximg_geo, yimg_geo, _ = p4d.img[img_name].cam_pos

        image_plot_dist = np.sqrt((ximg_geo-xctr_geo) ** 2 + (yimg_geo - yctr_geo) ** 2)

        if dist_thresh is not None and image_plot_dist > dist_thresh:
            # skip those image-plot geo distance greater than threshold
            continue
        else:
            # if not given dist_thresh, record all
            dist_geo.append(image_plot_dist)
            dist_name.append(img_name)

    if num is None:
        # not specify num, use all
        num = len(dist_name)
    else:
        num = min(len(dist_name, num))

    dist_geo_idx = np.asarray(dist_geo).argsort()[:num]
    img_dict_sort = {dist_name[idx]:img_dict[dist_name[idx]] for idx in dist_geo_idx}
    
    return img_dict_sort