-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
About the loftr result #40
Comments
@sanersbug Can you add your pics here? |
|
I have checked both the cpp code and original python code. For the indoor_new_ds weights, indeed the number of matched points is zero. However, for the outdoor_weights, LoFTR performed well as you can see in the following pic. (The following pic is from python code). Sadly, I doubt there is still some bug in loftr that makes the conversion of the outdoor weight unsuccessful. Hence we can not use the old outdoor weight for my C++ program. |
LISRD is actually a descriptor model, so somehow it works like SuperPoint (without SuperGlue). Though the inference part of LISRD is more complicated than SuperPoint. However, as many have asked for LISRD code, I may see what I can do later. |
It seems that LISRD works better, and the author of LISRD says that LISRD is faster. @xmba15 |
LoFTR and SuperGlue are scale-insensitive |
That's it, and I have lisrd onnx model, but when i try to write C++ program about lisrd, it failed. |
The application of Superglue and LoFTR in 3D comparison is carried out in point cloud point cloud point cloud point cloud or image reconstruction process, even if there is a large amount of occlusion between Matching and position estimation are performed on indoor and indoor and mid-level and mid-level modeling local and global features, so as to obtain better 3D point cloud and scene reconstruction results. In general, these two algorithms are advanced image registration and 3D reconstruction techniques, which can achieve brilliant results in a variety of scenarios. LISRD works better in the field of object tracking. The LISRD algorithm is a fast online multi-target tracking algorithm with high accuracy and real-time performance. The LISRD algorithm uses coupled single-model technology to simultaneously estimate and update all target states and corresponding appearance models during each frame tracking process, thereby avoiding the problem of model matching and selection in traditional multi-model tracking algorithms and improving tracking accuracy Spend. In addition, the LISRD algorithm also introduces a novel sparse representation-based spatio-temporal constraint for filtering and smoothing the target trajectory, which reduces unnecessary false alarms and jumps while maintaining tracking accuracy. In general, the LISRD algorithm not only achieves good results in the field of target tracking, but also has broad application prospects in other application fields (such as video processing, human-computer interaction, etc.). @sanersbug @xmba15 |
@xmba15 Have you try to make the LISRD code? In the post-processing , i can't understand the extract_descriptors function, especially the use of torch.nn.functional.grid_sample in grid_sample(descriptors[k], grid_points), dim=1). |
@sanersbug @xmba15 This is the translated LISRD I wrote on the basis of the @xmba15 's library, but the effect is a little worse than python, the @xmba15 can continue on my basis. include <torch/torch.h> using namespace std; std::vectorcv::KeyPoint getKeyPointFromSift(const cv::Mat& img, int nfeatures, float contrastThreshold) std::vectorcv::KeyPoint getKeyPoints(const std::vectorOrt::OrtSessionHandler::DataOutputType& inferenceOutput, cv::Mat detectorMat(detectorShape.size(), detectorShape.data(), CV_32F, inferenceOutput[0].first); // 65 x H/8 x W/8 cv::Mat buffer; for (int i = 0; i < detectorShape[1]; ++i) std::vectorcv::KeyPoint keyPoints; std::vector nmsFast(const std::vectorcv::KeyPoint &keyPoints, int height, int width, int distThresh = 4)
} std::vectorcv::KeyPoint nmsFast_good(std::vectorcv::KeyPoint& in_corners, int H, int W, int dist_thresh = 4) torch::Tensor keyPointsToGrid(const std::vectorcv::KeyPoint& in_keypoints, const cv::Size& img_size) std::pair<torch::Tensor, torch::Tensor> torch::Tensor lisrdMatcher(torch::Tensor desc1, torch::Tensor desc2, torch::Tensor meta_desc1, torch::Tensor meta_desc2) std::pair<std::vectorcv::KeyPoint, std::vectorcv::KeyPoint> void plot_keypoints(const cv::Mat &img, const std::vectorcv::KeyPoint& kpts, const std::vectorcv::Scalar& colors, float ps) int main() //-----------------------------------------------------// Ort::SuperPoint superpoint2(superpoint_model_path, 0, std::vector<std::vector<int64_t>>{ // std::ofstream file1 ("kp_sp_cpp1.txt"); float ps = 2; //-----------------------------------------------------// Ort::Lisrd lisrd2(lisrd_model_path, 0,std::vector<std::vector<int64_t>>{ std::pair<torch::Tensor, torch::Tensor> result1 = extractDescriptors(lisrd_output1, keypoint1, rgb1.size()); std::ofstream file1("desc1.txt"); std::ofstream file3("desc2.txt"); torch::Tensor matches = lisrdMatcher(result1.first, result2.first, result1.second, result2.second); //-------------------------------------------------------------// //-------------------------------------------------------------// std::vectorcv::DMatch matches_info; cv::Mat matchesImage; return EXIT_SUCCESS; ` |
@xmba15 @sanersbug @WithAIGC @ChmarsLuo @yeluo80808 Any one get good result about lisrd ? it's too hard using C++ make it. |
This is only hobby repository so I can only focus on it on my free time. It might take some time until I can get to do it. @sanersbug https://github.com/xmba15/onnx_runtime_cpp/blob/master/examples/Utility.hpp#L197 |
@yeluo80808 Thank you for the code. I will use it as reference when I explore LISRD. |
ok,thanks, I didn't notice it before |
It turns out that besides me, there are so many people using LISRD. Looking forward to it. |
@yeluo80808 I have tried the code, there is indeed a big problem with this result. Have you tried that ? how to solve it ? @xmba15 |
Has LISRD had any results and do you guys have any good news to share? @sanersbug @xmba15 @WithAIGC @yeluo80808 |
@xmba15 Have you tried the LISRD ? No one has made it for such a long time...... |
Is there any progress? |
Sorry, I am quite busy now so currently I have no specific plan to support this in a short term. If you want to accelerate the development, please consider creating an at least runnable PR yourself in this repository. I will take a look and improve from it. |
I have tested the loftr,but i find some problem, when i use two image which have same size , the result is normal,but when i use a big picture and a small picture ,it detect nothing!
The text was updated successfully, but these errors were encountered: