-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trouble reproducing ground truth results #159
Comments
I met the same problem too. And I am so confused about it.I have got this. But I can't get the right metrics....@junyanz Could I get some suggestions? |
Hi, please refer to the notes under "Evaluating Labels2Photos on Cityscapes" in the README. |
If anyone else encountered this problem - this is due to evaluating the test set and not the train or validation set. In the Cityscapes dataset the test set doesn't have the same label format and meaning as in the other splits. |
@erthher Excuse me . I am so curious about the solution of the problem ? Could I get your email and ask some questions ,plz ? |
Hello @erthher , @tinghuiz |
@KaitiSt As mentioned in Section 6.2 "Training details" of the paper, we used the Cityscapes validation set for testing. |
Hello @tinghuiz , I tried different scenarios in order to make the evaluation work . Based on the evaluation code the predicted images will be resized from 256 x 256 to the label size which is 1024 x 2048 and push them into the segmentation network. If I keep the code as it is i am getting the below exception : F0415 12:42:11.680351 10204 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0) out of memory Based on your comment , I can understand that the pre trained model does not work well with 1024 x 2048. So if i remove the resize of the input image to 1024 x 2048 , the segmentation is done , but it's failing in the fast_hist method with the below exception: IndexError: index 65553 is out of bounds for axis 0 with size 65536 But, as per your your comment once more , if I resize the original cityscape image - label to 256 x 256 the evaluation is working. The results though are not so good |
@KaitiSt did you ever solve this last problem? We're facing the same issue with the entire image being marked as road (purple). |
Hi,
I am trying to use the evaluation script to reproduce the ground truth results.
The label has a shape of (1024, 2048, 3) but the segmentation result has a shape of (1024, 2048). As a result the fast_hist function throws the following error:
IndexError: index 2097152 is out of bounds for axis 1 with size 2097152
.If I try to select only one of the label channels or use np.repeat to stack the segmentation result with itself I get very poor results for the ground truth (Mean Pixel accuracy < 0.04).
Is that the intended behavior of the script?
Thanks in advance.
The text was updated successfully, but these errors were encountered: