-
Notifications
You must be signed in to change notification settings - Fork 27
/
Code_Explanation.txt
122 lines (91 loc) · 5.28 KB
/
Code_Explanation.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
Derek Hoiem
July 31, 2014
This contains updated instructions for version 2 of the analysis code. You
may also want to consult the README (created for version 1) and the pdf
description, as well as the original paper:
D. Hoiem, Y. Chodpathumwan, and Q. Dai,
"Diagnosing Error in Object Detectors", ECCV 2012.
------------------------------------------------------------------------------
The main script is detectionAnalysisScript. Full functionality applies to
only the VOC2007 dataset because detailed annotations are available only
on that dataset. However, most functionality, including full false positive
analysis also applies to other VOC detection datasets. The code has also
been written to make it easy to adapt to other detection datasets that are
based on bounding box criteria. It may be applicable to datasets for other
localization criteria, but more revision may be necessary.
MAIN FUNCTIONS
---- Functions that likely require editing ----
detectionAnalysisScript: Script for running all code.
The first few lines contain parameters specifying what analysis and display
functions to call. The dataset type (e.g., 'VOC'), list of detectors to
analyze, and list of object classes to analyze is also specified here.
setDatasetParameters: Specifies specific datset (e.g.,
'VOC2007') and paths for images and annotations. Some parameters are
specific to the machine; others to the type of dataset. The list of
generally required parameters are explained in the function documentation.
setDetectorInfo: Specifies location of detection files
for each detector and directory to place results.
---- Functions that require editing if not using a VOC dataset ----
readDatasetAnnotations: For maximum compatibility (similar to say VOC2010),
if using another dataset, this should produce the following structure:
ann.
rec(image_number).
filename: name of image file
imgsize([w h]): width and height of image in pixels
objects(num_objects).
detailedannotation: true if detailed annotation is available
(probably false if not using VOC2007)
details: annotations for visible parts, etc., if available, empty if not
truncated: 0/1 for whether truncated
difficult: 0/1 for whether this object is a don't care
gt(object_class_number).
bbox(num_objects, [x1 y1 x2 y2]): ground truth bounding boxes
rnum(num_objects): index into rec, specifying source image
onum(num_objects): index into rec(rnum).objects, specifying which object
isdiff(num_objects): whether each object is don't care
istrunc(num_objects): whether each object is truncated
isocc(num_objects): whether each object is occluded
details(num_objects): detail structure if available (otherwise empty)
N: number of ground truth objects
Note that isdiff, isocc, istrunc can be set to zeros if that information is
not available; likewise trunacted and difficult in rec can be set to 0.
readDetections: Reads the output of the object detector. For maximum
compatibility, this should produce the following structure:
det.
bbox(ndetections, [x1 y1 x2 y2]): detection bounding box
conf(ndetections): confidence for each detection (higher is more confident)
rnum(ndetections): index into rec/gt structures, specifying source image
nimages: total number of images in dataset
N: total number of detections
If these functions can be written according to the above specifications
(which should be true for most bounding box based tasks), set dataset
in the main script to 'voc_compatible'.
---- Functions that may require editing if not using a VOC-compatible dataset ----
analyzeTrueDetections: Produces precision-recall curves and normalized PR
curves for objects with various characteristics.
matchDetectionsWithGroundTruth: Labels detections as incorrect, correct, or
don't care. localization parameter specifies whether localization error
should be ignored ('weak') or not ('strong'). If you want to change the
localization criteria, edit setDatasetParameters and/or this function. Also
assigned ground truth objects to detections with best score or overlap.
analyzeFalsePositives: Assigns detections to false positive types and
computes effects of each type of false positive.
displayPerCharacteristicPlots: Draws plots comparing normalized-AP for
various object characteristics
displayAverageSensitivityImpactPlot: Averages across objects to show impact
of various object characteristics
writeMissedObjectCharacteristics: summarizes statistics for objects that are
not detected
displayGtConfidencePredictions: displays detected objects sorted by
the difference of predicted confidence (based on characteristics) and
detection confidence (displays unexpectedly high confidence detections)
showSurprisingMisses: similar to above, but displays unexpectedly low
confidence detections
displayTopFP: displays and optionally saves most confident false positives
displayFalsePositiveImpactPlot: creates pie/bar charts for impact of
false positives on performance
displayFPTrend: displays stacked area plots of false positive with
increasing number of detections
writeFPAnalysisSummary: creates text file summarizing FP analysis
writeTexHeader: creates tex header file for creating pdf report
writeTexObject: creates tex material for results for one object category