-
Notifications
You must be signed in to change notification settings - Fork 0
/
Calibration.tex
71 lines (54 loc) · 8.47 KB
/
Calibration.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
\label{c:Experiments}
The experiments chapter first describes the final experimental pipeline, which is divided into the Stereo Camera Calibration (\autoref{sec:Calibration}) and into the actual reconstruction (\autoref{sec:Reconstruction}) and secondly follows several experimental sessions to point out problems and results which occurred during the tests.
\section{Stereo Camera Calibration}\label{sec:Calibration}
The stereo camera calibration estimates the intrinsic and extrinsic parameters of a set-up of two uncalibrated cameras. The calibration is done in two steps, first image pairs of the checkerboard pattern are captured (\autoref{ssec:PatternSequence}), then MATLAB's \textit{Stereo Camera Calibrator Toolboox}\index{Stereo Camera Calibrator Toolbox} is used to estimate the parameters (\autoref{ssec:estimateStereoParams}). The camera set-up used in the camera calibration must not be changed at all for the rest of the session: the sequence recorded for the actual 3-D reconstruction needs to have the same parameters.
\begin{figure}[htbp]
\centering
\includegraphics[width=0.7\textwidth]{figures/CameraCalibration}
\caption[Steps of the stereo camera calibration]{Steps of the stereo camera calibration in TimeBench and in the Stereo Camera Calibrator for MATLAB, using $n$ image pairs.}
\label{fig:stereoCamCalib}
\end{figure}
\subsection{Capturing Sequence of Calibration Pattern in TimeBench}\label{ssec:PatternSequence}
The two cameras should be set-up next to each other with little to no rotation. For stereo display (anaglyphic display) the cameras need to be placed about 55 mm apart, which is about the distance between our eyes. Since this thesis does not create stereoscopic displays, the cameras' baseline is set to approximately 19 cm, which is the minimum distance the two cameras can be apart from each other due to the camera bodies\footnote{Cameras which are placed farther apart from each other resolve in greater reconstruction accuracy.} (\cite{StereoCalib.2016}).
TimeBench should only be started after the two high-speed cameras have already been connected to the computer, otherwise the cameras may not be recognized by the software. After the hardware was installed (see \autoref{sec:architecture} for the cable set-up), the two cameras need to be dragged into a \textit{synchronization group} and the following configurations are recommended (\autoref{fig:timebanchRecord} shows a screenshot of the connected and synchronized cameras in Timebench):
\begin{enumerate}[i]
\item Both cameras: set the frame rate to 70 fps\footnote{The smaller frame rate gives more time for positioning the calibration pattern in different angles, since the data amount is decreased to the lower frame rate.}.
\item Right camera: check the box \enquote{Is Master} to set the right camera as the synchronization master.
\item Both cameras: set the exposure time slightly shorter than 1/frame rate.
\item Right camera: do not forget to white balance.
\item Save images as \textit{bmp}.
\end{enumerate}
\begin{figure}[htbp]
\centering
\includegraphics[width=1.0\textwidth]{figures/timebenchRecord}
\caption[The views of two synchronized cameras and their capturing options in TimeBench]{The views of two synchronized cameras and their capturing options in TimeBench (\textit{source: software TimeBench owned by} \cite{Optronis.2016}).}
\label{fig:timebanchRecord}
\end{figure}
Once the cameras are synchronized and configured the capturing of the calibration images can begin. The checkerboard pattern should be hold roughly at the same distance the objects will be placed in the actual scene. It needs to be in focus and must be fully visible in the view of both cameras. After starting the capturing sequence by clicking \enquote{Start} in the software and also activating the external trigger, the checkerboard pattern needs to be moved in different angles till the sequence stops. Out of the recorded image sequence 10-20 different image pairs have to be chosen (\autoref{fig:timebenchSequence} shows the timeline with the captured images of a session). Extreme angles are not likely to be detected later in MATLAB (\cite{StereoCalib.2016}).
\subsection{Estimating Stereo Parameters in MATLAB}\label{ssec:estimateStereoParams}
The selected image pairs are then loaded into the Stereo Camera Calibration Toolbox in MATLAB. They can be added by clicking \textbf{Add images} in the app. In the final experimental architecture the left camera is set as camera one and the right camera as camera two. For this step it is important to know which size the squares in the checkerboard pattern are, in this case 3 mm. The images then get analyzed by the software. This process can take some time depending on the amount and resolution of the images in the set. The result window shows how many stereo pairs were added and how many were rejected. The rejection can be caused by several factors, which will be discussed and summarized in \autoref{sec:ExperimentalResults}. The data browser displays a list of all added image pairs, which should be checked to insure that they are matched with their corresponding image partners. Clicking on a pair of images will display the detected points in a green color and, these points may be checked more closely using the zoom function (\cite{StereoCalib.2016}).
After reviewing the accepted data, the button \textbf{Calibrate} will start the camera calibration. A diagram shows the overall \textbf{reprojection errors} in pixels\index{Reprojection error} from all images, which is the difference between the detected and the reprojected points, as shown in \autoref{fig:ReprojectionError}. The error should not be higher than 1 pixel but is recommended to be below 0.5 pixel. Outliers, which means errors much higher than the acceptable value, can be selected by moving the red horizontal line. The respective images can then be deleted in the data browser. After that \enquote{Calibrate} needs to be clicked again (\cite{StereoCalib.2016}).
\begin{figure}[htbp]
\centering
\includegraphics[width=0.9\textwidth]{figures/ReprojectionError}
\caption[Calculation of the reprojection error]{Calculation of the reprojection error (\textit{source:} \cite{StereoCalib.2016}).}
\label{fig:ReprojectionError}
\end{figure}
The bottom right window shows the estimated \textbf{extrinsic parameters}\index{Extrinsic camera parameters} in a \textit{camera-centric view}. If the pattern was stationary and the cameras were moved, the plot can be changed to a \textit{pattern-centric view}. This plot should be examined for whether the relative poses of the cameras, the baseline and the checkerboard positions are matching the expected positions.
Another factor to be reviewed is the \textit{rectification}. By clicking on \textbf{Show rectified} in the middle of the screen, the rectified image pair should be shown. If the rectified images are still distorted and not row-aligned the calibration was not accurate enough. This problem is further discussed in the session examples and in the summarized results (\autoref{sec:ExperimentalResults}).
\begin{figure}[htbp]
\centering
\includegraphics[width=1.0\textwidth]{figures/ExRectified}
\caption[Calibrated camera system in MATLAB]{Calibrated camera system in the Stereo Calibrator with: \textbf{(left)} data browser with set of accepted image pairs; \textbf{(middle)} rectified images of one image pair; \textbf{(bottom left)} reprojection errors; \textbf{(bottom right)} estimated extrinsic parameters.}
\label{fig:ExRectified}
\end{figure}
The last step is to export the calibrated stereo camera parameters. \textbf{Export camera parameters} creates a \textit{stereoParameters object} in MATLAB's workspace and \textbf{Generate MATLAB script} saves the calibration steps of the session as well.
\subsection{The stereoParameters Object}\label{ssec:stereoParamsObj}
The stereoParameters object\index{StereoParameters object} stores the intrinsic\index{Intrinsic camera parameters} and extrinsic camera parameters\index{Extrinsic camera parameters} as well as the lens distortion (\cite{StereoCalib.2016}).
\begin{figure}[htbp]
\centering
\includegraphics[width=1.0\textwidth]{figures/StereoParams}
\caption[The stereoParameter object of session 14 in MATLAB]{The stereoParameter object of session 14 in MATLAB.}
\label{fig:StereoParams}
\end{figure}
Sine the experiments resulted in a rectification problem (see \autoref{sec:ExperimentalResults}), the fundamental matrix computed by the Stereo Camera Calibrator needed to be evaluated. The MATLAB file \textit{testEssentialandFundamentalMatrix.m} addresses this problem.