Computation of all camera parameters by a simultanous minimization process.
camera_calibration performs the calibration of a video camera. Thus, known 3D model points (with coordinates NX, NY, NZ) are projected in the image and the sum of the distance squares between these projections and the corresponding image points (with coordinates NRCoord, NCCoord) is minimized.
In case of converging, the exact internal (CamParam) and external (NFinalPose) camera parameters are determined by this minimization algorithm. The parameters StartCamParam and NStartPose are used as initial values for the minimization process. Since this algorithm simultaneously handles correspondences between image and model points from different images, it is also called multi image calibration.
In general camera calibration means the exact determination of the parameters, which model the (optical) projection of any 3D world point P_W into a (sub-)pixel [r,c] on the CCD sensor of the camera. This is of importance, if the original 3D pose of an object has to be computed using an image (e.g. measuring of industrial pieces).
The underlying camera model is a pinhole camera with radial distortions. It describes the transform of a 3D world point P_W into a (sub-)pixel [r,c] of the video image by the following equations:
P_C = (x,y,z) = R*P_W + T u = Focus * x / z v = Focus * y / z u' = (2*u) / (1+sqrt(1-4*Kappa*(u^2+v^2))) v' = (2*v) / (1+sqrt(1-4*Kappa*(u^2+v^2))) c = u' / Sx + Cx r = v' / Sy + CyThese equations consist of a coordinate transform from the world coordinate system into the camera coordinate system, a perspective projection into the image plane, a radial distortion of the projected point, and finally a sampling and a image center displacement.
The total of 15 camera parameters can be divided into the internal and external camera parameters:
Internal camera parameters: These parameters describe the characterisctic of the used video camera, especially the dimension of the CCD sensor itself and the projection properties of the used combination of lens, camera, and framegrabber. The model of a pinhole camera with radial distortions (as described above) contains the following 8 parameters: Focus: Focal length of the lens. Kappa: Distortion coefficient to model the pillow- respectively barrel-shaped distortions caused by the lens. Sx: Scale factor, corresponds to the horizontal distance between two neighboring cells on the CCD sensor. Attention: This value increases, if the image is subsampled! Sy: Scale factor, corresponds to the vertical distance between two neighboring cells on the CCD sensor. Since in most cases the image signal is sampled line-synchronously, this value is determined by the dimension of the CCD sensor and needn't be estimated by the calibration process. Attention: This value increases, if the image is subsampled! Cx: Column coordinate of the image center point(center of the radial distortion). Cy: Row coordinate of the image center point (center of the radial distortion). ImageWidth: Width of the sampled video image. Attention: This value decreases, if the image is subsampled! ImageHeight: Height of the sampled video image. Attention: This value decreases, if the image is subsampled! External camera parameters: These 7 parameters describe the position and orientation of the camera and are also called as \textbf{camera pose}. The relative position of the camera with regard of a world or object coordinate system is defined by a 3D translational vector (parameters 1,2, and 3). The orientation of the camera is described by the three rotation angles around the three coordinate axes (parameters 4,5, and 6). The last value within the camera pose encodes the representation type of the described 3D transform: Hereby the value 0 says, that the 3D rotation is performed before the 3D translation, that the three rotations are specified as angles (in degree), and that the rotation order is Gamma-Beta-Alpha, i.e., the first rotation is around the Z-axis, the second one around the new Y-axis, and the third one around the new X-axis. Other representation types for the 3D transform are not supported yet. Note, that in contrast to the external camera parameters the internal ones are the same for all positions and orientations of the camera.
The use of camera_calibration leads to some questions, which are dealed with in the following sections:
\textbf{How to generate a appropriate calibration table?}
The simplest method to determine the internal parameter of a CCD camera is the use of the planar calibration table generated by the operator create_caltab. In case of small distances between object and lens it may be sufficient to print the calibration pattern by a laser printer and to mount it on a cardboard. Otherwise -- especially by using a frog-eyed lens -- it is possible to print the PostScript file on a large inkjet printer and to mount it on a aluminium table. It is very important, that the mark coordinates in the calibration table description file correspond to the real ones on the calibration table with high accuracy. Thus, the calibration table description file has to be modified in accordance with the measurement of the calibration table!
\textbf{How to take a set of suitable images?}
If you use the planar calibration table, you can proceed in the following way: With the combination of lens (fixed distance!), camera, and framegrabber to be calibrated a set of images of the calibration table has to be taken, see open_framegrabber and grab_image. The following items have to be considered:
- At least a total of 10 to 20 images should be taken into account. - The calibration table has to be completely visible (incl. border!). - Reflections etc. on the calibration table should be avoided. - Within the set of images the calibration table should appear in different positions and orientations: Once left in the image, once right, once (left and right) at the bottom, once (left or right) at the top, from different distances etc. Thereby the calibration table should be rotated a little around its X- and/or Y-axis, so the perspective distortions of the calibration pattern are clearly visible. Thus, the external camera parameters (camera pose with regard of the calibration table) should be set to a large variety of different values! - The calibration table should fill at least a quarter of the whole image to ensure the robust detection of the marks.
\textbf{How to extract the calibration marks in the images?}
If a planar calibration table is used, for each image the operators find_caltab and find_marks_and_pose can be used to determine the coordinates of the calibration marks and to compute a rough estimate for the external camera parameters. The concatenation of these values can directly used as initial values for the external camera parameters (NStartPose) in camera_calibration.
Obviously, images, on which the segmentation of the calibration table (find_caltab) has failed or the calibration marks haven't been determined successfully by find_marks_and_pose, should be separated.
\textbf{How to find suitable initial values for the internal camera parameters?}
The both operators find_marks_and_pose (determination of initial values for the external camera parameters) and camera_calibration require initial values for the internal camera parameters. These parameters can be provided by a appropriate text file (see read_cam_par), which can be generated by write_cam_par or can be edited manually.
The following should be considered for the initial values of the single parameters:
- Focus: The initial value is the nominal focal length of the the used lens, e.g. 0.008 m. - Kappa: Use 0.0 as initial value. The calibratied value normally lies between -1000.0 and -50000.0 1/(m*m) dependent on the used lens. - Sx: The initial value for the horizontal distance between two neighboring CCD cells depends on the dimension of the used CCD chip of the camera (see technical specifications of the camera). Generally, common CCD chips are either 1/3''-Chips (e.g., SONY XC-73, SONY XC-777), 1/2''-Chips (e.g., SONY XC-999, Panasonic WV-CD50), or 2/3''-Chips (e.g., SONY DXC-151, SONY XC-77). Notice: The value of Sx increases, if the image is subsampled! Appropriate initial values are: Full image (768*576) Subsampling (384*288) 1/3"-Chip 0.0000110 m 0.0000220 m 1/2"-Chip 0.0000086 m 0.0000172 m 2/3"-Chip 0.0000055 m 0.0000110 m The value for Sx is calibrated, since the video signal of a CCD camera normally isn't sampled pixel-synchronously. - Sy: Since most off-the-shelf cameras have quadratic pixels, the same values for Sy are valid as for Sx. In contrast to Sx the value for Sy will NOT be calibrated, because the video signal of a CCD camera normally is sampled line-synchronously. Thus, the initial value is equal to the final value. Appropriate initial values are: Full image (768*576) Subsampling (384*288) 1/3"-Chip 0.0000110 m 0.0000220 m 1/2"-Chip 0.0000086 m 0.0000172 m 2/3"-Chip 0.0000055 m 0.0000110 m - Cx and Cy: Initial values for the coordinates of the image center is the half image width respectively half image height. Notice: The values of Cx and Cy decrease, if the image is subsampled! Appropriate initial values are: Full image (768*576) Subsampling (384*288) Cx 384.0 192.0 Cy 288.0 144.0 - ImageWidth and ImageHeight: These two parameters are set by the the used framegrabber and therefore are not calibrated. Appropriate initial values are: Full image (768*576) Subsampling (384*288) ImageWidth 768 384 ImageHeight 576 288
\textbf{How many camera parameters have to be estimated?}
The input parameter NumParam is used to set the desired number of camera parameters. Usually this parameter is set to 11, i.e., all 6 external camera parameters (translation and rotation) and all internal camera parameters excluding Sy have to be determined. Possible values for NumParam are the following:
- 6: Translation, Rotation - 9: Translation, Rotation, Kappa, Cx, Cy - 10: Translation, Rotation, Kappa, Cx, Cy, Focus - 11: Translation, Rotation, Kappa, Cx, Cy, Focus, Sx
\textbf{What is the order within the single parameters?}
The length of the tuple NStartPose corresponds to the number of calibration images, e.g., using 15 images leads to a length of the tuple NStartPose equal to 15*7=105 (15 times the 7 external camera parameters). The first 7 values correspond to the camera pose of the first image, the next 7 values to the pose of the second one, etc. .
This fixed number of calibration images has to be considered within the tuples with the coordinates of the 3D model marks and the extracted 2D marks. If 15 images are used, the length of the tuples NRCoord and NCCoord is 15 times the length of the tuples with the coordinates of the 3D model marks (NX, NY, and NZ). If every image consists 49 marks, the length of the tuples NRCoord and NCCoord is 49, while the length of the tuples NX, NY, and NZ is 15*49=735. The order of the values in NRCoord and NCCoord is ``image after image'', i.e., using 49 marks the first 3D model point corresponds to the 1., 50., 99., 148., 197., 246. etc. extracted 2D mark.
The 3D model points can be read from a calibration table description file using the operator caltab_points. Initial values for the camera pose can be determined by applying find_marks_and_pose for each image. The tuple NStartPose is set by the concatenation of all these camera poses.
\textbf{What is the meaning of the output parameters?}
If the camera calibration process is finished successfully, i.e., the minimization process is converged, the output parameters CamParam and NFinalPose contain the computed exact values for the internal and external camera parameters. The length of the tuple NFinalPose corresponds to the length of the tuple NStartPose.
The computed average errors (Errors) give an impression of the accuracy of the calibration. The error values (deviations in x- and y-coordinates) are measured in pixels.
\textbf{Must I use a planar calibration table?}
No. The operator camera_calibration is designed in a way, that the input tuples NX, NY, NZ, NRCoord, and NCCoord can contain any 3D/2D correspondences, see the above paragraph explaining the order of the single parameters.
Thus, it makes no difference, how the required 3D model marks and the corresponding extracted 2D marks are determined. On the one hand it is possible to use a 3D calibration pattern, on the other hand you also can use any characteristic points (natural landmarks) with known position in the world. By setting NumParam to 6, it is possible to compute the world position of the camera! Hereby at least three 3D/2D-correspondences are necessary as input. Homogeneous transformation matrices are useful to generate NStartPose, see program example in hom_mat_to_pose.
The minimization process of the calibration depends on the initial values of the internal (StartCamParam) and external (NStartPose) camera parameters. The computed average errors Errors give an impression of the accuracy of the calibration. The error squares (deviations in x- and y-coordinates) are measured in pixels.
NX (input_control) |
real-array -> real |
Ordered Tuple with all X-coordinates of the calibration marks (in meter). |
NY (input_control) |
real-array -> real |
Ordered Tuple with all Y-coordinates of the calibration marks (in meter). |
NZ (input_control) |
real-array -> real |
Ordered Tuple with all Z-coordinates of the calibration marks (in meter). |
NRCoord (input_control) |
real-array -> real |
Ordered Tuple with all row-coordinates of the extracted calibration marks (in pixel). |
NCCoord (input_control) |
real-array -> real |
Ordered Tuple with all column-coordinates of the extracted calibration marks (in pixel). |
StartCamParam (input_control) |
number-array -> real / integer |
Initial values for the internal camera parameters. |
NStartPose (input_control) |
number-array -> real / integer |
Ordered tuple with all initial values for the external camera parameters. |
NumParam (input_control) |
integer -> integer |
Number of the camera parameters to be estimated. | |
Default value: 11 | |
List of values: 6, 9, 10, 11 |
CamParam (output_control) |
number-array -> real / integer |
Internal camera parameters. |
NFinalPose (output_control) |
number-array -> real |
Ordered tuple with all external camera parameters. |
Errors (output_control) |
real-array -> real |
Average error distances in pixels. |
/* read calibration images */ read_image(:Image1:'calib-01.tiff':) > read_image(:Image2:'calib-02.tiff':) > read_image(:Image3:'calib-03.tiff':) > /* find calibration pattern */ find_caltab(Image1:Caltab1:'caltab.descr',3,112:) > find_caltab(Image2:Caltab2:'caltab.descr',3,112:) > find_caltab(Image3:Caltab3:'caltab.descr',3,112:) > /* find calibration marks and start poses */ find_marks_and_pose(Image1,Caltab1::'caltab.descr',[0.008,0.0, 0.000011,0.000011,384,288,768,576],128,10:RCoord1,CCoord1,StartPose1) > find_marks_and_pose(Image2,Caltab2::'caltab.descr',[0.008,0.0, 0.000011,0.000011,384,288,768,576],128,10:RCoord2,CCoord2,StartPose2) > find_marks_and_pose(Image3,Caltab3::'caltab.descr',[0.008,0.0, 0.000011,0.000011,384,288,768,576],128,10:RCoord3,CCoord3,StartPose3) > /* read 3D positions of calibration marks */ caltab_points(::'caltab.descr':NX,NY,NZ) > /* camera calibration */ camera_calibration(::NX,NY,NZ,[RCoord1,RCoord2,RCoord3], [CCoord1,CCoord2,CCoord3],[0.008,0.0,0.000011,0.000011,384,288,768,576], [StartPose1,StartPose2,StartPose3],11:CamParam,NFinalPose,Errors) > /* write internal camera parameters to file*/ write_cam_par(::CamParam,'campar.dat':).
camera_calibration returns TRUE if all parameter values are correct and the desired camera parameters have been determined by the minimization algorithm. If necessary an exception is raised.
find_marks_and_pose, caltab_points, read_cam_par
write_cam_pose, pose_to_hom_mat, disp_caltab, sim_caltab
find_caltab, find_marks_and_pose, disp_caltab, sim_caltab, write_cam_par, read_cam_par, write_cam_pose, read_cam_pose, pose_to_hom_mat, hom_mat_to_pose, caltab_points, create_caltab