Robot vision

In this dissertation, visual cues using an active monocular camera for autonomous vehicle navigation are investigated. A number of visual cues suitable to such an objective are proposed and effective methods to extract them are developed. Unique features of these visual cues include: (1) There is no need to reconstruct the 3D scene; (2) they utilize short image sequences taken by a monocular camera; and (3) they operate on local image brightness information. Taking these features into account, the algorithms developed are computationally efficient. Simulation and experimental studies confirm the efficacy of the algorithms developed. The major contribution of the research work in this dissertation is the extraction of visual information suitable for autonomous navigation in an active monocular camera without 3D reconstruction by use of local image information. In the studies addressed, the first visual cue is related to camera focusing parameters. An objective function relating focusing parameters to local image brightness is proposed. A theoretical development is conducted to show that by maximizing the objective function one can focus successfully the camera by choosing the focusing parameters. As a result, the dense distance map between a camera and a front scene can be estimated without using the Gaussian spread function. The second visual cue, namely, the clearance invariant (first proposed by Raviv (97)), is extended here to include arbitrary translational motion of a camera. It is shown that the angle between the optical axis and moving direction of a camera can be estimated by minimizing the relevant estimated error residual. This method needs only one image projection from a 3D surface point at an arbitrary time instant. The third issue discussed in this dissertation refers to extracting the looming and the magnitude of rotation using a new visual cue designated as the rotation invariant under the camera fixation. An algorithm to extract the looming is proposed using the image information available from only one 3D surface point at an arbitrary time instant. Further, an additional algorithm is proposed to estimate the magnitude of rotational velocity of the camera by using the image projections of only two 3D surface points measured over two time instants. Finally, a method is presented to extract the focus of expansion robustly without using image brightness derivatives. It decomposes an image projection trajectory into two independent linear models, and applies the Kalman filters to estimate the focus of expansion.

This dissertation deals with novel vision-based motion cues called the Visual Threat Cues (VTCs), suitable for autonomous navigation tasks such as collision avoidance and maintenance of clearance. The VTCs are time-based and provide some measure for a relative change in range as well as clearance between a 3D surface and a moving observer. They are independent of the 3D environment around the observer and need almost no a-priori knowledge about it. For each VTC presented in this dissertation, there is a corresponding visual field associated with it. Each visual field constitutes a family of imaginary 3D surfaces attached to the moving observer. All the points that lie on a particular imaginary 3D surface, produce the same value of the VTC. These visual fields can be used to demarcate the space around the moving observer into safe and danger zones of varying degree. Several approaches to extract the VTCs from a sequence of monocular images have been suggested. A practical method to extract the VTCs from a sequence of images of 3D textured surfaces, obtained by a visually fixation, fixed-focus moving camera is also presented. This approach is based on the extraction of a global image dissimilarity measure called the Image Quality Measure (IQM), which is extracted directly from the raw data of the gray level images. Based on the relative variations of the measured IQM, the VTCs are extracted. This practical approach to extract the VTCs needs no 3D reconstruction, depth information, optical flow or feature tracking. This algorithm to extract the VTCs was tested on several indoor as well as outdoor real image sequences. Two vision-based closed-loop control schemes for autonomous navigation tasks were implemented in a-priori unknown textured environments using one of the VTCs as relevant sensory feedback information. They are based on a set of IF-THEN fuzzy rules and need almost no a-priori information about the vehicle dynamics, speed, direction of motion, etc. They were implemented in real-time using a camera mounted on a six degree-of-freedom flight simulator.

This research introduces a unified approach to visual looming. Visual looming is related to an increasing projected size of an object on a viewer's retina while the relative distance between the viewer and the object decreases. Psychophysicists and neurobiologists have studied this phenomenon by observing vision and action in unison and have reported subject's tendency to react defensively or using this information in an anticipatory control of the body. Since visual looming induces senses of threat of collision, the same cue, if quantified, can be used along with visual fixation in obstacle avoidance in mobile robots. In quantitative form visual looming is defined as the time derivative of the relative distance (range) between the observer and the object divided by the relative distance itself. The visual looming is a measurable variable. Following the paradigm of Active Vision the approach in this research uses visual fixation to selectively attend a small part of the image, that is relevant to the task. Visual looming provides a time-based mapping from a "set of 2-D image cues" to "time-based 3-D space". This research describes how visual looming, which is a concept related to an object in the 3-D world, can be calculated studying the relative temporal change in the following four different attributes of a sequence of 2-D images: (i) image area; (ii) image brightness; (iii) texture density in the image; (iv) image blur. From a simple closed form expression it shows that a powerful unified approach can be adopted in these methods. An extension of this unified approach establishes a strong relationship with the Weber-Fechner law in Psychophysics. The four different methods explored for the calculation of looming are simple. The experimental results illustrate how the measured values of looming stay close to the actual values. This research also introduces one important visual invariant $\Re$ that exists in relative movements between a camera light-source pair and a visible object. Finally, looming is used in the sense of a threat of collision, to navigate in an unknown environment. The results show that the approach can be used in real-time obstacle avoidance with very little a-priori knowledge.

Robot calibration using a vision system and moving cameras is the focus of this dissertation. The dissertation contributes in the areas of robot modeling, kinematic identification and calibration measurement. The effects of perspective distortion of circular camera calibration points is analyzed. A new modified complete and parametrically continuous robot kinematic model, an evolution of the complete and parametrically continuous (CPC) model, is proposed. It is shown that the model's error-model can be developed easily as the structure of this new model is very simple and similar to the Denavit-Hartenbert model. The derivation procedure of the error-model follows a systematic method that can be applied to any kind of robot arms. Pose measurement is the most crucial step in robot calibration. The use of stereo as well as mono mobile camera measurement system for collection of pose data of the robot end-effector is investigated. The Simulated Annealing technique is applied to the problem of optimal measurement configuration selection. Joint travel limits can be included in the cost function. It is shown that trapping into local minimum points can be effectively avoided by properly choosing an initial point and a temperature schedule. The concept of simultaneous calibration of camera and robot is developed and implemented as an automated process that determines the system model parameters using only the system's internal sensors. This process uses a unified mathematical model for the entire robot/camera system. The results of the kinematic identification, optimal configuration selection, and simultaneous calibration of robot and camera using the PUMA 560 robot arm have demonstrated that the modified complete and parametrically continuous model is a viable and simple modeling tool, which can achieve desired accuracy. The systematic way of modeling and performing of different kinds of vision-based robot applications demonstrated in this dissertation will pave the way for industrial standardizing of robot calibration done by the robot user on the manufacturing floor.