Raviv, Daniel

This dissertation deals with novel vision-based motion cues called the Visual Threat Cues (VTCs), suitable for autonomous navigation tasks such as collision avoidance and maintenance of clearance. The VTCs are time-based and provide some measure for a relative change in range as well as clearance between a 3D surface and a moving observer. They are independent of the 3D environment around the observer and need almost no a-priori knowledge about it. For each VTC presented in this dissertation, there is a corresponding visual field associated with it. Each visual field constitutes a family of imaginary 3D surfaces attached to the moving observer. All the points that lie on a particular imaginary 3D surface, produce the same value of the VTC. These visual fields can be used to demarcate the space around the moving observer into safe and danger zones of varying degree. Several approaches to extract the VTCs from a sequence of monocular images have been suggested. A practical method to extract the VTCs from a sequence of images of 3D textured surfaces, obtained by a visually fixation, fixed-focus moving camera is also presented. This approach is based on the extraction of a global image dissimilarity measure called the Image Quality Measure (IQM), which is extracted directly from the raw data of the gray level images. Based on the relative variations of the measured IQM, the VTCs are extracted. This practical approach to extract the VTCs needs no 3D reconstruction, depth information, optical flow or feature tracking. This algorithm to extract the VTCs was tested on several indoor as well as outdoor real image sequences. Two vision-based closed-loop control schemes for autonomous navigation tasks were implemented in a-priori unknown textured environments using one of the VTCs as relevant sensory feedback information. They are based on a set of IF-THEN fuzzy rules and need almost no a-priori information about the vehicle dynamics, speed, direction of motion, etc. They were implemented in real-time using a camera mounted on a six degree-of-freedom flight simulator.

This research introduces a unified approach to visual looming. Visual looming is related to an increasing projected size of an object on a viewer's retina while the relative distance between the viewer and the object decreases. Psychophysicists and neurobiologists have studied this phenomenon by observing vision and action in unison and have reported subject's tendency to react defensively or using this information in an anticipatory control of the body. Since visual looming induces senses of threat of collision, the same cue, if quantified, can be used along with visual fixation in obstacle avoidance in mobile robots. In quantitative form visual looming is defined as the time derivative of the relative distance (range) between the observer and the object divided by the relative distance itself. The visual looming is a measurable variable. Following the paradigm of Active Vision the approach in this research uses visual fixation to selectively attend a small part of the image, that is relevant to the task. Visual looming provides a time-based mapping from a "set of 2-D image cues" to "time-based 3-D space". This research describes how visual looming, which is a concept related to an object in the 3-D world, can be calculated studying the relative temporal change in the following four different attributes of a sequence of 2-D images: (i) image area; (ii) image brightness; (iii) texture density in the image; (iv) image blur. From a simple closed form expression it shows that a powerful unified approach can be adopted in these methods. An extension of this unified approach establishes a strong relationship with the Weber-Fechner law in Psychophysics. The four different methods explored for the calculation of looming are simple. The experimental results illustrate how the measured values of looming stay close to the actual values. This research also introduces one important visual invariant $\Re$ that exists in relative movements between a camera light-source pair and a visible object. Finally, looming is used in the sense of a threat of collision, to navigate in an unknown environment. The results show that the approach can be used in real-time obstacle avoidance with very little a-priori knowledge.

This dissertation deals with vision-based perception-action closed-loop control systems based on 2-D visual cues. These visual cues are used to calculate the relevant control signals required for autonomous landing and road following. In the landing tasks it has been shown that nine 2-D visual cues can be extracted from a single image of the runway. Seven of these cues can be used to accomplish parallel flight and glideslope tracking tasks of the landing. For the road following task, three different algorithms based on two different 2-D visual cues are developed. One of the road following algorithms can be used to generate steering and velocity commands for the vehicle. Glideslope tracking of the landing task has been implemented in real-time on a six-degree-of-freedom flight simulator. It has been shown that the relevant information computed from 2-D visual cues is robust and reliable for the landing tasks. Road following algorithms were tested successfully up to 50km/h on a US Army High Mobility and Multipurpose Wheeled Vehicle (HMMWV) equipped with a vision system and on a Denning mobile robot. The algorithms have also been tested successfully using PC-based software simulation programs.