Shankar, Ravi

Software simulations of a scaleable VLSI implementable architecture and algorithm for character recognition by a research group at Florida Atlantic University (FAU) have shown encouraging results. We address here hardware implementation issues pertinent to the classification phase of character recognition. Using the digit classification techniques developed at FAU as a foundation, we have designed and simulated general purpose building blocks useful for a possible implementation of a Digital & Analog CMOS VLSI chip that is suitable for a variety of artificial neural network (ANN) architectures. HSPICE was used to perform circuit-level simulations of the building blocks. We present here the details of implementation of the recognition chip including the architecture, circuit design and the simulation results.

In this thesis we report a VLSI design implementation of an application specific, full-frame architecture CCD image sensor for a handwritten Optical Character Recognition system. The design is targeted to the MOSIS 2mu, 2-poly/ 2-metal n-buried channel CCD/CMOS technology. The front side illuminated CCD image sensor uses a transparent polysilicon gate structure and is comprised of 84 (H) x 100 (V) pixels arranged in a hexagonal lattice structure. The sensor has unit pixel dimensions of 18 lambda (H) x 16 lambda (V). A second layer of metal is used for shielding certain areas from incident light, and the effective pixel photosite area is 8 lambda x 8 lambda. The imaging pixels use a 3-phase structure (with an innovative addressing scheme for the hexagonal lattice) for image sensing and horizontal charge shift. Columns of charge are shifted into the vertical 2-phase CCD shift registers, which shift the charge out serially at high speed. The chip has been laid out on the 'tinychip' (2250 mu m x 2220 (mu m) pad frame and fabrication through MOSIS is planned next.

Optical Character Recognition systems have many applications in today's world of electronic computing. Various software implementations are currently being used. This thesis evolves a massively parallel hardware implementation for the system that is VLSI scaleable and may lead to substantial increase in the processing speed. This system involves various stages for preprocessing and processing of the image implemented with SIMD architecture, using simple processing elements and near neighbor communications. The architecture evolved is simulated using the Verilog Hardware Description Language. This project should provide a framework for a massively parallel processing architecture for such systems. It is expected that this project will lead to the design and implementation of a real time system.

Feature extraction for handwritten character recognition has always been a challenging problem for investigators in the field. The problem gets worse due to large variations present for each type of input character. Our algorithm computes directional features for alphanumeric input mapped on to a hexagonal lattice. The algorithm implements size and scale invariance that is a requirement for achieving a reasonably good recognition rate. Functional performance has been verified for an hexagonal lattice mapped input on the data obtained from the US postal service handwritten character database. In this thesis, we implemented the algorithm in a Xilinx FPGA (XC4xxx series).

A novel neural network, trained with the Alopex algorithm to recognize handprinted characters, was developed in this research. It was constructed by an encoded fully connected multi-layer perceptron (EFCMP). It consists of one input layer, one intermediate layer, and one encoded output layer. The Alopex algorithm is used to supervise the training of the EFCMP. Alopex is a stochastic algorithm used to solve optimization problems. The Alopex algorithm has been shown to accelerate the rate of convergence in the training procedure. Software simulation programs were developed for training, testing and analyzing the performance of this EFCMP architecture. Several neural networks with different structures were developed and compared. Optimization of the Alopex algorithm was explored through simulations of the EFCMP training procedure with the use of different parametric values for Alopex.

Alopex is a biologically influenced computation paradigm that uses a stochastic procedure to find the global optimum of linear and nonlinear functions. It maps to a hierarchical SIMD (Single-Instruction-Multiple-Data) architecture with simple neuronal processing elements (PE's), therefore the large amount of interconnects in other types of neural networks are not required and more efficient utilization of chip level and board level "real estate" is realized. In this study, verifications were performed on the use of a simplified Alopex algorithm in handwritten digit recognition with the intent that the verified algorithm be digitally implementable. The inputs to the simulated Alopex hardware are a set of 32 features extracted from the input characters. Although the goal of verifying the algorithm was not achieved, a firm direction for future studies has been established and a flexible software model for these future studies is available.

Development of a handwritten digit recognition system for real time applications is a feasible goal today due to the many advances pertinent to VLSI. In this research we address the issue of mapping our neural net classification algorithm to Intel's commercially available general purpose Neural Network Chip, 80170NX (ETANN). Most of the proposed techniques used for character recognition have been validated by our research group using various software and hardware simulation methods. The objective of this thesis was to develop a practical hardware system to perform the final step of classification of handwritten digits in an Optical Character Recognition (OCR) system. Such a hardware implementation would increase the classification speed and also would permit testing in a real life application environment. An efficient mapping scheme was evolved to map the modules of a limited interconnect classification algorithm, CLUMP, to a minimum number of ETANN chips. The hardware modules to interface the ETANN chips to MC68000 education board have been developed and tested. The proposed system is estimated to process the features input in 336 $\mu$s, for our specific implementation, with 12 clock phases and 3 ETANN chips.

Thinning is a very important step in a Character Recognition System. This thesis evolves a thinning algorithm that can be hardware implemented to improve the speed of the process. The software thinning algorithm features a simple set of rules that can be applied on both hexagonal and orthogonal character images. The hardware architecture features the SIMD structure, simple processing elements and near neighbor communications. The algorithm was simulated against the U.S. Postal Service Character Database. The architecture, evolved with consideration of both the software constraints and the physical layout limitations, was simulated using VHDL hardware description language. Subsequent to VLSI design and simulations the chip was fabricated. The project provides for a feasibility study in utilizing the parallel processor architecture for the implementation of a parallel image thinning algorithm. It is hoped that such a hardware implementation will speed up the processing and lead eventually to a real time system.

Alopex is a stochastic algorithm used to solve optimization problems in various types of systems. This thesis describes behavioral and structural hardware-description-language models which were developed for a three-stage VLSI-implementable Alopex architecture. The architecture features an SIMD structure and no communication between processing elements (PEs). Several approximations and simplifications were tested using the models to achieve a simple PE architecture and to implement the algorithm using integer arithmetic. Simulations were conducted with numerical image input to check the validity of these changes, and the timing relationships between PEs and controllers were explored. The use of a hardware description language provided an easy way to investigate timing and make architectural changes. The algorithm was found to function correctly under the digital hardware constraints and simplifications. The timing results gave an indication of the execution time for each step and pointed out areas in which the architecture may need to be improved.

Recent years have seen the renaissance of the neural network field. Significant advances in our understanding of neural networks and its possible applications necessitate investigations into possible implementation strategies. Among the presently available implementation medium, digital VLSI hardware is one of the more promising because of its maturity and availability. We discuss various issues connected with implementing neural networks in digital VLSI hardware. A new sigmoidal transfer function is proposed with that implementation in mind. Possible realizations of the function for stochastic and deterministic neural networks are discussed. Simulation studies of applying neural networks in constraint optimization and learning problems are carried out. These simulations were performed strictly in integer arithmetic. Simulation results provides an encouraging outlook for implementing these neural network applications in digital VLSI hardware. Important results concerning the sizes of various network values were found for learning algorithms.