Neural computers

Recent years have seen the renaissance of the neural network field. Significant advances in our understanding of neural networks and its possible applications necessitate investigations into possible implementation strategies. Among the presently available implementation medium, digital VLSI hardware is one of the more promising because of its maturity and availability. We discuss various issues connected with implementing neural networks in digital VLSI hardware. A new sigmoidal transfer function is proposed with that implementation in mind. Possible realizations of the function for stochastic and deterministic neural networks are discussed. Simulation studies of applying neural networks in constraint optimization and learning problems are carried out. These simulations were performed strictly in integer arithmetic. Simulation results provides an encouraging outlook for implementing these neural network applications in digital VLSI hardware. Important results concerning the sizes of various network values were found for learning algorithms.

Recently, Artificial Neural Network (ANN) computing systems have become one of the most active and challenging areas of information processing. The successes of experimental neural computing systems in the fields of pattern recognition, process control, robotics, signal processing, expert system, and functional analysis are most promising. However due to a number of serious problems, only small size fully connected neural networks have been implemented to run in real-time. The primary problem is that the execution time of neural networks increases exponentially as the neural network's size increases. This is because of the exponential increase in the number of multiplications and interconnections which makes it extremely difficult to implement medium or large scale ANNs in hardware. The Modular Grouped Weight Quantization (MGWQ) presented in this dissertation is an ANN design which assures that the number of multiplications and interconnections increase linearly as the neural network's size increases. The secondary problems are related to scale-up capability, modularity, memory requirements, flexibility, performance, fault tolerance, technological feasibility, and cost. The MGWQ architecture also resolves these problems. In this dissertation, neural network characteristics and existing implementations using different technologies are described. Their shortcomings and problems are addressed, and solutions to these problems using the MGWQ approach are illustrated. The theoretical and experimental justifications for MGWQ are presented. Performance calculations for the MGWQ architecture are given. The mappings of the most popular neural network models to the proposed architecture are demonstrated. System level architecture considerations are discussed. The proposed ANN computing system is a flexible and a realistic way to implement large fully connected networks. It offers very high performance using currently available technology. The performance of ANNs is measured in terms of interconnections per second (IC/S); the performance of the proposed system changes between 10^11 to 10^14 IC/S. In comparison, SAIC's DELTA II ANN system achieves 10^7. A Cray X-MP achieves 5*10^7 IC/S.