Learning in connectionist networks using the Alopex algorithm

The Alopex algorithm is presented as a universal learning algorithm for connectionist models. It is shown that the Alopex procedure could be used efficiently as a supervised learning algorithm for such models. The algorithm is demonstrated successfully on a variety of network architectures. Such architectures include multilayer perceptrons, time-delay models, asymmetric, fully recurrent networks and memory neuron networks. The learning performance as well as the generation capability of the Alopex algorithm are compared with those of the backpropagation procedure, concerning a number of benchmark problems, and it is shown that the Alopex has specific advantages over the backpropagation. Two new architectures (gain layer schemes) are proposed for the on-line, direct adaptive control of dynamical systems using neural networks. The proposed schemes are shown to provide better dynamic response and tracking characteristics, than the other existing direct control schemes. A velocity reference scheme is introduced to improve the dynamic response of on-line learning controllers. The proposed learning algorithm and architectures are studied on three practical problems; (i) Classification of handwritten digits using Fourier Descriptors; (ii) Recognition of underwater targets from sonar returns, considering temporal dependencies of consecutive returns and (iii) On-line learning control of autonomous underwater vehicles, starting with random initial conditions. Detailed studies are conducted on the learning control applications. Effect of the network learning rate on the tracking performance and dynamic response of the system are investigated. Also, the ability of the neural network controllers to adapt to slow and sudden varying parameter disturbances and measurement noise is studied in detail.