Information-theoretics based genetic algorithm: Application to Hopfield's associative memory model of neural networks

This thesis refers to a research addressing the use of information-theoretic techniques in optimizing an artificial neural network (ANN) via a genetic selection algorithm. Pertinent studies address emulating relevant experiments on a test ANN (based on Hopfield's associative memory model) wherein the said optimization is tried with different sets of control parameters. These parameters include a new entity based on the concept of entropy as conceived in the field of information theory. That is, the mutual entropy (Shannon entropy) or information-distance (Kullback-Leibler-Jensen distance) measure between a pair of candidates is considered in the reproduction process of the genetic algorithm (GA) and adopted as a selection-constraint parameter. The research envisaged further includes a comparative analysis of the test results which indicate the importance of proper parameter selection to realize an optimal network performance. It also demonstrates the ability of the concepts proposed here in developing a new neural network approach for pattern recognition problems.