Studies on nonlinear activity and cross-entropy considerations in neural networks

The objectives of this research as deliberated in this dissertation are two-folded: (i) To study the nonlinear activity in the neural complex (real and artificial) and (ii) to analyze the learning processe(s) pertinent to an artificial neural network in the information-theoretic plane using cross-entropy error-metrics. The research efforts envisaged enclave the following specific tasks: (i) Obtaining a general solution for the Bernoulli-Riccati equation to represent a single parameter family of S-shaped (sigmoidal) curves depicting the nonlinear activity in the neural network. (ii) Analysis of the logistic growth of output versus input values in the neural complex (real and artificial) under the consideration that the boundaries of the sets constituting the input and output entities are crisp and/or fuzzy. (iii) Construction of a set of cross-entropy error-metrics (known as Csiszar's measures) deduced in terms of the parameters pertinent to a perceptron topology and elucidation of their relative effectiveness in training the network optimally towards convergence. (iv) Presenting the methods of symmetrizing and balancing the aforesaid error-entropy measures (in the information-theoretic plane) so as to make them usable as error-metrics in the test domain. (v) Description and analysis of the dynamics of neural learning process in the information-theoretic plane for both crisp and fuzzy attributes of input values. Relevant to these topics portraying the studies on nonlinear activity and cross-entropy considerations vis-a-vis neural networks, newer and/or exploratory inferences are made, logical conclusions are enumerated and relative discussions are presented along with the scope for future research to be pursued.