For the nearly exponential type of feedforward neural networks (neFNNs), it is revealed the essential order of their approximation. It is proven that for any continuous function defined on a compact set of Rd, there exists a three-layer neFNNs with fixed number of hidden neurons that attain the essential order. When the function to be approximated belongs to the α-Lipschitz family (0<α≤2), the essential order of approximation is shown to be O(n-α) where n is any integer not less than the reciprocal of the predetermined approximation error. The upper bound and lower bound estimations on approximation precision of the neFNNs are provided. The obtained results not only characterize the intrinsic property of approximation of the neFNNs, but also uncover the implicit relationship between the precision (speed) and the number of hidden neurons of the neFNNs.

The neuron state modeling and the local field modeling provides two fundamental modeling approaches to neural network research, based on which a neural network system can be called either as a static neural network model or as a local field neural network model. These two models are theoretically compared in terms of their trajectory transformation property, equilibrium correspondence property, nontrivial attractive manifold property, global convergence as well as stability in many different senses. The comparison reveals an important stability invariance property of the two models in the sense that the stability (in any sense) of the static model is equivalent to that of a subsystem deduced from the local field model when restricted to a specific manifold. Such stability invariance property lays a sound theoretical foundation of validity of a useful, cross-fertilization type stability analysis methodology for various neural network models.

There have been various algorithms designed for simulating natural evolution. This paper proposes a new simulated evolutionary computation model called the abstract evolutionary algorithm (AEA), which unifies most of the currently known evolutionary algorithms and describes the evolution as an abstract stochastic process composed of two fundamental operators: selection and evolution operators. By axiomatically characterizing the properties of the fundamental selection and evolution operators, several general convergence theorems and convergence rate estimations for the AEA are established. The established theorems are applied to a series of known evolutionary algorithms, directly yielding new convergence conditions and convergence rate estimations of various specific genetic algorithms and evolutionary strategies. The present work provides a significant step toward the establishment of a unified theory of simulated evolutionary computation.

In this paper, a new concept called nonlinear measure is introduced to quantify stability of nonlinear systems in the way similar to the matrix measure for stability of linear systems. Based on the new concept, a novel approach for stability analysis of neural networks is developed. With this approach, a series of new sufficient conditions for global and local exponential stability of Hopfield type neural networks is presented, which generalizes those existing results. By means of the introduced nonlinear measure, the exponential convergence rate of the neural networks to stable equilibrium point is estimated, and, for local stability, the attraction region of the stable equilibrium point is characterized. The developed approach can be generalized to stability analysis of other general nonlinear systems.

The Hopfield-type networks with asymmetric interconnections are studied from the standpoint of taking them as computational models. Two fundamental properties, feasibility and reliability, of the networks related to their use are established with a newly-developed convergence principle and a classification theory on energy functions. The convergence principle generalizes that previously known for symmetric networks and underlies the feasibility. The classification theory, which categorizes the traditional energy functions into regular, normal and complete ones according to their roles played in connection with the corresponding networks, implies that the reliability and high efficiency of the networks can follow respectively from the regularity and the normality of the corresponding energy functions. The theories developed have been applied to solve a classical NP-hard graph theory problem: finding the maximal independent set of a graph. Simulations demonstrate that the algorithms deduced from the asymmetric theories outperform those deduced from the symmetric theory.