Motivated by the problem of training multilayer perceptrons in neural networks, we consider the problem of minimizing E(x)=n=1 fi (ξi·x), where ξi ∈ Rs, 1≤i≤n, and each fi (ξi·x) is a ridge function. We show that when n is small the problem of minimizing E can be treated as one of minimizing univariate functions, and we use the gradient algorithms for minimizing E when n is moderately large. For largen, we present the online gradient algorithms and especially show the monotonicity and weak convergence of the algorithms.