A class of simple Hopfield neural networks with a parameter is investigated. Numerical simulations show that the simple Hopfield neural networks can display chaotic attractors and limit cycles for different parameters. The Lyapunov exponents are calculated; the bifurcation plot and several important phase portraits are presented as well. By virtue of a recent result of horseshoe theory in dynamical systems, we present rigorous computer-assisted verifications for chaotic behavior in the simple Hopfield neural networks for certain parameters and give a brief discussion on the robustness of the chaotic behavior. Quantitative descriptions of the complexity of these neural networks are also given in terms of topological entropy