A flexible neural network (FNN) is a multilayer feedforward neural network with the characteristics of: (1) overlayer connections; (2) variable activation functions for different nodes and (3) sparse connections between the nodes. A new approach for designing the FNN based on neural tree encoding is proposed in this paper. The approach employs the ant programming (AP) to evolve the architecture of the FNN and the particle swarm optimization (PSO) to optimize the parameters encoded in the neural tree. The performance and effectiveness of the proposed method are evaluated using time series prediction problems and compared with the related methods.