A proper design of the architecture of Artificial Neural Network (ANN) models can provide a robust tool in water resources modeling and forecasting. The performance of different neural networks in a groundwater level forecasting is examined in order to identify an optimal ANN architecture that can simulate the decreasing trend of the groundwater level and provide acceptable predictions up to 18 months ahead. Messara Valley in Crete (Greece) was chosen as the study area as its groundwater resources have being overexploited during the last fifteen years and the groundwater level has been decreasing steadily. Seven different types of network architectures and training algorithms are investigated and compared in terms of model prediction efficiency and accuracy. The different experiment results show that accurate predictions can be achieved with a standard feedforward neural network trained with the Levenberg–Marquardt algorithm providing the best results for up to 18 months forecasts.