Interfaces often play a significant role in many physical properties and phenomena of nanocrystalline materials (NcMs). In the present paper, the interface effects on the effective elastic property of NcMs are investigated. First, an atomic potential method is suggested for estimating the effective elastic modulus of an interface phase. Then, the Mori-Tanaka effective field method is employed to determine the overall effective elastic moduli of a nanocrystalline material, which is regarded as a binary composite consisting of a crystal or inclusion phase with regular lattice connected by an amorphous-like interface or matrix phase. Finally, the stiffening effects of strain gradients are examined on the effective elastic property by using the strain gradient theory to analyze a representative unit cell. Our analysis shows two physical mechanisms of interfaces that influence the effective stiffness and other mechanical properties of materials. One is the softening effect due to the distorted atomic structures and the increased atomic spacings in interface regions, and another is the baffling effect due to the existence of boundary layers between the interface phase and the crystalline phase.