Extensive studies show that the naturally developed rock joint surfaces have the properties of fractals. The surface roughness of rock joints can be well described within the framework of fractal geometry. To give a better understanding of the roughness of rock fracture surfaces in relation to the mechanical properties and behavior of rock joints in loading, a systematic investigation has been carried out. By means of a laser scanning instrument, the fracture surfaces induced in rocks are measured. The fractal characters of rough fracture surfaces of rocks are analysed according to the variogram method, which elaborates explicitly the importance of the geometric parameters-fractal dimension D and the intercept A. Investigation extends to the anisotropy and heterogeneity of rock fracture surfaces, and the scale effect on the fractal estimation. The shear tests on rock joints show a combined effect of the fractal parameters on the mechanical properties and behavior of rock joints. To control the roughness and show the effects in a direct manner, a series of fractal joints with different fractal dimensions are generated on the basis of the Weierstrass-Mandelbrot function and then manufactured in the polycarbonate plates. By using the photoelastic method, the manners of normal and shear deformation, the stress field in the vicinity of the joint surface and the contact behaviour of the fractal joints have been studied in detail under uniaxial compression and direct shear. Based on the experimental results, the dependence of deformation stiffness on the fractal dimensions of rock joints has been recognized. The empirical criteria of shear strength and the evolution law of surface damage during the shear process are developed.