Using first-principles total-energy calculations, we have investigated the adsorption and diffusion of Si and Ge adatoms on Ge/Si(001)-(2 8) and Ge/Si(105)-(1 2) surfaces. The dimer vacancy lines on Ge/Si(001)-(2 8) and the alternate SA and rebonded SB steps on Ge/Si(105)-(1 2) are found to strongly influence the adatom kinetics. On Ge/Si(001)-(2 8) surface, the fast diffusion path is found to be along the dimer vacancy line (DVL), reversing the diffusion anisotropy on Si(001). Also, there exists a repulsion between the adatom and the DVL, which is expected to increase the adatom density and hence island nucleation rate in between the DVLs. On Ge/Si(105)-(1 2) surface, the overall diffusion barrier of Si(Ge) along e(501T direction is relative fast with a barrier of) 0.83(0.61) eV, despite of the large surface undulation. This indicates that the adatoms can rapidly diffuse up and down the (105)-faceted Ge hut island. The diffusion is also almost isotropic along [010] and ½(501) directions.