Novikov algebras were introduced in connection with the Poisson brackets of hydrodynamic-type and Hamiltonian operators in the formal variational calculus. It is well known that the radical of a finite-dimensional Novikov algebra is transitive. In this paper, we prove that a kind realization of Novikov algebras given by S Gel'fand is transitive and we give a deformation theory of Novikov algebras. In two and three dimensions, we find that all transitive Novikov algebras can be realized as the Novikov algebras given by S Gel'fand and their compatible infinitesimal deformations.