We study the $L^p$-theory for the Schr\"odinger operator with critical rough potential of $a|x|^{-2}$ type. The developed harmonic analysis tools, such as multiplier estimate, Littlewood-Paley theory and the equivalence of Sobolev norms, will be employed to study the scattering theory of energy-critical defocusing nonlinear Schr\"odinger equation with the inverse-square potential. These tools in $L^p$-theory are new and the range of $p$ depends on the parameter $a$, which is different from the classical theory. The main difficulty is raised from the failure of Gaussian boundedness of the heat kernel associated with the operator $P_a=-\Delta+a|x|^{-2}$ when $a$ is negative.