Symmetry exists widely in the real world and plays a remarkable role in perception problems. The existing symmetry detection algorithms are mainly based on luminance or gradient information. Upon analyzing the relationship between symmetry and phase, based on phase congruency, we propose a phase-based symmetry detection (PSD) algorithm. PSD is calculated based on log Gabor wavelet. The symmetric points of objects are obtained by inspecting the phase information. The feasibility analysis, phase-based symmetry detection definition, and rationality demonstrations established the theoretic foundation for this algorithm. The experiments show that this algorithm can be applied directly to original images without segmentation-it is invariant to rotation, luminance and contrast-and it can detect several types of symmetries at the same time and it is compared with several other methods.