Since the concept of left-handed medium (LHM) was proposed, the causality has been a big concern in the understanding of LHM. Through an exact analysis of a 1D transient current source radiating in a LHM, the causality in the propagation of electromagnetic waves is investigated where three cases are considered for different frequency dispersions in LHM. Numerical experiments have shown that the causality would be violated if the LHM were homogeneous and frequency nondispersive in the whole frequency range or in a certain frequency band. However, such a nondispersive LHM does not exist. For a realistic artificial LHM which is frequency dispersive [Science 292, 77 (2001)], we have shown that the causality is not violated at all.