An adaptive ghost fluid-nite volume method is developed for one-and two-dimensional compressible multi-medium flows in this work. It couples the real ghost fluid method (GFM) [SIAM J. Sci. Comput. 28 (2006) 278] and the adaptive moving mesh method [SIAM J. Numer. Anal. 41(2003) 487; J. Comput. Phys. 188(2003) 543], and thus retains their advantages. This work shows that the local mesh clustering in the vicinity of the material interface can effectively reduce both numerical and conservative errors caused by the GFM around the material interface and other discontinuities. Besides the improvement of flow field resolution, the adaptive ghost fluid method also largely increases the computational efficiency. Several numerical experiments are conducted to demonstrate robustness and efficiency of the current method. They include several 1D and 2D gas-water flow problems, involving a large density gradient at the material interface and strong shock-interface interactions. The results show that our algorithm can capture the shock waves and the material interface accurately, and is stable and robust even solution with large density and pressure gradients.