We study the performance of Gas Electron Multipliers (GEMs) in gaseous He, Ne and Ne+H2 at temperatures in the range of 2.6-293K. In He, at temperatures between 62 and 293 K, the triple-GEM structures often operate at rather high gains, exceeding 1000. There is an indication that this high gain is achieved by the Penning effect in the gas impurities released by outgassing. At lower temperatures, the gain-voltage characteristics are significantly modified probably due to the freeze-out of impurities. In particular, the double-and single-GEM structures can operate down to 2.6K at gains reaching only several tens at a gas density of about 0.5g/l; at higher densities the maximum gain drops further. In Ne, the maximum gain also drops at cryogenic temperatures. The gain drop in Ne at low temperatures can be reestablished in Penning mixtures of Ne+H2: very high gains, exceeding 104, have been obtained in these mixtures at 50-60K, at a density of 9.2g/l corresponding to that of saturated Ne vapor near 27K. The results obtained are relevant in the fields of two-phase He and Ne detectors for solar neutrino detection and electron avalanching at low temperatures.