Red clump stars are fundamental distance indicators in astrophysics, although theoretical stellar models predict a dependence of absolute magnitudes with age. This effect is particularly strong below ~2 Gyr, but even above this limit a mild age dependence is still expected. We use seismically identified red clump stars in the Kepler field for which we have reliable distances, masses, and ages from the SAGA survey, to first explore this effect. By excluding red clump stars with masses larger than $1.6\,{M}_{\odot }$ (corresponding to ages younger than 2 Gyr), we derive robust calibrations linking intrinsic colors to absolute magnitudes in the following photometric systems: Strömgren by, Johnson BV, Sloan griz, 2MASS JHKs, and WISE $W1W2W3$. With the precision achieved we also detect a slope of absolute magnitudes $\sim 0.020\pm 0.003\,\mathrm{mag}\,{\mathrm{Gyr}}^{-1}$ in the infrared, implying that distance calibrations of clump stars can be off by up to $\sim 0.2\,\mathrm{mag}$ in the infrared (over the range from 2 to 12 Gyr) if their ages are unknown. Even larger uncertainties affect optical bands, because of the stronger interdependency of absolute magnitudes on colors and age. Our distance calibrations are ultimately based on asteroseismology, and we show how the distance scale can be used to test the accuracy of seismic scaling relations. Within the uncertainties our calibrations are in agreement with those built upon local red clumps with Hipparcos parallaxes, although we find a tension, which, if confirmed, would imply that scaling relations overestimate the radii of red clump stars by 2 ± 2%. Data releases post Gaia DR1 will provide an important testbed for our results.