The arbuscular mycorrhizal (AM) symbiosis is a nearly ubiquitous association formed by most land plants. Numerous insights into the molecular mechanisms governing this symbiosis have been obtained in recent years leading to the identification of a core set of plant genes essential for successful formation of the AM symbiosis by angiosperm hosts. Recent phylogenetic analyses indicate that while the origin of some of these symbiotic genes predated the first land plants, the rest appeared through processes including de novo evolution and gene duplication that occurred specifically in the land plants. Purifying selection on this core gene set has been maintained over millions of years of plant evolution to conserve the AM symbiosis. However, several independent losses of this association have been recorded in numerous embryophyte lineages. In these lineages, potential compensatory mechanisms have been identified that could have helped these plants overcome the adversities imposed by the loss of the AM symbiosis. This review will focus on the processes governing the conservation of the AM symbiosis in the land plant lineage.