Legumes can establish intracellular interactions with symbiotic microbes to enhance their fitness, including the interaction with arbuscular mycorrhizal (AM) fungi. AM fungi colonize root epidermal cells to gain access to the root cortex and this requires the recognition by the host plant of fungal made Myc factors. Genetic dissection has revealed the symbiosis signaling pathway that allows recognition of AM fungi, but the downstream processes that are required to promote fungal infection are poorly understood. Abscisic acid (ABA) has been shown to promote arbuscule formation in tomato. Here we show that ABA modulates the establishment of the AM symbiosis in Medicago truncatula by promoting fungal colonization at low concentrations and impairing it at high concentrations. We show that the positive regulation of AM colonization via ABA requires a PP2A holoenzyme subunit PP2AB’1. Mutations in PP2AB’1 cause reduced levels of AM colonization which cannot be rescued with permissive ABA application. The action of PP2AB’1 in response to ABA is unlinked to the generation of calcium oscillations as the pp2aB’1 mutant displays a normal calcium response. This contrasts with application of high concentrations of ABA that impairs Myc factor induced calcium oscillations suggesting different modes of action of ABA on the AM symbiosis. Our work reveals that ABA functions at multiple levels to regulate the AM symbiosis and that a PP2A phosphatase is required for ABA promotion of AM colonization.