The reprogramming of epigenetic states in gametes and embryos is essential for correct development in plants and mammals. In plants, the germ line arises from somatic tissues of the flower, necessitating the erasure of chromatin modifications that have accumulated at specific loci during development or in response to external stimuli. If this process occurs inefficiently, it can lead to epigenetic states being inherited from one generation to the next. However, in most cases, accumulated epigenetic modifications are efficiently erased before the next generation. An important example of epigenetic reprogramming in plants is the resetting of the expression of the floral repressor locus FLC in Arabidopsis thaliana. FLC is epigenetically silenced by prolonged cold in a process called vernalization. However, the locus is reactivated before the completion of seed development, ensuring the requirement for vernalization in every generation. In contrast to our detailed understanding of the polycomb-mediated epigenetic silencing induced by vernalization, little is known about the mechanism involved in the reactivation of FLC. Here we show that a hypomorphic mutation in the jumonji-domain-containing protein ELF6 impaired the reactivation of FLC in reproductive tissues, leading to the inheritance of a partially vernalized state. ELF6 has H3K27me3 demethylase activity, and the mutation reduced this enzymatic activity in planta. Consistent with this, in the next generation of mutant plants, H3K27me3 levels at the FLC locus stayed higher, and FLC expression remained lower, than in the wild type. Our data reveal an ancient role for H3K27 demethylation in the reprogramming of epigenetic states in plant and mammalian embryos.