Angiosperms produce seeds as their progeny enclosed in maternally-derived structures called fruits. Evolutionarily, fruits have contributed enormously to the success of the Angiosperms phylum by providing protection and nutrition to the developing seeds, while ensuring the efficient dispersal upon maturity. Fruits vary massively in both size and shape and certain species have been targeted for domestication due to their nutritional value and delicious taste. Among the vast array of 3D fruit shapes that exist in nature, the mechanism by which growth is oriented and coordinated to generate this diversity of forms is unclear. In this review, we discuss the latest results in identifying components that control fruit morphology and their effect on isotropic and anisotropic growth. Moreover, we will compare the current knowledge on the mechanisms that control fruit growth, size and shape between the domesticated Solanaceae species, tomato and members of the large family of Brassicaceae.