Objective: Sex hormones are not only involved in the formation of reproductive organs, but also induce sexually-dimorphic brain development and organization. Cross-sex hormone administration to transsexuals provides a unique possibility to study the effects of sex steroids on brain morphology in young adulthood. Methods: Magnetic resonance brain images were made prior to, and during, cross-sex hormone treatment to study the influence of anti-androgen + estrogen treatment on brain morphology in eight young adult male-to-female transsexual human subjects and of androgen treatment in six female-to-male transsexuals. Results: Compared with controls, anti-androgen + estrogen treatment decreased brain volumes of male-to-female subjects towards female proportions, while androgen treatment in female-to-male subjects increased total brain and hypothalamus volumes towards male proportions. Conclusions: The findings suggest that, throughout life, gonadal hormones remain essential for maintaining aspects of sex-specific differences in the human brain.