Puberty is a time of striking changes in cognition and behavior. To indirectly assess the effects of puberty-related influences on the underlying neuroanatomy of these behavioral changes we will review and synthesize neuroimaging data from typically developing children and adolescents and from those with anomalous hormone or sex chromosome profiles. The trajectories (size by age) of brain morphometry differ between boys and girls, with girls generally reaching peak gray matter thickness 1-2 years earlier than boys. Both boys and girls with congenital adrenal hyperplasia (characterized by high levels of intrauterine testosterone), have smaller amygdala volume but the brain morphometry of girls with CAH did not otherwise significantly differ from controls. Subjects with XXY have gray matter reductions in the insula, temporal gyri, amygdala, hippocampus, and cingulate-areas consistent with the language-based learning difficulties common in this group.