From invertebrates to humans, males and females of a given species display identifiable differences in behaviors, mostly but not exclusively pertaining to sexual and social behaviors. Within a species, individuals preferentially exhibit the set of behaviors that is typical of their sex. These behaviors include a wide range of coordinated and genetically pre-programmed social and sexual displays that ensure successful reproductive strategies and the survival of the species. What are the mechanisms underlying sex-specific brain function? Although sexually dimorphic behaviors represent the most extreme examples of behavioral variability within a species, the basic principles underlying the sex specificity of brain activity are largely unknown. Moreover, with few exceptions, the quest for fundamental differences in male and female brain structures and circuits that would parallel that of sexual behaviors and peripheral organs has so far uncovered modest quantitative rather than the expected clear qualitative differences. As will be detailed in this review, recent advances have directly challenged the established notion of the unique role of steroid hormones in organizing and activating male- and female-specific brain circuits and have uncovered new mechanisms underlying the neural control of sex-specific behaviors.