Mammalian sex determination is controlled by the antagonistic interactions of two genetic pathways: The SRY-SOX9-FGF9 network promotes testis determination partly by opposing proovarian pathways, while RSPO1/WNT-β-catenin/FOXL2 signals control ovary development by inhibiting SRY-SOX9-FGF9. The molecular basis of this mutual antagonism is unclear. Here we show that ZNRF3, a WNT signaling antagonist and direct target of RSPO1-mediated inhibition, is required for sex determination in mice. XY mice lacking ZNRF3 exhibit complete or partial gonadal sex reversal, or related defects. These abnormalities are associated with ectopic WNT/β-catenin activity and reduced expression during fetal sex determination. Using exome sequencing of individuals with 46,XY disorders of sex development, we identified three human variants in very rare cases of XY female presentation. We tested two missense variants and show that these disrupt ZNRF3 activity in both human cell lines and zebrafish embryo assays. Our data identify a testis-determining function for ZNRF3 and indicate a mechanism of direct molecular interaction between two mutually antagonistic organogenetic pathways.