A major unresolved challenge of evolutionary biology is to determine the nature of the allelic variants of “speciation genes”: those alleles whose interaction produces inviable or infertile interspecific hybrids but does not reduce fitness in pure species. Here we map quantitative trait loci (QTL) affecting fertility of male hybrids between D. yakuba and its recently discovered sibling species, D. santomea. We mapped three to four X chromosome QTL and two autosomal QTL with large effects on the reduced fertility of D. yakuba and D. santomea backcross males. We observed epistasis between the X-linked QTL and also between the X and autosomal QTL. The X chromosome had a disproportionately large effect on hybrid sterility in both reciprocal backcross hybrids. However, the genetics of hybrid sterility differ between D. yakuba and D. santomea backcross males, both in terms of the magnitude of main effects and in the epistatic interactions. The QTL affecting hybrid fertility did not colocalize with QTL affecting sexual isolation in this species pair, but did colocalize with QTL affecting the marked difference in pigmentation between D. yakuba and D. santomea. These results provide the basis for future high-resolution mapping and ultimately, molecular cloning, of the interacting genes that contribute to hybrid sterility.