Seed number per pod (SNPP) is one of the major yield components and breeding targets in rapeseed that shows great variation and is invaluable for genetic improvement. To elucidate the genetic architecture and uncover the mechanism of SNPP, we identified five quantitative trait loci (QTLs) using the BnaZNRIL population, which were integrated with those of previous studies by physical map to demonstrate a complex and relatively complete genetic architecture of SNPP. A major QTL, qSN.A6, was successfully fine-mapped from 1910 to 267 kb using near-isogenic line (NIL). In addition, qSN.A6 exhibited an antagonistic pleiotropy on seed weight (SW), which is caused by a physiological interaction in which SNPP acts "upstream" of SW. Because the negative effect of qSN.A6 on SW cannot fully counteract its positive effect on SNPP, it also enhanced the final yield (17.4%), indicating its great potential for utilization in breeding. The following genetic and cytological experiments further confirmed that the different rate of ovule abortion was responsible for the ~5 seed difference between Zhongshuang11 and NIL-qSN.A6. This systematic approach to dissecting the comprehensive genetic architecture of SNPP and characterizing the underlying mechanism has advanced the understanding of SNPP and will facilitate the development of high-yield cultivars.