Human RBMY1 genes are located in four variable-sized clusters on the Y chromosome, expressed in male germ cells and possibly associated with sperm motility. We have re-investigated the mutational background and evolutionary history of the RBMY1 copy-number distribution in worldwide samples, and its relevance to sperm parameters in an Estonian cohort of idiopathic male-factor infertility subjects. We estimated approximate RBMY1 copy numbers in 12 181 000 Genomes Project phase 3 males from sequencing read-depth, then chose 14 for validation by multicolour fibre-FISH. These fibre-FISH samples provided accurate calibration standards for the entire panel, and led to detailed insights into population variation and mutational mechanisms. RBMY1 copy number worldwide ranged from three to 13 with a mode of eight. The two larger proximal clusters were the most variable, and additional duplications, deletions and inversions were detected. Placing the copy number estimates onto the published Y-SNP-based phylogeny of the same samples suggested a minimum of 562 mutational changes, translating to a mutation rate of 2.20x10-3 (95% CI 1.94x10-3 to 2.48x10-3) per father-to-son Y-transmission, higher than many Y-STRs, and showed no evidence for selection for increased or decreased copy number, but possible copy-number-stabilizing selection. An analysis of RBMY1 copy numbers among 376 infertility subjects failed to replicate a previously-reported association with sperm motility and showed no significant effect on sperm count and concentration, serum FSH, LH and testosterone levels, or testicular and semen volume. These results provide the first in-depth insights into the structural rearrangements underlying RBMY1 copy number variation across diverse human lineages.