Formulation of fertility selection

I introduced the idea of a fitness matrix earlier when we were discussing selection at one locus with more than two alleles. Even if we have only two alleles, it becomes useful to describe patterns of fertility selection in terms of a fitness matrix. Describing the matrix is easy. Writing it down gets messy. Each element in the table is simply the average number of offspring produced by a given mated pair. We write down the table with paternal genotypes in columns and maternal genotypes in rows:

	Paternal genotype
Maternal genotype	$A_1A_1$	$A_1A_2$	$A_2A_2$
$A_1A_1$	$F_{11,11}$	$F_{11,12}$	$F_{11,22}$
$A_1A_2$	$F_{12,11}$	$F_{12,12}$	$F_{12,22}$
$A_2A_2$	$F_{22,11}$	$F_{22,12}$	$F_{22,22}$

Then the frequency of genotype $A_1A_1$ after one generation of fertility selection is:⁷

$$x_{11}' = \frac{x_{11}^2F_{11,11} + x_{11}x_{12}(F_{11,12} + F_{12,11})/2 + (x_{12}^2/4)F_{12,12}}{\bar F} \quad ,$$ (1)

where $\bar F$ is the mean fecundity of all matings in the population.⁸

It probably won't surprise you to learn that it's very difficult to say anything very general about how genotype frequenices will change when there's fertility selection. Not only are there nine different fitness parameters to worry about, but since genotypes are never guaranteed to be in Hardy-Weinberg proportion, all of the algebra has to be done on a system of three simultaneous equations.⁹ There are three weird properties that I'll mention:

1. $\bar F'$ may be smaller than $\bar F$. Unlike selection on viabilities in which fitness evolved to the maximum possible value, there are situations in which fitness will evolve to the minimum possible value when there's selection on fertilities.¹⁰

2. A high fertility of heterozygote $\times$ heterozygote matings is not sufficient to guarantee that the population will remain polymorphic.

3. Selection may prevent loss of either allele, but there may be no stable equilibria.