Which bearing retainer are you talking about? Are you talking about the keeper or "wedding ring" that is pressed onto the axle to hold in on?
Or the plate that bolts on the outside of the brake backing plate that tightens the axle to the housing end?
The prior bearings had over 5,000 hard miles on them and were both just starting to get a bit rough feeling when turning them, there was no side to side movement to them at all yet. I replaced them just because. I changed nothing else, the axle tubes have billet ends on them that look fine and the axle themselves are straight and true. Here we are using a lathe to turn the old keepers off of the axles...if anything was out of whack it would have shown here.
We'll double check the trueness of everything axle related later on today when swapping the bearing.