Through a Distorted Lens: Navigating Bias in ERM | Cential