I would always fit a loose ball joint as soon as possible, if it breaks you could hurt yourself or others.