As you drive that wheel without the weight will unevenly spin and force a vibration through the car and/or steeringwheel. It may also cause cupping or scalloping wear on the tire itself if its really bad.

Take it to like a tire shop and have the wheel+tire balanced.