Get an alignment. At the alignment place they will tell you whether or not your tie rod(s) is/are messed up, which they might be. Either way the tie rod being incorrectly set is causing this.

This really isn't a 'major problem'. Major problem would be ball joint failing at 85 MPH You can correct this for under $100.