I would recommend replacing the tranny.

On a side note... you do not need the push in the button on the shifter go between Neutral and Drive. More importantly the button is REQUIRED to shift into Reverse, "2" or "L". This is done intentionally so you cannot accdentally shift into those gears. The tranny is designed ot easily slip between neutral and drive. IF your going ot be stopped for more than 10 seconds, it's good for fuel consumption and tranny wear to go to neutral. It's a bad habit to use the button while shifting form drive ot neutral. Live and learn I guess.