port and polish just makes the passages for air smoother. The less resistance that the air has, the faster it will go. If your going to consider doing things like port and polish, you midas well replace the valves with skinnier ones. That will help reduce resistance as well.
Just make sure not to port and polish too much or you'll make the walls too thin.