You must wait and find him at night when he's least expecting it. Then let him know that you're waiting for him to slip up and he's gonna get it. Don't tell what you will do, and don't use physical violence. That will only mess your situation up more.
Just let him live in fear. And watch them