Yeah, he prolly just wanted to bitch at you for passing across the double yellow. While he would be right and you shouldn't do it, if you can see and there is no traffic I can't say that I wouldn't have been tempted to do it myself.