People are different. Some people are ok with your actions, some are not.
There's nothing right or wrong about what you did or his reaction. It's just the way it is. I, don't quite care for my woman to be having a good time with other guys. Thats me. The next couple, who knows, maybe he encourages his woman to get near other guys, maybe he likes that. It's not about right or wrong, it's about what works for him and what works for you.
I can't say there is no perfect relationship, but everyone that I've been in has been a bit of give and take. Nothing is exactly the way I want it, which is good. It helps me learn and grow. It's really between you and him if this is something you want to give up or police yourself on more or if he needs to go.