If you'll remember the episode where Stewie was toilet trained... Hmm, what's the name of that one... Ah yes, Brian In Love, season 2 episode 4, Stewie and Peter are in the bathroom and it goes something like this... I know the picture is from a completely different episode. Ignore that. Just listen.

Stewie and Brian hold conversations all the time and understand each other perfectly well. The rest of the family, I guess it depends.