I think the main reason he is afraid of you is that you are really good at constructing persuasive arguments. In a board meeting, it doesn't matter how big and strong you are. Words are the weapons, and you probably know how to use them much more effectively than he does..