I think this was the real payload in your response to him. I don't know whether you intended this, but if I had heard this, I would take this as saying, "I have the power to fire you". A strong message for him to stay in line, and yet also a fully professional and impartial delivering of information. A perfect combination of iron fist and velvet glove. Perhaps we could call this a counter-micro-aggression.