Your fear of loosing your job is right on one part but you can’t see your wife or any other woman to feel uncomfortable or to be  disrespectful, being a man you should be concern about what your wife feel and she should be given more priority. If you will raise voice in-order to protect her than their will be these two conditions either your boss would stop all these things and disturbing her or else he will kick you out from his department. So, without being conscious you should protect her and help her to feel comfortable because while marrying you would have given her the vow that you will be protecting her at any cost. So, now is the time to be more responsible towards her and show that vows really means something to you and, while doing this if you miss the job than also it will be okay you might get the other one and it could be possible that the next would be more beneficial. So, without thinking much protect your wife.