You have two ways to sort this problem out. Either you bring your wife together and ask her to take responsibility for herself while you’re on your duty or you should communicate with her. You should take enough time to communicate with her and make her understand why you can’t take her along. Divorce would not be a good idea on such a small understanding issue. Communication might solve this issue and your wife can understand the difference.