I've definitely had experiences like this. Whether from a couple rungs up the ladder or a direct report. I've definitely felt like throwing in the towel, but I didn't. . . and I'm glad I didn't. In each situation, I found that I had to step back, put personalities aside and logically evaluate why I am choosing to be there and it's role in achieving my larger goals. Once deciding that you are in the right place for your personal goals, (I've found) there are three ways to deal with difficult people:
(1) Out smart them,
(2) out last them,
(3) or win them over.
I usually use (1) when dealing with power-crazed regulators, (2) is passive but takes a bit of patience, if someone is a jerk to you, they are probably a jerk to everyone and won't last long for one reason or another, (3) takes a lot of effort but is the most rewarding.
I had one boss years ago that was brought on because of his reputation in industry and he knew people who knew people. To find a position for him, they slid him in on a project position between me and my supervisor. He demanded that I undo just about everything I had designed, purchased and implemented leading up to the physical execution of the project. A year and a half of work . . . and he was not nice about it, in fact, your description sounds pretty familiar. Early on I tried to make peace, apologizing for the friction I caused him, but he said he wasn't their to make friends, he was there to get the project done. There was a lot of friction. Both he an I were brought in separately for a conversation with his boss (my boss's boss). Our boss told me "you can't fire your boss." Not sure this would work in every situation, but after I decided I was going to stay, I took an online personality test (as goofy as that sounds) . . . twice. The first time, for myself. The second time, I pretended I was him. WOW! Reading the results of the second test, it gave me a lot of insight into how he was motivated, what he was looking for, and where our personalities clash. I adjusted my attitude and my work to suit him (he was the boss, after all). He loved to beat opposition down with what he knew so I made sure that I was always loading him up with ammunition. Whatever his concern was at the moment, even if not really the most important in the big picture, became my focus. He would start to ask, "can we do this?" or "can we make this happen?" and I always made sure my answer was "YES." The company got into a legal dispute that was far more intensified by this guy's personality. Things completely changed one day when I was in the conference room with his boss, waiting for a meeting to start. His boss asked how much of this I thought was my boss's fault. I said, I stand behind [boss's name] 100%. My boss walked in just as I said it. From that point on, he knew I was on his side. Together, between my technical knowledge and his intense personality where he could push ideas through and achieve some amazing things for the company. I did eventually leave, but I am happy to say, it wasn't because of him. Do you know what I found at the next company? More people to win over.