Well, I think it depends on the environment and the company - but design is a constant battle. You are always fighting the constraints of time, technology, and budget to get the best quality for the user. For me, one of the downsides is you can’t win every battle. Sometimes, (to quote “Pulp Fiction”) the path of righteousness is beset on all sides by the tyranny of evil and ignorance, and the technical, political, and budget realities of your project. The problem is as a designer it is your job to understand the user so well, that you develop a lot of compassion for them and you WANT to help them. You want so desperately to give them user experience they need and deserve. But sometimes, the data just isn’t there. The web service you need is not exposed. The cost out weighs the benefits and there is nothing you can do.
When this happens it hurts. It physically makes you feel sick sometimes when you have to compromise on a design you believe in passionately and completely. But as long as you fought hard, did your research, and tested every path available, then maybe, just maybe you can sleep at night.
Being a designer means its your job to see the path forward. When, for whatever reason, you are unable to walk that path it can be frustrating and stressful.
Another problem (which happens to me a lot more rarely now since I work with people who understand the value of UX) is that some people do not understand design. Sometimes the client doesn’t get it, sometimes the project manager doesn’t get it and you find yourself having to fight just to get a basic process in place. Installing a design culture at a big company takes time. The bigger the company, them more ingrained they are in their processes and it’s hard to convince a stakeholder who had to fight for a million dollar budget that we are going to give authority over the design to the user. We are going to make the user the owner, and that means every feature you THINK you wanted now needs to be validated by the user. A lot of executives have a problem at first with that concept. Ultimately it sometimes boils down to a lack of respect (bordering on contempt) for the end user. As a designer, it’s your job to get the entire project team to adopt the same love and compassion of the user that you have. This is not easy. It is also a hard, frustrating battle that you have to fight over and over again.
It’s tempting to just overwhelm them with beauty. Beauty is instantly seductive. A great graphic design can get approved before a stakeholder even takes time to understand what they are really looking at. But that is the dark side of design. That is using your power to create beauty to pervert the pursuit of truth. Beauty should lead someone toward the truth, not simply silence dissent. Forcing a busy stakeholder to take the time to go through the ugly details with you is another challenge.
Finally, design is never really done. Design is an activity that will consume all the time you give it. That means at some point, ready or not, someone is going to blow the whistle and say “pencils down, it’s time to ship something”. That is frustrating as well.
In the end - UX design means going to bat for the user. Fighting the good fight, sticking to your principles and enforcing your process. It means that sometimes your the bad guy. There will be yelling. There will be frustration and sleepless nights. But pursuing a vision is never easy. If it were, they would not need someone to lead them to it.