Because he, more than anyone, is responsible for taking physics down the wrong path, leading to a theory that even Feynman admits is incomprehensible, while refusing to accept Schwinger’s formulation of Quantum Field Theory, a theory that does make sense. Well, that’s my opinion, even though many physicists disagree with me. Here’s how I describe it in my book “Fields of Color”, which explains Schwinger’s QFT to a lay audience:

When it came to the fields-vs.-particles debate, Feynman was strictly a particle man:

We know that light is made of particles because we can take a very sensitive instrument that makes clicks when light shines on it, and if the light gets dimmer, the clicks remain just as loud — there are just fewer of them. Thus light is something like raindrops — each little lump of light is called a photon... You might wonder how it is possible to detect a single photon. One instrument that can do this is called a photo­multiplier... You might say that it's just the photomultiplier that detects light as particles, but no, every instrument that has been designed to be sensitive enough to detect weak light has always ended up discovering the same thing: light is made of particles. — R. Feynman

It did not occur to Feynman that these discrete clicks could also be produced by light quanta collapsing into atoms of the photomultiplier — or if it did, he dismissed the idea. By overlooking or ignoring this possibility, Feynman was forced to accept a theory that he himself admitted doesn’t make sense:

It is not a question of whether a theory is philosophically delightful, or easy to understand, or perfectly reasonable from the point of view of common sense. The theory of quantum electrodynamics describes Nature as absurd from the point of view of common sense. And it agrees fully with experiment. So I hope you can accept Nature as She is — absurd. - R. Feynman

I consider it a tragedy that Feynman, perhaps the most influential physicist of his time, could not accept QFT, which is “philosophically delightful” and “agrees fully with experiment”.

Note added 1/14/2020: Because of the huge interest in this answer (30,000 views in one week), I’m adding another excerpt from my book that follows the passage above:

Fields vs. Particles (Round 3). For the third time in physics history there had been a momentous battle between particles and fields, and for the third time fields lost and particles won. In 1905, Einstein’s view of light as particles (later recanted) replaced Planck’s field quantum picture (Chapter 3). In 1933, Dirac’s particle theory of QM won over Schrödinger’s field picture. And then in 1948, Feynman's approach to renormalization based on particles and virtual particles won over Schwinger's fields, in large part because his diagrams proved easier to work with than Schwinger's field equations. Even though Feynman eventually changed his mind (see “Feynman converts” below), two generations of physicists have been brought up on Feynman diagrams and believe that nature is made of particles.

One cannot fault Feynman for finding a method that works and is easy to use, nor can one fault him for being on the "particle" side, as was Dirac and, for a time, Einstein. However, I do fault him for his dismissal of QFT, an elegant and consistent theory that explains so many things, as a "shell game". I even heard him make sneering references to Schwinger's field approach during a lecture before the American Physical Society. While "bringing computation to the masses", as Schwinger said in his eulogy for Feynman, Feynman gave them an admittedly absurd theory without any theoretical basis while ignoring a well-founded theory that provides a consistent, paradox-free picture of nature.

Feynman converts! Acording to Frank Wilczek, Feynman eventually lost confidence in his particles-only view of nature:

Feynman told me that when he realized that his theory of photons and electrons is mathematically equivalent to the usual theory, it crushed his deepest hopes… He gave up when, as he worked out the mathematics of his version of quantum electrodynamics, he found the fields, introduced for convenience, taking on a life of their own. He told me he lost confi­dence in his program of emptying space – F. Wilczek (W2008, p. 84, 89)

Unfortunately, Feynman’s “conversion” is not generally known. Most phys­icists today routinely use Feynman diagrams while promulgating and perpetuat­ing the particle picture, puzzling and paradoxical as that picture may be.

I suggest that you also look at my other “viral” Quora answer (over 450,000 views) about Julian Schwinger here.