While I wouldn’t be surprised if Ed Witten did say this, it’s not as if it is something that he came up with—mathematical physicists have always known that QFT lacks mathematically rigorous foundations.
Funnily enough, not even all physicists—including those that work with QFT—are aware that this is the case. From their perspective, they are able to do computations and get out predictions that have been experimentally verified so… what’s wrong? The short answer is that, yes, there are techniques that physicists have for making computations but nobody really knows why they are valid and, unfortunately, saying that they are valid “by experiment” is a bit questionable from a theoretical standpoint. (Even though, practically, it is not a horrible solution.)
Let me provide an analogy that might help to explain. Suppose that nobody really knew how to compute integrals, but physicists came up with a handy rule: in order to compute the integral of a polynomial like
[math]\displaystyle \int_0^1 (x^2 - 3x + 1) \ \text{d}x, \tag*{}[/math]
just take your polynomial, for each term raise the power by one and divide by it, and then evaluate this new polynomial at the endpoints and subtract the two—e.g.
[math]\begin{align*} \int_0^1 (x^2 - 3x + 1) \ \text{d}x &= \left.\frac{1}{3}x^3 - \frac{3}{2}x^2 + x\right|_{x = 0}^{x = 1} \\ &= \frac{1}{3} - \frac{3}{2} + 1 \\ &= -\frac{1}{6}. \end{align*} \tag*{}[/math]
The physicists might have some intuitive ideas for why this rule should work and they might have an interpretation of such an integral as the energy required to do something. They might then do some experiments to confirm that the prediction obtained this way exactly agrees with what is observed in reality.
But there is a bit of a problem here. The integral [math]\int_0^1 (x^2 - 3x + 1) \ \text{d}x[/math] has specific mathematical meaning. There is a fact of the matter about whether or not it is actually equal to [math]-1/6[/math]; it is not a case where we could define it to be that if we really felt like it. This means that one of two things has to be true: either there must be some way to prove mathematically that the result is [math]-1/6[/math], or the result is not [math]-1/6[/math] and the way that physicists are describing their models and experiments is simply wrong and they shouldn’t be talking about integrals at all. In that case, it would be nice to work out what should we be talking about, anyway.
(Of course, in the real world, we do know how calculus works and we know that the rule stated above is an easy consequence of the Fundamental Theorem of Calculus, which itself is not so difficult to prove from first principles.)
In any case, quantum field theory is in a similar state: there are rules that physicists have for doing computations (which, to be fair, are better motivated than the example above) but nobody really knows what is the right setting in which those computations actually correspond to something justifiable. Take, for example, the Feynman path integral—in quantum field theory, this has a very important role in that it is supposed to give a way of averaging out over all possible paths that a particle could take in space. We call it an integral… but is it actually? If it is, what type of integral is it? It’s certainly not the usual Riemann integral. There are all sorts of proposed definitions, but it is not known how to justify all of the computational tricks in quantum field theory using any of them. Some approaches to quantum field theory opt to ignore the path integral entirely and build up other aspects.
I am not a mathematical physicist and so this is rather outside of my field of expertise. However, you might start by looking at some of these references on StackExchange: