Imagine trying to swim by nervously shaking your arms and legs in every direction in some chaotic and out of sync manner. That wouldn’t be very efficient, would it?

DDPG is mainly used for continuous control tasks, such as locomotion. In this setting, the actions [math]a[/math] often correspond to exerting forces so as to move a physical body. Forces directly translate to accelerations, so they need to be integrated twice to affect the position [math]x[/math] of the system that we wish to control.

[math]m \ddot x = a [/math]

[math]x = \frac{1}{m}\int\int a[/math]

Yet, the integration operator is a low-pass filter, which means that any high-frequency present in the action [math]a[/math] will be dampened.

(See the integrator Bode diagram: Control Systems/Bode Plots)

Hence, if you generate a decorrelated random signal with zero mean, its effect will be averaged over time and the system will simply oscillate in place without making much progress. Here is an example with Gaussian noise [math]N(0,1)[/math].

On the contrary, if the noise generated at a given timestep is correlated to previous noise, it will tend to stay in the same direction for longer durations instead of immediately canceling itself out, which will consequently allow to increase velocity and unfreeze the position. Here is the same example, but with an Ornstein-Uhlenbeck process generated with [math]N(0,1)[/math] and time constant of [math]1s[/math].

The velocity and position are consistently pushed in the same direction, which indeed favors exploration.