So what we understand theoretically is this:
Gluons are vector gauge bosons, the generators of the SU(3) (or the color) gauge symmetry. So they only couple with the particles having a non-zero color charge: {Red, Green, Blue} (they're also called SU(3) quantum numbers).
Other particles (like photons) which have no color charge are singlet with respect to the action of this symmetry, or in other words they transform trivially under SU(3) and hence remain unaffected by the SU(3) gauge bosons, i.e. the gluons.
While on the other hand, quarks do have a non-zero SU(3) charge, they form an SU(3) triplet and therefore, transform non-trivially according to the fundamental representation of SU(3). Hence having SU(3) charge, these are acted upon by the gluons and transformed.
Physically it could be understood in the following way:
The gluon interacts with a quark and rotates it in the (internal) color space. So in the above diagram, a gluon acts on a red-colored quark and transforms it into a blue-colored quark.
An analogy with the electromagnetic interactions, which are U(1) gauge symmetric, is very clear:
The generator of this U(1) is another vector gauge boson i.e. the photon. And it couples with anything having a non-zero U(1) charge, which is nothing but the good old electric charge (also called the U(1) quantum number).
So photon interacts with charged particles like electrons, quarks and W+/W- etc. while it electromagnetically doesn't affect the gluons, Z bosons, neutrinons or the neutral pions etc.
The similarity in the above diagrams is clear: the photon interacts with an electron with positive charge (or a positron if you like) and rotates it in the internal U(1) space. It acts on it to transform its electric charge from +1 to -1, just like gluon turns red to blue.
But it's important to keep in mind that there is also a very subtle difference between the two, which arises because of the fact that SU(3) is non-Abelian unlike U(1). This changes many things like: there are 8 gluons while only 1 photon, gluons interacts with themselves while photons don't etc.
PS: I edited the question since neutrons do have strong interactions. Of course, they are made up of quarks and gluons, after all.