A small amount of low order distortion can go unnoticed but this does not hold true when nonlinearities are large. This is almost always the case at low frequencies. Here again we see many people saying bass distortion is less noticable than higher frequencies, usually pointing to the Fletcher Munson curve or something like that. If it were just harmonics that were generated that might be true but the thing that tips the scales is intermodulation distortion.

Harmonic distortion is easy to measure and gives an indirect indication of nonlinearity. Whether or not harmonic distortion is "euphonic" or not is not the issue. The same nonlinearities that create harmonic distortion are what also cause intermodulation distortion. IM distortion generates nonharmonically related tones that easily stand out. When two or more frequencies are present in the input signal, nonlinearities in the speaker produce sub and difference tones. Not only that but the harmonics also combine to generate sum and difference tones, creating high order intermodulation. The result is a garbled sound that is not harmonically related.

It is easy to demonstrate the harmonic spectrum of distortion in a loudspeaker system. For constant SPL the displacement decreases as 1/f2 with increasing frequency and it could be expected that distortion decreases correspondingly. Distortion in a speaker is caused primarily by three factors: force factor Bl(x), stiffness of suspension Kms(x), and electrical inductance Le(x). The effect of Bl(x) and Kms(x) should decrease as 1/w2 with frequency, but since the inductance increases as wLe its effect should only decrease as 1/w and possibly overtake the effects of the other two non-linearities.

In a nonlinear system the relationship between input 'x' and output 'y' is often of the form
y = a0 + a1 x + a2 x2 + a3x3 + .....+ anxn
which is a power series expanded around x = 0.

If the input is a sinusoidal voltage 
x = A sin(wt), then
x2 = A2 [1 - 0.5 cos(2wt)]
x3 = A3 [0.75 sin(wt) - 0.25 sin(3wt)]
x4 = A4 [0.375 - 0.5 cos(2wt) + 0.125 cos(4wt)]
and so on for higher powers of x.

Thus harmonics nw are generated and dc terms. w will also decrease by dA, but the second harmonic amplitude will decrease by dA2, the third harmonic amplitude by dA3, the fourth by dA4, and so on, at least to a first order approximation.

This by itself is not the biggest problem. The implication is however. Wherever you see harmonic distortion, it is created by nonlinearity. Wherever you see nonlinearity, you will find intermodulation distortion. That means even low order distortion is bad, because it indicates nonlinearity in the system.