The math of DSP, part 5: Orthogonality

Part 4 looks at convolution, the Fourier series, and the Nyquist sampling theorem.

ORTHOGONALITY
The term orthogonality derives from the study of vectors. Most likely you have run across the term in basic trigonometry or calculus. By definition, two vectors in a plane are orthogonal when they are at a 90° angle to each other. When this is the case, the dot product of two vectors is equal to zero:

The main point here is that the idea of multiplying two things together and getting a result of zero has been generalized in mathematics under the term orthogonality.

We will get back to this shortly, but let's look at another case where an interesting function has a zero value: the average value of a sine wave. Figure 4-14 shows one cycle of a sine wave. We have shaded in the area under the curve for the positive cycle and the area above the curve for the negative cycle. Notice that the area for the negative portion of the waveform is labeled with a negative symbol. A "negative area" is a hard concept to imagine, but be reassured that we are simply talking about an area that has a negative sign in front of it. If we add the two areas together we will, naturally, get a value of zero. This may seem too obvious to bother pointing out, but it is just the first step.

Figure 4-14. The average area under a sine wave is zero.

Insider Info
As an interesting side note, this fact was used in the early days of electricity to "prove" that AC voltages were of no practical use. Since they averaged to zero, so the analysis went, they could not do work!

The process of integration can be viewed as finding the area under a curve. Therefore, you can write this idea mathematically as follows, for any integer value of k:

Now, if you multiply by a constant, on both sides of the integral, the result is still the same:

That is, the amplitude of the waveform may be larger or smaller, but the average value is still zero.

Now we come to the interesting part. What if we put in, not a constant, but some function of time? That is:

The answer naturally depends upon what our function of g(t) is. But as we saw in the last chapter, we really only need to worry about sinusoidal functions for g(t). We can extend our analysis to other waveforms by simply considering the Fourier representation of the waveform. Let's look at the specific case where g(t) = sin ηt.

Equation 4-57 is called the orthogonality of sines. It tells us that, as long as the two sinusoids do not have the same frequency, then the integral of their products will be equal to zero. This may be a little hard to visualize. If so, think back to Equation 4-55. When the frequencies are not the same, the amplitude of the resulting waveform will tend to be symmetrically pushed both above and below the x-axis. This may result in some strange-looking waveforms but, over time, the average will come out to zero. In effect, even though g(t) is a function of time, it will have the same effect as if it were the simple constant A.

So what about the case when η = ω? If we substitute ω for η in Equation 4-57:

That is, we get the sum of the square of the sine wave. When we square the sine waveform, we get a figure like the one shown in Figure 4-15. Since a negative value times a negative value gives a positive value, the negative portion of the original sine wave is moved vertically above the x-axis. The resulting waveform is always positive, so its average value will not be zero.

(Click to enlarge)
Figure 4-15. The average of the square of a sine wave is greater than zero.

So far the discussion has made use of analytical functions which are useful in developing algorithms and theoretical concepts. As a practical matter, however, in DSP work we are generally more interested in testing a sequence of numbers (the sampled signal) for orthogonality. At this point, we need to take a slight diversion through the subject of continuous functions versus discrete sequences.