Inner Products & Orthogonality
The single most important operation in DSP
This is the lesson that ties everything together. The inner product (multiply two signals together, then sum the result) is the single most important operation in DSP.
Filters use it. Correlation uses it. The DFT? The DFT is literally nothing but a list of inner products.
The Inner Product
Below are two sinusoids. The first slider controls signal A's frequency (f₁), the second controls signal B's frequency (f₂). The third panel shows the product A·B at every sample. The fourth panel shows the running sum of that product, the inner product so far.
Set both sliders to the same value. The product panel is all positive (sin²(2π·f·t) ≥ 0 on average), and the running-sum panel climbs steadily upward. The two signals correlate, they have something in common.
Now bump one slider so f₁ ≠ f₂. The product panel oscillates between positive and negative, and the running sum collapses back toward zero. The two signals are orthogonal, independent, no overlap.
Why This Matters for DSP
The DFT, the most important algorithm in DSP, is exactly what you just did. Take your signal, take inner products with sin and cos at every frequency from 0 to fs/2, return the list. Every spectrum analyzer, every audio plugin, every WiFi receiver, runs this operation under the hood.
Filters are inner products too. A convolution between input x and impulse response h is, at every sample, an inner product of x with a flipped + shifted h. Every filter you'll ever use is built on this.
Correlation, the operation that asks "how similar are these two signals?", is an inner product. Matched filters in radar? Inner products. Pattern detection? Inner products. Even the geometric idea of "angle between two vectors" generalizes from school-geometry vectors to signals via the inner product.
You now have every tool you need to understand convolution and the DFT. Onward.
Frequently Asked Questions
What is the formal definition of an inner product?
For real-valued sampled signals x and y: <x, y> = Σ x[n] · y[n]. For complex signals (like the complex exponentials used in the DFT), one of them gets conjugated: <x, y> = Σ x[n] · y[n]*. The conjugation matters for getting the math to work cleanly with rotating phasors.
Why are different-frequency sinusoids orthogonal?
Their product is itself a sum of sinusoids at the sum and difference frequencies (a trig identity from the Trig chapter). Each of those sinusoids integrates (or sums) to zero over a full period, just as you saw in the Integrals lesson. So the inner product is zero. Same frequencies are different: their product has a non-zero average (sin² or cos²), so the sum grows.
What if my signals are not perfectly periodic?
Orthogonality becomes approximate, and the DFT shows spectral "leakage" — energy at one frequency bleeds into neighboring frequency bins. Windowing functions (Hann, Hamming, Blackman, etc.) reduce this leakage by tapering the signal at its edges before the inner products. The orthogonality principle still applies; it just needs more care in practice.
Quick Check
Test your understanding of the key concepts from this lesson.
