Flipping the impulse response in convolution

During convolution on a signal, why do we need to flip the impulse response during the process?

Answer

Adapted from an answer to a different question (as mentioned in a comment) in the hope that this question will not get thrown up repeatedly by Community Wiki as one of the Top Questions....

There is no "flipping" of the impulse response by a linear (time-invariant) system. The output of a linear time-invariant system is the sum of scaled and time-delayed versions of the impulse response, not the "flipped" impulse response.

We break down the input signal $x$ into a sum of scaled unit pulse signals. The system response to the unit pulse signal $\cdots, ~0, ~0, ~1, ~0, ~0, \cdots$ is the impulse response or pulse response $$h[0], ~h[1], \cdots, ~h[n], \cdots$$ and so by the scaling property the single input value $x[0]$, or, if you prefer $$x[0](\cdots, ~0, ~0, ~1, ~0,~ 0, \cdots) = \cdots ~0, ~0, ~x[0], ~0, ~0, \cdots$$ creates a response $$x[0]h[0], ~~x[0]h[1], \cdots, ~~x[0]h[n], \cdots$$

Similarly, the single input value $x[1]$ or creates $$x[1](\cdots, ~0, ~0, ~0, ~1,~ 0, \cdots) = \cdots ~0, ~0, ~0, ~x[1], ~0, \cdots$$ creates a response $$0, x[1]h[0], ~~x[1]h[1], \cdots, ~~x[1]h[n-1], x[1]h[n] \cdots$$ Notice the delay in the response to $x[1]$. We can continue further in this vein, but it is best to switch to a more tabular form and show the various outputs aligned properly in time. We have $$\begin{array}{l|l|l|l|l|l|l|l} \text{time} \to & 0 &1 &2 & \cdots & n & n+1 & \cdots \\ \hline x[0] & x[0]h[0] &x[0]h[1] &x[0]h[2] & \cdots &x[0]h[n] & x[0]h[n+1] & \cdots\\ \hline x[1] & 0 & x[1]h[0] &x[1]h[1] & \cdots &x[1]h[n-1] & x[1]h[n] & \cdots\\ \hline x[2] & 0 & 0 &x[2]h[0] & \cdots &x[2]h[n-2] & x[2]h[n-1] & \cdots\\ \hline \vdots & \vdots & \vdots & \vdots & \ddots & \\ \hline x[m] & 0 &0 & 0 & \cdots & x[m]h[n-m] & x[m]h[n-m+1] & \cdots \\ \hline \vdots & \vdots & \vdots & \vdots & \ddots \end{array}$$ The rows in the above array are precisely the scaled and delayed versions of the impulse response that add up to the response $y$ to input signal $x$. But if you ask a more specific question such as

What is the output at time $n$?

then you can get the answer by summing the $n$-th column to get $$\begin{align*} y[n] &= x[0]h[n] + x[1]h[n-1] + x[2]h[n-2] + \cdots + x[m]h[n-m] + \cdots\\ &= \sum_{m=0}^{\infty} x[m]h[n-m], \end{align*}$$ the beloved convolution formula that befuddles generations of students because the impulse response seems to be "flipped over" or running backwards in time. But, what people seem to forget is that instead we could have written $$\begin{align*} y[n] &= x[n]h[0] + x[n-1]h[1] + x[n-2]h[2] + \cdots + x[0]h[n] + \cdots\\ &= \sum_{m=0}^{\infty} x[n-m]h[m], \end{align*}$$ so that it is the input that seems "flipped over" or running backwards in time! In other words, it is human beings who flip the impulse response (or the input) over when computing the response at time $n$ using the convolution formula, but the system itself does nothing of the sort.