Why Backprop Goes Backward

Backpropagation must run backward because computing gradients forward would repeatedly propagate the same intermediate terms, causing quadratic time instead of the linear time achieved by the backward pass.