In order to motivate why we separate the training and inference phases of neural networks, it can be useful to analyse the computational complexity.This essay assumes familiarity with analytical complexity analysis of algorithms, and hereunder big-O notation. If you need a recap, you should read the essay on computational complexity before continuing.Forward propagationLooking at inference part of a feed forward neural network, we have forward propagation.Finding the asymptotic complexity of the...