When we offered CS231n (Deep Learning class) at Stanford, we intentionally designed the programming assignments to include explicit calculations involved in backpropagation on the lowest level. The students had to implement the forward and the backward pass of each layer in raw numpy. Inevitably, some students complained on the class message boards:“Why do we have to write the backward pass when frameworks in the real world, such as TensorFlow, compute them for you automatically?”This is seeming...