I implemented a neural network from scratch in numpy.
It is meant to be easily extendable with new layers, losses, activations and so on.
If I feel like it, I might try to add support for more complicated network structures which require us to trace the computational graph in each forward pass in order to properly execute the backward pass.
Note: this took less than a day, so there is nothing very fancy at the moment.