Deep Learning Framework from Scratch
forgeNN
forgeNN is a deep learning framework built entirely from scratch in Python and NumPy, without any automatic differentiation libraries. It implements a dynamic computation graph with reverse-mode autograd, mirroring how PyTorch tracks gradients at runtime. The goal was to understand every layer of abstraction that frameworks like PyTorch and TensorFlow abstract away, from the chain rule to memory-efficient convolution via im2col.
Motivation
Most practitioners use deep learning frameworks as black boxes. Building one from the ground up forces a genuine understanding of backpropagation, numerical stability, and the engineering tradeoffs that production frameworks make. forgeNN started as a learning exercise and evolved into a fully-featured library published on PyPI.
Technical Approach
The core is a Tensor class that records operations into a dynamic computation graph. Gradients flow backward through this graph via a topological sort and recursive chain rule application. Conv2D is implemented using the im2col transformation to express convolution as a single matrix multiplication, enabling NumPy's BLAS-optimized routines. Multi-head attention and transformer blocks are built on the same primitives. Adam and AdamW optimizers are included with correct bias correction.
Results
forgeNN achieves above 95% accuracy on MNIST with a convolutional network. On small batch inference benchmarks, it outperforms PyTorch on CPU by 2.2× for the specific workloads tested, primarily because it avoids Python-side dispatch overhead for simple feedforward paths. The library has accumulated over 2,000 downloads on PyPI.