Micrograd From Scratch

This repository contains a from-scratch reimplementation of a minimal automatic differentiation engine and a fully connected neural network, inspired by Andrej Karpathy’s micrograd.

The goal of this project was not performance, but deep conceptual understanding of backpropagation, reverse-mode autodifferentiation, and neural network training.

---

What This Project Implements

1. Scalar Autograd Engine

• Custom Value class supporting automatic differentiation
• Operator overloading for arithmetic operations
• Dynamic computation graph construction
• Reverse-mode backpropagation via .backward()
• Correct gradient accumulation for reused nodes

2. Visualization

• Computation graph visualization using Graphviz
• Forward and backward flow inspection

3. Neural Network From First Principles

• Neuron, Layer, and MLP abstractions
• Fully connected architecture
• Tanh activation
• Manual parameter collection

4. Training Loop

• Mean squared error loss
• Manual gradient descent
• Parameter updates without external ML libraries

---

Why This Matters

Modern deep learning frameworks (PyTorch, TensorFlow) hide most of these details. This project reconstructs the core mechanics explicitly:

• How gradients flow through a computation graph
• Why backpropagation works
• How neural networks are trained via gradient descent on a composed function

---

Technologies Used

• Python
• NumPy
• Graphviz (for visualization)

---

References

• Andrej Karpathy — Neural Networks: Zero to Hero