Gradient Descent into Madness - Building an LLM from scratch
Setup
from typing import Any, Optional, List
import networkx as nx
LLM from scratch: Automatic Differentiation
I’m building a modern language model with all the bells and whistles completely from scratch: from vanilla python to functional coding assistant. Borrowing (shamelessly stealing) from computer games, I’ve built a tech tree of everything that I think I’ll need to implement to get a fully functional language model. If you think anything is missing, please let me know:
The LLM from scratch...
Read more at bclarkson-code.github.io