A few months ago, I had the idea of making a LLM from scratch as a personal project (for learning and partly for improving my resume). Since I learned a lot from other posts on here over the past year, I wanted to share the results. TLDR: I trained a 1.1B param model on 20B tokens from fineweb-edu,
