Could an AI with a knowledge cutoff before Attention Is All You Need independently invent the Transformer architecture?
Via r/LocalLlama
Thursday, Aug 13, 2026 · 2:51PM
Summary
If you took today’s most capable reasoning model, but trained it only on knowledge available before 2017—before Attention Is All You Need—could it independently discover the Transformer architecture?
