The current generative AI boom is built on the foundations of the Transformer architecture used to create the large language models (LLM). The technical details of the Transformer architecture was described in the Google paper that first introduced it: “Attention is all you need“. Unless you are a trained data scientist or a machine learning engineer, the details in that paper will not make much sense to you.

There are plenty of good resources on the internet that go into details of this architecture to explain it to someone who has basic engineering background. I’d like to highlight 2 sources that I have found very helpful.

The first resource that I’d like to point you to is Jay Alammar’s blog. Jay has since expanded the blog to include lots of videos as well to explain these concepts.

The second resource that I’d like to point you to is the amazing 3 Blue 1 Brown math videos channel. Grant Sanderson has created some really intuitive visualizations to explain math concepts. He also has an entire series of videos dedicated to explaining deep learning and transformer architectures. Check out his neural network series of videos to get a good understanding of the topic.