This week Andrew Barto and Rich Sutton were awarded the ACM Turing Award, the highest award in the field of Computer Science, for their pioneering contributions to the field of Reinforcment Learning. With the advances in Artificial Intelligence over the last few years, pioneering work in this field has been rightfully acknowledged and rewarded. In 2024, neural network pioneers John Hopfield and Jeffrey Hinton were awarded the Nobel prize in Physics.

The concepts of reinforcement learning is rooted in basic principles of animal behavior. As everyone who has trained a pet knows, we can reinforce certain behavior using “carrots” and discourage unwanted behavior using “sticks”. Animals learn this behavior with proper training. Alan Turing’s seminal paper in the early 1950’s proposed this same idea for machines to learn based on “rewards” and “punishments”. Barto and Sutton built on this idea, and used principles from psychology to formulate the fundamental principles of training computers used on reinforcement learning.

One of the most high profile success for RL came when Google’s DeepMind defeated the human world champion in the game of Go. The game of Go was one of the hardest challenges for computers to beat humans and Google’s Deepmind used RL to win the tournament. I highly recommend this documentary on AlphaGo, if you have not yet watched it. Even the most recent success of LLMs such as ChatGPT are based on a critical step of RLHF (Reinforcement Learning based on Human Feedback).

It is exciting to see what other CS breakthroughs related to AI get recognition over the next few years. The breakthrough of transformer architectures is certainly a huge one. Almost all deep learning success in the last few years has been based on the transformer architecture.