Martin Riedmiller — Control Team Lead, Google DeepMind

My interview with Martin Riedmiller, the Control Team Lead at Google DeepMind. In 2023 I went to the International Conference On Machine Learning Waikiki Beach, Hawaii and on the first night I went to see Shoot Ogawa, multiple time Close Up Magician of the Year winner. Martin happened to be at the show, and the next day we bumped into eachother at the conference and hit it off from there. You can find links and videos to many of the things we discuss, such as the RoboCup competitions on Martin's website.

Listen or watch on your favorite platforms

💡 Key Moments

• Introduction and Magic Trick (00:00:01): Introduction and discussion about meeting at a conference, shared interests, and an icebreaker with a magic trick.
• Early Interest in Computer Science (00:08:14): Martin Riedmiller discusses his early interest in computer science and programming at a young age.
• Impact of Resilient Back Propagation Algorithm (00:16:41): Discussion about the impact and success of the resilient back propagation algorithm (PROP) and its influence on Martin's career trajectory.
• Decision to Pursue Ph.D. (00:19:53): Martin's decision-making process for choosing his Ph.D. topic and its relation to board games, reinforcement learning, and robotics.
• Transition to Robotics (00:21:48): Martin's transition to robotics, application of reinforcement learning, and participation in RoboCup competitions.
• Research and Evolution (00:28:26): Martin Riedmiller's work across different universities and problem spaces, focusing on reinforcement learning and robotics.
• Joining DeepMind (00:30:27): Martin's experience of joining DeepMind and the motivation behind working on deep reinforcement learning.
• Breakthroughs in the DeepMind Atari Team (00:32:22): The significance of deep Q networks, learning from raw pixels, experience replay and generalisation in advancing the field of reinforcement learning.
• Deterministic Policy Gradient (00:40:40): Discussion on the critical step of moving from discrete action spaces to continuous action spaces in reinforcement learning.
• Artificial General Control Intelligence (00:42:49): Martin's concept of AGCI and its focus on low-level controls and the impact of continuous action spaces on AI pursuit.
• Nuclear Fusion Reactor Project (00:45:00): The collaboration with EPFL on controlling a tokamak reactor using reinforcement learning and its significance.
• Potential of reinforcement learning in process control (00:54:41): The conversation explores the untapped potential of reinforcement learning in complex control problems like chemical plants and automotive engines.
• Assessing the value of reinforcement learning (00:56:32): The discussion addresses the approach to evaluating whether reinforcement learning can outperform traditional control techniques in specific control settings.
• Differentiating the current AI hype (00:58:36): Riedmiller reflects on the current AI hype and discusses the differences and similarities with previous hype cycles.
• The "scaling effect" and its impact (01:01:33): The conversation delves into the scaling effect in language models and its potential impact on domains outside of language, like reinforcement learning.
• Fundamental principles for autonomous robots (01:06:36): Riedmiller shares insights on the fundamental principles and breakthroughs needed to enable robots to perform a wide range of useful autonomous tasks.
• Reflections on career and achievements (01:08:11): Riedmiller reflects on his career, decisions, and the aspects he is most proud of, both scientifically and personally.
• Future projects and interests (01:11:36): The conversation explores Riedmiller's potential interest in working on autonomous control systems for extreme sports like mountain biking or skiing.
• Book recommendation (01:13:17): Riedmiller suggests Daniel Kahneman's "Thinking, Fast and Slow" and Hermann Hesse's "Siddhartha" as notable reads.