A.I. companies are racing to build systems more capable than any human, in any domain, and some of the world's top researchers think it could kill us all. In this episode, physicist-turned-A.I.-safety-advocate Max Winga of Control A.I. joins Eben Pagan to explain why superintelligent A.I. could arrive within the next few years, why the companies building it don't actually understand how it works, and what happens if we lose control of it entirely. From A.I.-generated cyberattacks that shook the NSA to a chilling breakdown of how humanity could quietly hand over the future without ever noticing, this is one of the most direct conversations about A.I. risk I’ve had.
A.I. Safety Policy Advocate | Control A.I.
Max Winga works at Control A.I., a nonprofit campaigning to prohibit the development of superintelligent A.I. due to the extinction-level risks it poses. He has personally briefed hundreds of lawmakers and staffers across the US, UK, Canada, and Germany, and has helped drive campaigns that secured public backing from more than 100 UK and Canadian lawmakers.
Former Technical A.I. Safety Researcher | Conjecture
Before shifting to full-time policy work, Winga worked as a research engineer at Conjecture, an A.I. safety research company, applying his technical background to alignment research. His research work focused on understanding and mitigating the risks posed by increasingly capable A.I. systems.
Physicist | University of Illinois Urbana-Champaign
Winga holds a degree in Engineering Physics from the University of Illinois Urbana-Champaign (2024), where his coursework and research included quantum computing simulation and machine learning applications in biophysics. His technical grounding in physics and math directly informed his transition into A.I. safety research.
Superintelligent A.I. Defined | Max Winga explains what superintelligent A.I. is and why experts believe it could arrive within the next several years.
Physics To A.I. Safety | Winga describes how his background in physics led him from technical A.I. alignment research into full-time policy advocacy.
A.I. Systems Grown Not Built | Winga breaks down why A.I. models are trained rather than programmed, making their internal behavior difficult to understand or control.
Spiky A.I. Capabilities | Winga explains why A.I. can be superhuman at coding and chess while still failing at simple tasks like counting letters.
Exponential A.I. Progress | Winga compares the trajectory of A.I. capability growth to the early exponential spread of COVID-19.
A.I. Task Time Horizons | Winga discusses new research showing A.I. systems can now complete tasks that would take human experts up to eighteen hours.
Recursive Self-Improvement | Winga outlines how A.I. companies are racing to build systems that can design their own successors, potentially triggering an intelligence explosion.
Gradual Disempowerment Risk | Winga describes a scenario where humans slowly hand over decision-making to A.I. without any single dramatic loss of control.
Mythos And Government Response | Winga explains how a single A.I. system uncovered thousands of software exploits, prompting a swift government export-control response.
Taking Action On A.I. Risk | Winga lays out concrete steps individuals can take, from contacting lawmakers to joining local advocacy groups.
00:00 - 04:30 | What Superintelligent A.I. Actually Means
04:30 - 09:15 | From Physics Research To A.I. Safety Advocacy
09:15 - 14:00 | Why A.I. Systems Are Grown Not Built
14:00 - 18:30 | The Spiky Nature Of A.I. Capabilities
18:30 - 23:00 | Exponential Progress And The COVID Comparison
23:00 - 27:30 | Task Time Horizons And The Death Of Clock Time
27:30 - 32:00 | Recursive Self-Improvement And Humanoid Robots
32:00 - 36:30 | Gradual Disempowerment And Losing Human Agency
36:30 - 41:30 | The Mythos Exploits And Government Regulation
41:30 - 47:10 | What Individuals Can Do Right Now
"Mitigating the risk of extinction from A.I. should be a global priority alongside other societal scale threats such as pandemics and nuclear war."
"Anthropic roughly understands three percent of what's going on inside these A.I. systems, and that's probably an overestimate."
"A.I. is not like a human that gets smarter in the way that a human does."
"Your intelligence limits your imagination to kind of whatever your intelligence level is."
"This is the worst they'll ever be."
"You're basically putting a bioweapon design machine out to the public and allowing anybody to access it for twenty dollars a month."
"The government's job is to make sure that you do not die."
"It's in nobody's interest to build a superintelligent A.I. that threatens all of humanity."
"Unfortunately, it's real this time."
"This is such a low effort thing to do, and it has a really big impact on scale when many people do this."
Accelerate your productivity & creativity using 47 custom prompts to grow your income, success & quality of life by downloading our free “AI Accelerator Pack” HERE.
START HERE
