The Race to Control AI and Protect What Makes Us Human

Summary

AI safety researchers, including Evan Hubinger from Anthropic, have expressed serious concerns that misaligned AI could pose an existential threat to humanity within the next decade. This debate highlights the critical need for AI alignment, ensuring AI systems act according to human intentions. While some, like Bill Gates, believe AI can be a force for good if developed responsibly, others, like Stephen Cobb, emphasize human agency in preventing negative outcomes, drawing parallels to the tobacco industry's past practices. OpenAI's CEO Sam Altman has acknowledged safety concerns, leading to slowed model development and discussions with the White House about pacing AI advancements.

Full text

A current debate is whether artificial intelligence (AI) is a force for good or bad; or perhaps both. What follows is the argued opinion of the author – it is not the inevitable outcome of current AI development. On August 26, 2026, Bill Gates published a 6,000 word memo on his personal website, titled The turbulent AI era is here. The choices we make now are critical. It starts, “The transition to the AI era will be one of the most turbulent times in human history. Right now, we are not preparing adequately for that transition. If the world takes the right steps, AI will be a force for good and leave everyone better off.” The implication is that AI contains danger if we don’t behave responsibly toward it, but that we can and will probably learn to behave responsibly. Overall, AI is likely to be a force for good. Not everybody agrees. Responding to a tweet on X from Jacob Coxon, Evan Hubinger (a safety researcher at Anthropic leading the firm’s Alignment Science org) replied on September 9, 2026, “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.” Could Misaligned AI Kill Us All? Coxon, a former AI researcher at both OpenAI and Anthropic, had suggested “The people building AI earnestly believe that it could kill us all by the end of the decade.” Advertisement. Scroll to continue reading. Hubinger’s reply focuses on ‘alignment’, saying we cannot yet control it. Alignment in AI is the assurance that AI actions are tied to its developer’s intentions. Misalignment, in the vernacular, is what allows AI models to go rogue – to take unintended actions that can result in attacking unintended targets. Hubinger is saying that we are not on track to solving this problem. The problem would be worsened if we achieve, but misalign, future Artificial General Intelligence (AGI). AGI is an ill-defined concept that effectively implies artificial intelligence that can match or surpass human capabilities across all cognitive tasks and economic domains. Misaligned AGI could lead to a technological singularity where the technology accelerates beyond human control; that is, the AI will begin to ‘reprogram’ itself by itself. This is the potential result if we fail Gates’ warning to take the right steps today. But market forces work against this. Quite simply, the developer of the most powerful AI is likely to become the most powerful and richest developer of AI. AI has been advancing at a phenomenal speed, and nobody really knows where it is going. The debate between the potential effect of this is widening. Stephen Cobb, a former senior security researcher at ESET, and now an independent researcher, said on LinkedIn, “We’ve been told: ‘The people building AI earnestly believe that it could kill us all by the end of the decade.’ To which I say: Not unless we allow it or enable it,” (taking the Gates’ line of thought). Dan Tynan responded, “Neither will cigarettes. You have to stick them in your mouth and light them. But the companies making cigarettes knew about the dangers for decades, and hid them, knowing that their product was so addictive, and so pervasive, it would overcome people’s normal inclination towards self-preservation.” Cobb believes in humanity’s good sense, while Tynan believes in market forces. There is, however, one promising sign for the future. OpenAI’s CEO Sam Altman is not an innocent in AI. On September 11, 2026, Bloomberg reported: “OpenAI said it has slowed parts of model development and paused certain internal AI training recently due to safety concerns. In July, Altman also said that he’d spoken with White House officials about the ‘need’ to pace AI development.” Altman sat down with Fortune’s Editor-in-Chief Alyson Shontell to discuss the high-stakes balancing act of AI in an episode that aired on Saturday, September 12th (embedded below). It seems the AI developers are recognizing the potential danger of headlong development. It may be that Gates’ request for doing the right thing now may yet be heeded, and we may still avert the destruction of the human species. But let us return to his 6,000 word memo. “One preliminary survey,” he writes, “suggested that heavier AI use was associated with less critical thinking. The effect was stronger for younger people.” So even if we succeed in aligning AI models for only beneficial purposes, the effect is still likely to be a degradation in the human ability to reason for itself. It could be argued that the ability to reason is what distinguishes humanity from the world’s other animals. We may succeed in preventing the destruction of the human species by AI, but can we prevent the destruction of the distinguishing humanity of humans? Learn More at the AI Risk Summit Related: Anthropic Chief Says AI Industry Needs to Give Safety Measures Time to Catch Up Related: Users in Houthi-Held Yemen Tried to Develop Advanced Weapons With AI, Anthropic Says Related: Kiteworks Acquires Bonfy.AI to Fill the AI Gap in Data Governance Related: Anthropic Says Russian Hackers Used Claude AI to Automate Malware Evasion Written By Kevin Townsend Kevin Townsend is a Senior Contributor at SecurityWeek. He has been writing about high tech issues since before the birth of Microsoft. For the last 15 years he has specialized in information security; and has had many thousands of articles published in dozens of different magazines – from The Times and the Financial Times to current and long-gone computer magazines. Daily Briefing Newsletter Subscribe to the SecurityWeek Email Briefing for the latest cybersecurity threats, trends, and expert insights. More from Kevin Townsend Phishing Research Challenges Conventional Security Awareness TestingKiteworks Acquires Bonfy.AI to Fill the AI Gap in Data GovernanceHacker Conversations: Vinnie Liu, Performer Turned RingmasterDeceptive Android Apps Exploit Google Play Early Access to Evade ReviewsAI Is Giving Lesser-Resourced Attackers Nation-State-Level Reach, Google WarnsUS Agencies Warn China Is Systematically Extracting Frontier AI CapabilitiesNew Phishing Attack Creates Malicious Pages Inside the Victim’s BrowserThe Hidden Instructions That Can Hijack AI Agents Latest News New Warnings About the Risks of AI to Humanity Revive a Long-Running DebatePersonal, Financial Info Exposed in Revolut Data BreachChinese Hackers Exploit Critical Tencent Software Flaw for One-Click Code ExecutionCISOs Race to Control AI Agents Without Destroying Their ValueTelus Warns Customers of Account BreachesThree JFrog Artifactory Flaws Exploited for Backdoor DeploymentConnectWise Patches ScreenConnect Vulnerability Exploited in Worm-Like AttacksAnthropic CEO Dario Amodei Says AI Industry Needs to Give Safety Measures Time to Catch Up Trending Daily Briefing NewsletterSubscribe to the SecurityWeek Email Briefing to stay informed on the latest threats, trends, and technology, along with insightful columns from industry experts. Virtual Event: Attack Surface Management Summit 2026 September 16, 2026 Join as speakers examine the various components of ASM strategy, the push to mandate continuous asset visibility and inventory tools, and the use of red-teaming, bug bounties and pen-tests in modern security programs. Register Webinar: Minimum Viable Business: Can You Prove Your Organization Would Recover? September 2, 2026 In this live webinar, learn how to define your minimum viable business, identify the systems it depends on, measure actual recovery time against business requirements, and present the gaps to the board as measurable risk. Register People on the MoveZero Networks has named Yossi Dagan as Chief Financial Officer.Manifold has appointed Joe Sullivan to its Board of Directors.Patrick McKinney has joined Turing as Chief Information Security Officer.More People On The MoveExpert In

Entities