The head of Microsoft AI has warned that without adequate safeguards, the development of AI could lead to the emergence of a new "silicon species" which competes with humans.
Mustafa Suleyman said rival firm Anthropic is treating AI like it is human, an approach he called "misguided" which could create technology that humanity cannot control.
Speaking to the BBC's Today programme, the chief executive said it was "right" for people to be concerned about AI, but there were "very practical things that we can do" to control it.
The comments are the latest in a series of stark warnings from the AI industry about the technology's possible dangers. Anthropic has been approached for comment.
Speaking to the Today programme on Thursday, Suleyman warned against firms continuing to create AI systems capable of setting their own objectives, earning money and owning assets.
That would mean "essentially seeding a new silicon species", he said, which could "no doubt compete with us for resources, no matter how much it cares about humanity and loves us".
Suleyman's comments to the BBC follow an essay he published earlier this week, in which he questioned Anthropic's approach to training its AI model Claude, despite praising boss Dario Amodei and his team as "intellectually honest people".
In it, he criticised Anthropic for teaching its AI to have human-like qualities, a practice known as anthropomorphising, which he said made it seem as though Claude had its own desires, values and sense of self.
He also warned tech firms risk creating something "impossible" to control by treating the technology like a human.
"AIs are not conscious," he wrote.
"They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.
"They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans."
Bloomberg via Getty Images
Dario Amodei is the head of the AI firm Anthropic, which Suleyman criticised for treating its AI as if it were human
The Microsoft boss said greater transparency was needed around how AI systems are trained and evaluated.
This, he said, included independent scrutiny of AI behaviour and stronger tools to monitor and control the technology.
"We must not sleepwalk our way into a decision we later come to bitterly regret," he wrote.
Dame Wendy Hall, professor of Computer Science at the University of Southampton, described the comments as "the sort of conversation we need to be having internationally", contrasting it with the "histrionics" from some AI companies which she said only served to "scare everyone".
'Humanist' superintelligence
The question of how much autonomy AI systems should have is increasingly dividing the industry.
Speaking to the BBC about an upcoming AI summit, Suleyman called for "alignment" to create technology that must be "subordinate" to humanity.
Alignment is a field which aims to build human ethical ideas and principles into AI. In other words, it aims to keep it on track with what humans value.
"I think that the good news here is that everybody who is human is going to have a very strong interest in making sure that the systems that we all create and... are used around the world in every nation are safe and controllable and subordinate to humanity," he said.
"Everybody must be aligned."
Microsoft founded its own superintelligence team in October 2025 and the company, much like its rival Anthropic, is also pursuing efforts in advanced AI development.
Suleyman's comments reflects Microsoft's initial draft of its Humanist AI Code of Conduct, a "training manual" published on Monday for how the company is developing its AI, which sets out the idea of "humanist superintelligence".
This, the code states, is "very advanced AI that always works for people, stays within limits, and remains under human control".
The company's vision of a "humanist" AI highlights the central challenge facing the technology - how to make it more powerful, without losing sight of the values it is meant to serve.
Additional reporting by Henry Moore
