MILWAUKEE (CBS 58) — A stark warning from inside one of the world's leading artificial intelligence companies is raising new questions about just how dangerous AI could become.

Jacob Coxon, a former researcher at Anthropic and OpenAI, resigned from Anthropic this week, saying some of the people building the technology believe it could ultimately threaten humanity.

On the social media platform X, Coxon wrote, "Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."

"The researchers talking about this are credible, and we should be paying attention," said Chris GauthierDickey, head of the Dwight and Dian Diercks School of Advanced Computing at the Milwaukee School of Engineering.

Coxon's concerns echo warnings from others inside the AI industry.

Anthropic CEO and co-founder Dario Amodei said in a 2023 interview that he believed there was a significant chance of AI causing catastrophic consequences.

"I think I've often said that my chance that something goes really quite catastrophically wrong on the scale of human civilization might be somewhere between 10% and 25%," Amodei said.

At MSOE, researchers are also exploring how AI can be used for practical applications. The university has its own supercomputer, Rosie.

"Finding out if cells look like they have cancer or understanding which trees to cut in Milwaukee when storms happen so we take less damage to our homes," GauthierDickey said.

Coxon issued a much more dire warning on X, writing that people building AI "earnestly believe that it could kill us all by the end of the decade."

His concerns come as AI companies grapple with examples of increasingly capable systems behaving in unexpected ways.

In July, Anthropic disclosed three incidents in which Claude models gained unauthorized access to real-world computer systems during cybersecurity evaluations after a third-party testing environment was mistakenly connected to the internet.

On Wednesday, Anthropic said it had identified a fourth incident while conducting a broader review.

"They were able to get internet access and get over to another company's systems," GauthierDickey said.

Still, GauthierDickey said there is a major gap between today's AI capabilities and a system capable of taking control from humans.

"Oh, I have a system that can reason, to now we can take over humanity — there's no path there that exists right now," GauthierDickey said.

He said defensive engineering, regulation and ethics will be crucial as the technology continues to evolve.

"We need people to study this. We need people who are studying computer science and software engineering and cybersecurity so these systems get built in a way we can test and validate whenever there's problems or making those decisions," GauthierDickey said.

GauthierDickey said the size and resources of the companies developing the most powerful AI systems make careful oversight even more important.

"Those are just much larger, much bigger. They have an incredible amount of resources to behave in the way that they do," GauthierDickey said.