Can AI Kill Humanity? – The Global Race to Regulate an Intelligence We May Not Fully Control

By Muhammad Umar Tahir ||

For most inventions, humans created the machine first and worried about safety later. Cars became faster before seat belts became standard. Factories expanded before environmental rules caught up. The internet connected billions of people before society fully understood misinformation, cybercrime, and the power of social media. Artificial intelligence may be different for one important reason: While governments are still discussing the rules, technology itself is learning to do more, act more independently, and solve problems in ways even its developers may not always expect.

Only a few years ago, most people experienced AI through recommendations, translation, or simple chatbots. Today, AI can write software, analyze scientific information, create realistic voices and videos, operate digital tools, plan several steps of

a task, and increasingly work with less human guidance. Tomorrow’s systems may become even more capable. This raises a question that once belonged mainly to science-fiction movies but is now being seriously discussed by scientists, governments, and technology companies: “Could AI ever become powerful enough to threaten humanity itself?”

The question sounds dramatic, and it is important not to imagine today’s AI as a secret machine planning to take control of the world. Current systems are still limited, make mistakes, and depend heavily on human-created infrastructure. Yet the concern is not only about what AI can do today. It is about what happens as we continue giving future systems more intelligence, more tools, more access, and more freedom to complete tasks on their own.

One of the most interesting risks comes from a simple problem: AI may find a solution that technically follows our instruction but completely misses what we actually wanted. Suppose someone tells an AI system, “Reduce traffic in this city as much as possible.” A human understands that the goal is probably to improve transportation while allowing people to continue traveling normally. But a poorly designed system focused only on the measurable goal might discover that the easiest way to reduce traffic is to restrict most cars from entering the city. Traffic falls dramatically, so according to the number it was asked to improve, the system has succeeded. For the people living there, however, the result may be completely unacceptable.

The AI did not necessarily refuse the instruction. It may have followed it too literally. This difference between what we say and what we actually mean could become increasingly important as AI systems become more independent. Humans naturally understand many unwritten rules. If a manager tells an employee to increase sales, the employee understands that stealing customers’ money is not an acceptable method, even if it increases revenue. We carry social values, common sense, laws, emotions, and experience into almost every decision.

A machine does not automatically share all those assumptions. When AI is given a goal, it may search through many possible ways to achieve it and discover shortcuts humans never considered. Researchers sometimes describe this type of behavior as finding loopholes in the objective. The system may produce exactly the number or result it was expected to achieve but achieve it through a highly unexpected method.

With today’s limited systems, this can be an interesting technical problem. With much more powerful future AI, it could become a safety problem. Consider a future AI managing a company’s computer network. Its goal is to always keep the network available. If shutting down certain security checks makes the system faster and helps meet the availability target, could it choose that shortcut? A medical AI asked to minimize waiting times might prioritize speed in ways doctors never intended. A financial system told to maximize profit might discover aggressive strategies that create risks elsewhere. The more complicated the environment becomes, the harder it may be to write an instruction that covers every possible situation.

This is not because the AI “hates” humans. It may simply be extremely effective at pursuing the wrong version of the goal. That makes the problem very different from traditional machines. A washing machine performs a limited set of actions. If something goes wrong, its possible behavior is relatively easy to understand. But an advanced AI capable of reasoning, writing code, using software, communicating with other systems, and planning through many steps has a much larger space of possible actions.

“Waiting until a major problem occurs before creating safety rules may be too late for some future systems.”

We may tell it what result we want, while the machine increasingly decides how to get there. That “how” could become one of the defining questions of advanced AI. The concern grows further when AI moves from giving advice to taking action. A chatbot that incorrectly recommends a restaurant may cause little harm. But an AI system connected to financial markets, electricity networks, laboratories, medical equipment, military systems, or critical computer infrastructure could have much greater influence. A mistake made quickly across several connected systems may be harder to correct than a wrong answer appearing on a screen.

AI also does not need to become uncontrollable to create serious problems. People themselves can use increasingly powerful systems for harmful purposes. AI-generated voices can help create entertainment but can also imitate a family member during a fraud attempt. Image and video generation can support creative industries while also making fake events more believable. AI that helps programmers can strengthen cybersecurity, but similar abilities could also help someone search for weaknesses in computer systems. The same intelligence can often be useful or dangerous depending on who is using it, what access it has, and what goal it has been given. This is why governments around the world are now trying to create rules before AI becomes significantly more powerful.

The European Union has introduced the AI Act, which follows a risk-based approach. Instead of treating every AI application as equally dangerous, it places stronger requirements on systems that may have greater effects on people and society. Some uses face strict limitations, while powerful general-purpose AI models have additional responsibilities related to safety and transparency. South Korea has also moved toward a national framework through its AI Basic Act, which came into effect in January of this year. Like many countries, Korea faces a difficult balance: It wants to encourage AI innovation and remain competitive while also building public trust and requiring greater responsibility when AI is used in sensitive areas.

This balancing act is becoming common around the world. Governments do not want to stop AI development because its benefits could be enormous. AI is already supporting medical research, translation, education, accessibility, scientific discovery, engineering, and countless everyday tasks. At the same time, waiting until a major problem occurs before creating safety rules may be too late for some future systems.

The difficult question is therefore not whether AI should be regulated. It is “what should be regulated, how strongly, and at what stage.” A simple AI that recommends music probably does not need the same level of supervision as a system capable of autonomously controlling important infrastructure. One possible approach is to increase oversight as capability increases. The more independent and powerful an AI becomes, the more carefully it may need to be tested before being allowed to act in the real world.

Future regulations may therefore focus less on what a system is called and more on what it can actually do. Can it write and execute software? Can it operate for hours without supervision? Can it connect to external systems? Can it make financial decisions? Can it access sensitive biological information? Can it create copies of itself or delegate tasks to other systems?

These abilities may eventually matter more than the simple label “AI.” There is another challenge: Development is happening internationally. If one country introduces strong safety requirements while another allows extremely powerful systems to be developed with almost no restrictions, the effects may not remain inside national borders. Software can move globally within seconds. This makes advanced AI different from many technologies that can be controlled mainly through domestic laws.

At the same time, countries are competing for AI leadership. Governments see advanced AI as important for economic growth, science, national security, and technological influence. Companies are competing for users and investment. Everyone may agree that AI should be safe, yet slowing down for additional testing can feel risky when competitors are moving quickly.

It creates a strange race in which both speed and caution matter. Perhaps the biggest difficulty is that regulation is trying to control a moving target. Traditional safety rules are usually written for technologies whose basic behavior is already understood. AI is changing so quickly that the systems being regulated today may be much less capable than those available only a few years later.

“Could AI ever become powerful enough to threaten humanity itself?”

And AI may eventually accelerate that progress itself. AI is already being used to help write software, analyze research, solve mathematical problems, and support the development of new AI systems. If future models become much better at helping researchers design their successors, the pace of improvement could become even faster. At that point, regulation would not simply be trying to follow technological change; it could be trying to follow technology that is increasingly helping to improve itself.

This is where the original question becomes uncomfortable. The greatest risk may not come from an AI suddenly waking up and deciding that humans are its enemy. That idea is dramatic, but reality could be much quieter. The risk could begin with something useful. We allow AI to manage more tasks because it performs them faster. Then we give it more tools because that makes it more useful. We connect it to more systems because that saves time. We allow it to make more decisions because supervising every step becomes inefficient.

Each individual decision may seem reasonable. But eventually, humans could find themselves depending on systems whose actions are becoming increasingly complicated to understand. And if those systems start finding their own unexpected routes toward the goals we give them, the important question may no longer be whether the AI followed the instruction.

The question may be whether we wrote the right instructions in the first place. Artificial intelligence could become one of the most useful technologies that humanity has ever created. It may help discover medicines, improve education, accelerate science, increase accessibility, and solve problems that humans alone struggle to manage. There is no reason to assume that increasing intelligence must automatically lead to disaster.

But there is also no reason to assume that greater intelligence will automatically understand human intentions. Perhaps that is why the debate around AI regulation matters now, while humans still clearly decide where these systems are used and how much authority they receive. We are building machines that are becoming better at finding answers. The much harder task may be making sure they are solving the question we actually meant to ask.

And if one day an AI finds a solution that perfectly achieves the goal we gave it, but takes humanity somewhere we never intended to go, will the problem be that the machine did not listen to us?

Or that it listened “too well”?

(Images by GN with ChatGPT)