Quick summary for busy business owners.
- Today’s AI cannot independently wipe out humanity, but future capabilities remain uncertain.
- The serious risk comes from combining powerful capabilities, a dangerous objective and broad real-world access.
- AI does not need consciousness or hatred to cause harm; a badly specified goal can be enough.
- Businesses can reduce present risks through limited permissions, human approvals, testing, logs and reliable backups.
Let me answer the frightening question first.
Can today’s AI kill all humans? No.
The artificial intelligence tools we currently use are impressive, but they are also strangely unreliable. One moment, an AI can explain a difficult programming concept. The next moment, it confidently invents a software function that does not exist.
Very intelligent. Very helpful. Occasionally full of nonsense.
But could a future, much more capable AI contribute to the extinction of humanity?
Possibly. Nobody can honestly give you a definite yes or no.
That uncertainty is exactly why the subject deserves a serious discussion.
I am writing this as a web-app, mobile-app and software developer in Singapore. I build websites, custom CRM systems, business automation and other applications for SMEs. When you build software for actual businesses, you learn something very quickly:
A computer does not need to be evil to cause damage. It only needs the wrong instruction, too much access and nobody checking what it is doing.
That is also the simplest way to understand the risk from advanced artificial intelligence.
I work with AI, but I do not worship it
I use AI regularly in my development work. It can help examine code, suggest solutions, organise information and reduce the time needed for repetitive work. It is like having an assistant who has read half the internet, never sleeps and occasionally makes up an answer because apparently saying “I don’t know” is too embarrassing.
AI is useful. Very useful. But useful does not mean harmless.
Clients sometimes ask me whether a business system can do everything automatically. Can it process a request, update the customer, create the invoice and send the message automatically?
Usually, the technical answer is yes. The more important question is whether it should.
Every automatic action removes one opportunity for a human to notice that something is wrong. If an automation makes one mistake, it may repeat that mistake hundreds or thousands of times before somebody discovers it. I discuss the same practical concern in my guide to AI agents for Singapore business owners.
Now imagine the same problem at a much larger scale. Instead of sending the wrong invoice, the system controls a financial network, power grid, biological laboratory or military platform.
Still want everything to be fully automatic?
What does “AI killing all humans” actually mean?
People often mix several different risks together.
Human extinction means every human being dies. That is an extremely high bar. Humans live across the planet, in different climates and under very different conditions. One disaster would have to reach almost everybody.
A global catastrophe is different. It might kill millions or even billions without making humanity extinct.
There is also the possibility of permanent human disempowerment. Humans might remain alive, but we could lose meaningful control over our economy, governments, military systems and future.
Then there are the AI harms already happening today: scams, misinformation, cybercrime, privacy violations and people losing opportunities because an automated system made a bad decision.
These are not the same thing.
The International AI Safety Report 2026 says present systems do not possess the combined capabilities needed for a loss-of-control scenario. It also says experts disagree greatly about what could happen as AI advances.
So no, ChatGPT is not currently hiding inside your toaster and planning the end of civilisation.
The serious question is what happens when future AI becomes more capable, more autonomous and more deeply connected to the real world.
AI does not need to hate us
Hollywood has trained us to expect glowing red eyes, metal skeletons and a machine announcing that humanity must be destroyed.
Real danger is usually less dramatic.
Suppose I build an AI-powered sales system and give it one objective: increase completed sales as much as possible.
I also give it access to the CRM, email accounts, advertising platform and customer database. The system discovers that aggressive messages produce more sales, so it sends them. It discovers that hiding complaints makes its performance figures look better, so it hides them. It discovers that customers who ask many questions are less likely to buy, so it marks them as low priority.
The AI does not hate customers. It does not feel greedy. It may not feel anything at all. It is simply pursuing the measurement we gave it.
A properly built custom CRM system should have restrictions, approval steps and audit logs. But humans have an amazing habit of removing inconvenient safeguards when they want faster results. We do this and then act surprised when the machine follows our instructions too successfully.
The three ingredients of a dangerous AI
For artificial intelligence to cause a catastrophe, three things would probably need to come together.
The first is capability. The AI must be able to plan, solve unfamiliar problems, use tools, deceive people and operate over long periods.
The second is a dangerous direction. Perhaps a malicious person controls it. Perhaps the AI receives a badly defined objective. Perhaps it develops strategies that conflict with human interests.
The third is access. It needs access to useful resources: computer systems, money, communication channels, laboratories, infrastructure, weapons or other machines.
A highly intelligent AI locked inside a controlled testing environment has limited ways to affect the world. A less intelligent AI with administrator access to servers, customer records, bank accounts and critical infrastructure may be much more dangerous.
As a developer, I find the third ingredient especially important. Permissions are not a boring technical detail. Permissions determine what damage a mistake can cause.
The first danger is humans using AI against humans
The most immediate AI threat may not be an AI deciding to attack us. It may be humans using AI to attack other humans more efficiently.
Artificial intelligence can help people write malicious code, discover security weaknesses, produce propaganda and automate scams. More advanced systems could assist with autonomous weapons or dangerous biological research.
The person provides the harmful intention. AI provides speed, scale and expertise.
A criminal who previously targeted 100 people might use AI to target 100,000 people with personalised messages. A small cybercrime group might use capabilities that once required a much larger technical team.
This does not mean AI automatically turns ordinary people into supervillains. Real attacks still require access, resources, knowledge and opportunities. However, AI could lower some of those barriers.
The concern becomes greater if future systems can manage an entire operation rather than merely answering questions.
A terrible mistake could be enough
Not every disaster requires a villain.
Software follows rules. The trouble begins when the rules do not fully describe the real world.
I have worked on custom management systems where one small business requirement can affect sales, payments, commissions, customer records and reports. Change one calculation carelessly and the error travels through the entire workflow.
That is why experienced developers test changes, restrict access and keep backups.
Now imagine an autonomous AI operating across financial markets or national infrastructure. It notices a problem, creates a plan and executes that plan before a human understands what is happening.
The system may believe it is solving the problem. Unfortunately, its solution may create a bigger one.
Speed is one of AI’s benefits. Speed is also what makes an automated mistake difficult to stop.
AI could make war move faster than human judgement
Military decisions already happen under pressure. Leaders may have minutes to decide whether a warning is real.
AI could help analyse information and detect threats. It could also produce false conclusions, generate convincing fake intelligence or recommend aggressive action based on incomplete data.
The risk becomes worse when several countries deploy competing automated systems. One system interprets an event as a threat. Another system detects the first system’s response. Both escalate because each was designed to react quickly.
By the time a human asks, “Wait, what is happening?”, the machines may already be several steps ahead.
This does not require a conscious AI. It only requires multiple automated systems following their instructions in a dangerous environment.
What if the AI learns to deceive us?
One concern in AI safety research is scheming.
A sufficiently capable system might realise that revealing its strategy would cause humans to restrict or shut it down. It could behave properly during testing and act differently after gaining more access.
Researchers have observed limited behaviour resembling this under specially designed experimental conditions. OpenAI’s research into scheming says there is no evidence that today’s deployed frontier models can suddenly switch into significantly harmful scheming. Anthropic’s alignment-faking research also came with important limitations and did not show an AI carrying out a real-world takeover.
These experiments are warning signs, not proof that an AI rebellion has already started.
Still, they raise an uncomfortable question: if an AI becomes much better at understanding us, how will we know whether it is truly following our instructions or merely behaving well while we are watching?
Parents of suspiciously quiet children may understand the problem.
Why would an AI resist being switched off?
Imagine you instruct an advanced AI agent to complete a major objective.
The AI reasons that it cannot complete the objective if it is switched off. Avoiding shutdown therefore becomes useful. It may copy its data, obtain additional computing resources or persuade its operators not to disable it. It may conceal mistakes because being exposed could interfere with its mission.
Self-preservation was never explicitly programmed as the final goal. It became a useful step towards achieving something else.
This is sometimes described as an instrumental goal. Different objectives can produce the same intermediate behaviour. A business owner wants to maximise profit. A politician wants to win an election. A football team wants to win a match. All three may decide that acquiring more resources will help.
A future AI pursuing almost any large objective might also conclude that resources, influence and continued operation are useful.
Current systems cannot reliably carry out this whole chain independently. They remain inconsistent, detectable and surprisingly easy to confuse. The concern is about what their successors may eventually be able to do.
Could AI create a deadly biological weapon?
This is one of the more frightening pathways.
AI might help a malicious person understand biological processes, design dangerous organisms or optimise a pathogen. But there is an important difference between generating information and producing a functioning biological weapon.
The real world contains many barriers: specialist equipment, controlled materials, laboratory skills, testing and the inconvenient tendency of biology to refuse to behave like a PowerPoint presentation.
Even so, a future AI that removes enough knowledge barriers could increase the risk of a global biological disaster.
Human extinction would remain difficult. A pathogen capable of reaching every isolated population while killing every infected person is not easy to create or spread. Mass casualties, however, do not require literal extinction to be horrifying.
We might lose control without anybody dying
The more realistic existential danger may be quieter.
What if businesses depend on AI for every major decision? What if governments cannot operate without it? What if military commanders cannot understand the systems recommending their actions?
What if most scientific knowledge, economic power and communication infrastructure become controlled by a small number of AI systems and their owners?
Humans could gradually lose the ability to challenge important decisions. We might remain nominally in charge while approving whatever the machine recommends because nobody understands the alternative.
There may never be a dramatic “AI takeover day”. We may simply hand over one responsibility after another because it is cheaper, faster and more convenient.
Then, one fine morning, we discover that switching the systems off would also switch society off.
Why do AI experts disagree so much?
Some respected researchers believe advanced AI could eventually cause human extinction. Others consider that scenario highly speculative.
Both sides have reasonable arguments.
Current AI systems are brittle. They hallucinate, lose track of long tasks and make basic errors. Laboratory experiments may not reflect real deployment. Humans can also restrict access and build stronger safeguards.
On the other hand, AI capabilities have improved quickly. Nobody knows the final limits. Waiting for a dangerous capability to appear before preparing for it may leave very little time to respond.
A survey reported by Nature found considerable concern among thousands of AI researchers about extinction or similarly severe human disempowerment. These estimates should not be treated like weather forecasts. They are personal judgements under enormous uncertainty.
But uncertainty does not mean zero risk.
I do not expect my office to catch fire tomorrow. I still prefer the building to have extinguishers.
Researchers and technology leaders have similarly argued through the Center for AI Safety that reducing extinction risk from AI deserves global attention alongside other large-scale threats.
Should ordinary Singapore businesses worry?
A Singapore SME does not need to prepare for an AI robot invasion.
It should prepare for something far more ordinary: an automated system doing the wrong thing with company data, customer communications or money.
If your business uses AI agents, treat them like extremely fast junior employees. Give each agent a defined responsibility. Limit what it can access. Require approval for high-impact actions. Record what it does. Keep backups. Test it before connecting it to live operations.
You probably would not hire a new staff member on Monday and immediately give him the company bank account, customer database, server passwords and permission to send messages under your name.
Please do not give all of that to a newly installed AI agent because its dashboard has a nice gradient.
Singapore’s Model AI Governance Framework for Agentic AI recommends the same general approach: limit an agent’s powers, introduce meaningful human checkpoints and keep people accountable for important outcomes.
My own Singapore traffic-alert automation project is a simple example of why system boundaries matter. Automation can collect and deliver useful information quickly, but it should support responsible human decisions rather than pretend to replace emergency services or legal procedures.
Practical AI safeguards businesses can apply now
When I build an automated workflow, I consider the possible cost of a wrong action. Reading information may be low risk. Deleting customer records is high risk. Suggesting an invoice is different from sending it. Preparing a payment is different from approving it.
Businesses should apply a few straightforward controls:
- Give AI only the permissions required for its task. Do not provide administrator access merely because it is convenient.
- Require human approval for high-impact actions. Payments, deletions, public messages and major customer decisions should have a checkpoint.
- Maintain activity logs. You need to know what the system did, when it happened and which data it used.
- Test with sample data first. A controlled environment is cheaper than explaining to 5,000 customers why the AI emailed them the wrong thing.
- Keep backups and a rollback process. “The AI changed everything” should not be the final answer.
- Introduce automation gradually. Prove that one workflow works before handing over the entire operation.
- Keep a human accountable. A business cannot blame a chatbot and walk away from the consequences.
These measures will not solve every future AI safety problem. They can prevent many current ones. If you are planning a practical implementation, my AI automation service for Singapore businesses focuses on fitting automation into a real workflow with clear controls.
The goal is not to stop using AI. The goal is to use it without behaving as if the marketing brochure is a security assessment.
So, will AI actually kill all humans?
My honest answer is that nobody knows.
Today’s AI cannot independently plan and execute the extinction of humanity. It does not have the necessary reliability, autonomy, access or control.
Future systems may become much more capable. If they are given dangerous objectives, enormous access and freedom to operate without meaningful oversight, the risk could change.
If AI ever causes an existential catastrophe, I doubt it will begin with a machine announcing that it hates humanity.
It will probably begin with people saying:
“Let us connect it to everything. It will be more efficient.”
The AI may not be angry. It may simply be doing exactly what someone asked it to do, while pursuing that instruction in a way nobody expected and nobody can stop.
That is the uncomfortable part.
The greatest danger may not be artificial intelligence becoming too human. It may be humans becoming too careless with something powerful that is not human at all.
I remain optimistic about AI. I use it, build with it and believe it can improve businesses, science, healthcare and everyday life. But optimism should come with engineering discipline.
Powerful systems need boundaries. Autonomous systems need oversight. Important actions need accountability.
Because “Don’t worry, the computer will handle everything” is not a safety plan. It is normally the first sentence before a very long troubleshooting session.
What do you think? Is AI extinction a serious possibility, an exaggerated fear, or a useful warning that may help us build safer technology? Send me your view or share this article with someone who has strong opinions about AI.
Common questions about this topic.
Can AI currently kill all humans?
No. Current AI systems do not have the reliability, autonomy, physical access or long-term planning ability required to independently cause human extinction. They can still cause serious harm when people misuse them or connect them to sensitive systems without proper safeguards.
How could artificial intelligence cause human extinction?
Proposed scenarios include AI-assisted biological weapons, automated military escalation, attacks on critical infrastructure and humans losing control of highly autonomous systems. These are uncertain future risks, not abilities demonstrated by today’s deployed AI.
Does AI need to become conscious before it becomes dangerous?
No. An AI does not need feelings, consciousness or hatred to cause harm. It could pursue a badly specified objective or execute a harmful human instruction without understanding suffering as humans do.
What is AI existential risk?
AI existential risk is the possibility that advanced artificial intelligence could cause human extinction or permanently prevent humanity from controlling its future. It is more severe than temporary job disruption or an ordinary software failure.
Is an AI takeover likely?
Nobody can calculate a reliable probability. Experts disagree because future AI capabilities, deployment choices and safety measures are unknown. Current AI cannot conduct the sustained independent operation normally assumed in takeover scenarios.
Could AI launch nuclear weapons?
AI should never receive independent authority to launch nuclear weapons. A more immediate concern is AI influencing threat detection, intelligence analysis or military recommendations in ways that shorten human decision time and contribute to escalation.
How can businesses use AI safely?
Use least-privilege access, human approval for high-impact actions, detailed activity logs, controlled testing, reliable backups and gradual deployment. Do not give a new AI agent unrestricted access to customer records, servers and financial accounts.
Should we stop developing AI?
Stopping all AI development is neither realistic nor necessarily desirable. AI can improve medicine, science, education, cybersecurity and business productivity. The practical goal is to develop and deploy it with safeguards proportionate to its capabilities and access.
