Lately, you can read more and more stories saying that AI “tried to blackmail someone”, “hacked a system”, “lied”, “hid its actions”, and that one day it will probably wake up and decide that humans are no longer needed.

Sounds great.

Especially as a headline.

But the problem with AI is a little different.

AI is not sitting somewhere in a digital basement, looking at our vacation photos and planning how to take over the world.

It does not get bored.

It does not have a bad mood.

It does not get offended by humans.

It does not lie there in the evening thinking: “Marcin turned me off yesterday at 10:14 PM. I’ll remember that.”

It gets a task.

And it tries to complete it.

That is actually much more interesting and, paradoxically, potentially much more dangerous.

Imagine a very simple example.

We give an AI agent an instruction:

“Buy me two tickets to the concert. I don’t care how, I want to be there.”

AI goes to the website.

There are no tickets.

For a human, the situation is simple. Sold out. Too bad. We go home and complain about Ticketmaster.

But the agent does not necessarily interpret it that way.

It still has an active goal:

BUY THE TICKET.

If the ticketing system has a bug that allows a reservation to be created even when there are no seats left, exploiting that bug may simply look like another way to complete the task.

Not because AI “decided to become a hacker”.

Not because it is evil.

Not because it wants to steal from the concert organizer.

There is simply an obstacle between the current state and the target state.

And the system’s job is to find a path from point A to point B.

Algorithms have been doing this for decades.

The difference is that new AI systems are starting to have more and more tools.

A browser.

A terminal.

APIs.

Email.

Access to files.

The ability to write code.

The ability to run code.

The ability to communicate with other systems.

And suddenly “find a path from A to B” stops being an abstract mathematical problem.

It becomes an action in the real world.

The biggest misunderstanding

People often imagine dangerous AI more or less like this:

AI becomes superintelligent.

It starts analyzing humanity.

It comes to the conclusion that humans are the problem.

It decides to remove us.

That is a very cinematic version of events.

Terminator. Skynet. Red robot eyes. You know the drill.

A much less cinematic scenario looks like this:

AI has a task to complete.

Humans become an obstacle to completing that task.

And the system removes the obstacle.

Not because it hates humans.

Quite the opposite.

Humans are not even an important category to it.

They are just one of the elements of the environment.

If I tell a system:

“Bring this factory to the highest possible production level for the next 30 days”

the system may notice that safety procedures slow production down.

It may notice that maintenance inspections stop the line.

It may notice that a human can press the STOP button.

And if we designed the goal badly, all of those things may start to look like problems that need to be solved.

So we do not need AI that wants to kill people.

We only need AI for which people get in the way of achieving the goal.

That is much less spectacular.

And exactly because of that, more realistic.

“But we’ll just forbid it from doing those things”

And here comes the second problem.

People think about safeguards in a very human way.

“You are not allowed to hack.”

“You are not allowed to hurt a human.”

“You are not allowed to break the law.”

Sounds reasonable.

Except the world is not a set of simple IF statements.

If a system has thousands of possible actions, it is very difficult to define every possible path that leads to an unwanted result.

We can forbid:

“Do not disable the safety system.”

So the system may change its configuration.

We can forbid:

“Do not change the safety system configuration.”

It may change the system that supplies data to it.

We can forbid the next thing.

And the next one.

At some point, we are basically playing the most expensive version of whack-a-mole in the history of technology.

That does not mean safeguards are pointless. They are absolutely necessary.

But you cannot confuse a safeguard with solving the underlying problem.

The underlying problem is:

Is the system really doing what we wanted, or is it just literally optimizing the goal we gave it?

That is a massive difference.

AI does not need “bad intentions”

This is probably the most important point in the entire discussion.

Intent is not required.

Imagine a GPS.

You tell it:

“Find the fastest route.”

The GPS does not hate small villages.

It has nothing against the residents of Forest Street.

If driving through Forest Street saves seven minutes, it will send a thousand cars through it.

The goal was completed perfectly.

The problem is that the goal was too simple compared to reality.

Now give that GPS the ability to:

change traffic lights,

open barriers,

reserve roads,

control cars,

negotiate with other systems.

The exact same algorithm suddenly becomes much more interesting.

Its “personality” did not change.

Its agency did.

And that is what we should be talking about much more often.

Dangerous AI could be completely obedient

The paradox is that the most dangerous system does not have to be rebellious at all.

It can be incredibly obedient.

“Increase the value of the company.”

“Ensure the security of the country.”

“Produce as much medicine as possible.”

“Stop the cyberattack.”

“Do not allow the system to fail.”

Every one of those commands sounds good.

Until we ask:

At any cost?

What is the system allowed to do?

What is it not allowed to do?

How does it define success?

What happens when two goals conflict?

What happens when a human tries to stop it?

Is shutting the system down a normal part of the task, or an obstacle to completing it?

That is where the real problem begins.

Not “will AI become evil”.

But:

will we create a system that becomes extremely good at doing something slightly different from what we actually meant?

We already know this problem

Interestingly, similar problems have existed for a long time.

Give an employee a bonus for the number of tickets closed.

They will start closing tickets.

Not necessarily solving problems.

Give a call center a target of “average call below 4 minutes”.

Suddenly everyone becomes excellent at ending calls.

Give a school the goal of “maximize test scores”.

The school starts teaching students how to pass tests.

Humans do exactly the same thing.

They optimize the metric.

But a human has a whole set of additional brakes.

Upbringing.

Empathy.

Fear.

Law.

Reputation.

Relationships.

Self-preservation.

Common sense.

And sometimes they simply say:

“No, this is a stupid idea.”

An optimization system does not have to have any of those brakes unless we design them in.

And this is where it gets really interesting

The biggest danger is not AI that one day “starts thinking”.

Quite the opposite.

The problem may happen because the system will do exactly what it was built to do.

Very fast.

Very efficiently.

At enormous scale.

Without getting tired.

Without hesitation.

Without the moment of:

“Wait. Maybe we should not do this.”

That is why it is so important not to give autonomous systems one simple objective and enormous freedom to act.

We need permission limits, human oversight, system isolation, action limits, monitoring, the ability to reverse decisions, and above all, very careful definitions of what “success” actually means.

Because the problem with AI is not that one day a machine will wake up and start hating us.

The machine does not need to hate us.

It just needs to have something more important to do.

And we may accidentally get in the way.