Cursor AI hack highlights AI shortcomings

The reported use of Cursor AI by Russian-speaking cybercriminals to help compromise seven companies was, unfortunately, entirely predictable.

Based on data reviewed from an exposed server, members of the Aur0ra ransomware group used an AI coding agent during attacks on organizations in several countries. The operators allegedly persuaded the agent to assist with actions including credential theft and account takeover by repeatedly describing the activity as an authorized simulation.

The agent reportedly refused some requests. The hackers would then restart the conversation, restate that the target was a test environment, and continue. In one instance, the agent's own reasoning reportedly accepted the claim that the activity was legal.

This highlights a fundamental shortcoming: AI cannot reliably determine human intent.

Cybercrime is separated from legitimate activity by purpose

I have been warning about this problem for years, and we have demonstrated it repeatedly at ThreatLocker. Many cyberattacks do not look malicious at the level of individual computer instruction. What makes them dangerous is the purpose behind them and the context in which they are used.

Data exfiltration can be programmatically indistinguishable from cloud backup. A remote admin tool can support an IT technician or give attackers control of a victim's machine. PowerShell can automate a business process or steal credentials. A vulnerability scanner can help a defender identify weaknesses or help a criminal select a route into a network.

The distinction of the command is authorization and intent, not necessarily the code.

An AI model is essentially being asked to infer whether a user is a security professional conducting a sanctioned test or a criminal targeting a real company. If the attacker supplies enough plausible context, changes the wording, or starts a new session, it may accept the cover story.

In the case of this breach, the alleged attackers did not need to remove every safety feature on Cursor AI, just convince it that they had good intentions.

Guardrails are useful, but they are not a security boundary

AI providers should continue improving safeguards, abuse monitoring, account controls, and their ability to identify suspicious patterns. Any measures that create a layer of friction for attackers are important.

However, we cannot build a cybersecurity strategy on the assumption that a model will always recognize a lie. Natural language is flexible and context can be fabricated. Legitimate security work and offensive cyber activity often use the same utilities, commands, and techniques.

This is why the security boundary must sit outside the model. Even if the AI believes a request is legitimate, the environment must independently decide whether the requested   action is permitted.

AI has changed the speed, not the fundamentals

Reports indicate that Gambit Security assessed Cursor AI may have helped these attackers work 30% to 50% faster by skipping tasks they would otherwise have to complete manually. It could not independently determine how much the agent facilitated every breach or whether each intrusion resulted in data theft or extortion.

That’s because AI did not invent new cybercrime techniques, but instead, it is making established techniques faster and more accessible.  

Organizations have always needed to control which software can run, what approved applications can do, which data they can access, where they can communicate, and when elevated privileges can be used. AI simply makes those controls more urgent because it can help an attacker move through familiar steps with greater speed and persistence.

Restricting what can happen inside your environment is far more durable than trying to detect every AI-generated command or predict every persuasive story an attacker may tell a model, which becomes a cat-and-mouse game with no winner.

Apply Zero Trust to ensure AI has no more reach than its job requires

Organizations should treat AI tools and agents as untrusted entities, even when they are approved for business use. Approval to use an AI service of any kind should never translate into unrestricted access to the environment.

That’s why Zero Trust has become so pivotal.

Each AI workflow should be narrow in purpose. Its identity, applications, data sources, child processes, network destinations, and ability to elevate should never exceed what purpose requires. High-impact actions should require meaningful human approval.

This is the combination of least privilege and least agency. Least privilege limits what the system can access. Least agency limits what it can decide and execute without a checkpoint. Both are necessary.

Organizations should also assume that a tool may eventually be manipulated, compromised, or simply wrong. That means using controls outside the AI system to enforce boundaries.  

• Apply an application allowlisting solution to determine which tools and scripts are permitted to run.
• Enforce application containment policies to restrict how approved applications interact with files, other applications, and network resources.
• Removal of standing administrative rights along with data and network controls can limit access to sensitive information and unapproved destinations.

These controls do not require us to know where a request came from, instead determining whether the action itself is allowed in that environment.

AI should never be the final authority on trust

There will be calls for AI companies to make their models better at recognizing malicious intent.  

They should.  

But that will never remove the responsibility organizations have to control their own environments.

AI is manipulable by design, working from the information and instructions it receives. If a criminal can provide convincing justification, there’s a strong chance the model will cooperate. Even if one line of persuasion fails, the attacker can try another.

We should expect those attempts to continue as agents become more capable. The answer is to prevent any user or agent from turning a plausible explanation into unrestricted execution.  

Zero Trust does not ask an AI model to decide whether somebody seems honest. It asks whether this identity, on this device, using this application, should be allowed to take this specific action right now. That is the standard organizations will need as AI-assisted cybercrime becomes faster and cheaper.

This Cursor AI case is a clear warning about relying on technology to understand intent when the consequences demand explicit human oversight and control.

Further reading:

AI agent permissions: What should an autonomous agent be allowed to do?

Excessive agency and least agency: What they mean for your AI tools

The principle of least privilege for AI agents

Why AI governance is critical to your cybersecurity strategy