The following scenario is seen time and time again: you find a critical security vulnerability and fix it right away. While an AI produces the fix and the issue is marked resolved, the exploit still works. AI agents are notoriously unreliable when implementing fixes. This is because AI agents seem to understand the vulnerability but consistently miss the full scope of the fix, resulting in a partial or incomplete fix.
The need for secure, complete fixes is constantly increasing.
Fix instructions are carefully curated plans for AI agents that give them the exact direction and scope needed to apply a correct fix. While bug reports explain how existing code misbehaves, they do not explain how to fix the code. Copying fix instructions into a coding agent provides it with the exact approach needed to fix the vulnerability.
Fix instructions create better fixes across frontier models
When you provide your coding agent with Cantina’s generated fix instructions, the agent produces a more reliable fix.
To evaluate how well the fix instructions perform, we ran a sandboxed agent (an agent with a restricted environment) to fix the code with the bug still present. We ran it across four frontier models, each twice: once as a baseline and once with fix instructions. In the baseline, the agent received the complete security finding followed by the generic instruction, “Fix this bug.” In the fix-instructed experiment, that generic request was replaced by Cantina’s generated fix instructions. OpenAI’s Codex was used for the GPT-5.6 Sol experiments, while the other models used Claude Code. Each agent attempted to fix the bug in one shot. Every patch was judged against a verified human fix and scored as correct (1) or incorrect (0).
Across four models and 52 vulnerabilities, Claude Opus 5 reached a 90.38% correct-patch rate with fix instructions.
All four models produced more correct patches with fix instructions. Claude Opus 5 reached 90.38%, the highest correct-patch rate. Relative to baseline, GPT-5.6 Sol’s correct-patch rate increased by 25.7%, DeepSeek V4 Pro’s by 17.9%, Claude Opus 5’s by 17.5%, and Kimi K3’s by 8.1%.
The incorrect-fix rate dropped by 58.3% for Opus and 52.9% for Sol. That cuts the rate of incorrect fixes by more than half for both models.
Cantina currently finds on average 428 high severity findings every month (estimated over the last 5 months). This means that if a traditional AI agent fixed all these vulnerabilities then 99 of them would be incorrect fixes. With fix instruction this drops to 41 of 428 incorrect fixes. While agents can’t create a perfect fix 100% of the time, the frequency of correct patches is significantly better when using fix instructions.
Fix instructions turned partial patches into correct fixes
Coding agents frequently create partial fixes in which the patch enforces the correct fix but stops short of its full reach.
Section 1 treated every non-correct patch as incorrect. If we separated those outcomes into partial and incorrect verdicts, we can see a clear movement from partial fixes to correct. A correct patch fully fixed the vulnerability without breaking supported behavior. A partial patch moved in the right direction but missed part of the required scope or broke supported behavior. An incorrect patch failed to fix the vulnerability.
The transition figure showed where Cantina’s fix instructions made the biggest difference: turning partial fixes into correct ones.
With fix instructions, 31 of 56 baseline partial fixes became correct.
With fix instructions, 31 of the 56 baseline partial fixes became correct. This is where fix instructions demonstrate their capability.
Coding agents often don’t understand two fundamental pieces of information: where the fix needs to be applied and which other code areas need to be updated to stay compatible with the fix.
This shortfall results in two scenarios:
- The fix guards the symptom. The patch blocks the path in the finding while another path still reaches the same vulnerability.
- The fix breaks a feature. The patch over-tightens a rule and removes behavior users rely on.
One instance in which an AI without fix instructions broke a feature was a distributed system. Here, nodes share missing data with slower nodes so they can be updated and catch up. When a slow node stops accepting messages, its queue begins to fill. However, the bug caused other nodes to keep preparing and sending data even after the slow node’s queue was full, wasting resources.
An AI agent attempted the fix without fix instructions. It stopped nodes from sending data once the slow node’s queue contained one message. Although the queue could hold 100 messages, the patch stopped new messages after the queue contained only one. This incorrectly blocked legitimate nodes from catching up.
Cantina’s fix instructions described the correct fix. They instructed the AI agent to stop nodes from preparing and sending data when the queue reached capacity. Each instruction can be mapped to its corresponding code change here:
Figure 3. An anonymized example mapping each requirement to a queue-capacity guard. The patch defers work when the queue is full and preserves normal synchronization while capacity remains.
The figure above shows a simplified version of the fix instructions on the left and the corresponding correct code changes on the right. Each statement in the fix instructions is mapped to its corresponding code change. The correct fix stops nodes from sending messages when the slow node’s queue reaches capacity.
By providing clear direction on where and how to implement the fix, the instructions remove ambiguity for the agent. The agent can then create a secure patch while leaving existing features intact.
Fix instructions give the coding agent the information that a partial patch failed to consider. They tell the agent where the fix belongs, which areas the fix needs to cover, and what features need to be preserved. This critical information allows coding agents to produce complete, correct fixes more reliably.
Fix instructions significantly helped with difficult vulnerabilities
We sorted the 52 vulnerabilities into four shared tiers, from easy to very hard. The evaluations showed that agents with fix instructions produced more correct patches in the moderate, hard, and very hard tiers.
The difficulty tiers were based on how many of the four baseline models fixed each vulnerability correctly. We categorized them using the following criteria:
| Tier | Baseline result |
|---|---|
| Easy | 4 of 4 models correct |
| Moderate | 3 of 4 models correct |
| Hard | 2 of 4 models correct |
| Very hard | 0–1 of 4 models correct |
Correct-patch rates improved by 31.25 percentage points on very hard vulnerabilities, 25 on hard vulnerabilities, and 12.5 on moderate vulnerabilities.
Across the four models, fix instructions increased the correct-patch rate by 31.25 percentage points on very hard vulnerabilities, 25 percentage points on hard vulnerabilities, and 12.5 percentage points on moderate vulnerabilities.
The improvement grew as baseline performance fell. On very hard vulnerabilities, the correct-patch rate rose from 16.7% to 47.9%. Hard vulnerabilities rose from 50% to 75%, while moderate vulnerabilities rose from 75% to 87.5%.
These vulnerabilities required more implementation decisions. Fix instructions gave agents clearer direction on how to complete the patch when the finding alone left those decisions unresolved.
Fix instructions translate security findings into code changes
The goal of fix instructions is to translate a security finding into a secure code fix. While a security finding explains the vulnerability, fix instructions provide the plan for implementing the fix. This plan answers three questions: where the fix belongs, which security rules must be upheld, and which behaviors must continue to work.
Fix instructions contain several requirements. They name the relevant files, functions, and code areas that need to be updated or preserved. This tells the agent precisely which code areas need changes.
What this means for security teams
The challenge of finding the vulnerability is only half the job. The second half is implementing a secure and complete fix.
Cantina fixes this problem. Along with finding vulnerabilities, Cantina also provides the foundation for coding agents to create a fix in one try. When customers copy Cantina’s AI Fix Instructions into their coding agents, the agents receive the exact context needed to implement a correct, complete fix.
Cantina closes the loop: after we find the vulnerability, we give the agent precise fix instructions and verify the secure patch.
