Measuring phishing simulations: the metrics that matter

What the training cohorts showed#

The 2025 Verizon Data Breach Investigations Report compared phishing campaigns associated with training in the previous 30 days against campaigns without such recent training. It analysed 36,325 recent-training campaigns and 68,492 other campaigns. The recent-training cohort had roughly 21% reporting versus a 5% base rate; the reported relative difference in click rate was much smaller, about 5%. These are observational campaign comparisons, not a randomised experiment that proves training caused each difference.

The result supports measuring reporting alongside clicking. It does not justify promising a fourfold improvement to every organisation or concluding that training cannot affect clicks. Audience, lure difficulty, delivery and other controls can differ between cohorts. Use your own comparable observations to decide what is helping, and describe those limits when reporting to management.

The 2026 edition adds a warning about where you run the simulations. Across the phishing simulations in its dataset, the median successful "click" rate in mobile-centric vectors – voice and text messaging – was 40% higher than via email. Pretexting, the synchronous cousin of phishing, reached 6% of all breaches as an initial access vector while phishing held at 16%. A programme that only ever tests the inbox may miss important risks in other channels; compare their relevance to your actual workforce.

The metrics, and what each one hides#

Every metric in a simulation platform answers one narrow question and quietly invites a wrong inference. Write the second column down before you present the first.

Simulation metrics and their blind spots

Metric	What it tells you	What it does not
Click rate	How many recipients opened the link in this specific lure	Whether anyone would be compromised; whether the lure was hard or easy
Submission rate	Who attempted a simulated submission; record the event without collecting real passwords or secrets	Whether MFA or conditional access would have stopped the login anyway
Report rate	How many people actively told you – a useful signal of reporting behaviour	Whether they reported because it looked odd or because a colleague warned them
Reporting speed	How fast the first and the bulk of reports arrive; can support a faster response	Anything about the people who never reported at all
Resilience ratio	Reporters divided by clickers, and whether that ratio is climbing	The absolute size of either group; a small ratio change can hide a large one
Coverage	Who has been trained, and how recently – the 30-day window matters here	Whether the training was understood or merely completed

A zero-click target cannot replace technical protection#

The DBIR is candid about the limit of any awareness programme: the click number "doesn’t quite go to zero … This might indicate that there is a ceiling to the effectiveness of education programs over a long period of time … maybe 1.5% of employees in the median case are the ones you can fool all of the time – since they’re still clicking after all these training sessions."

A click is not automatically a compromise: the destination, browser protections, identity controls and subsequent actions affect the outcome. Equally, a low average click rate does not remove the need for detection and response. NCSC recommends several layers of defence, including making suspicious messages easy to report and limiting the impact when a message is missed.

The resilience ratio#

The resilience ratio divides the number of reporters by the number of clickers in a campaign. Show both counts and the delivered population alongside it. If nobody clicked, the ratio cannot be calculated; show zero clickers rather than an infinite score. Lure difficulty can alter both counts, and an easy lure can flatter the ratio. Some people both click and report. A rising ratio is therefore a discussion prompt, not proof of reduced compromise risk.

Read trends, not snapshots#

A single campaign is noisy. Its numbers depend on the lure’s difficulty, the send timing, the audience and whether someone warned a group chat in the first ten minutes. Judge the programme on the trend across many campaigns with varied scenarios, and record enough metadata – theme, difficulty, sending domain, audience – that you can explain any outlier later. Change one variable at a time.

Teaching to the test can make results look better: easier or familiar lures may reduce clicks without demonstrating a broader improvement. Compare scenarios of similar difficulty and explain deliberate changes. A falling click rate can be meaningful when the underlying conditions are comparable. Keep all scenarios within the limits on lure design; technical subtlety is preferable to exploiting personal distress.

Metrics express what you value#

NCSC makes the sharpest point about measurement in this field: "metrics express an organisation’s values, and if you appear to value the absence of reports of problems, you incentivise people to keep quiet about issues. You should consider how you can formulate your security metrics to also include successes. For example, as well as measuring how many people clicked on a phishing email, focus on how many people reported it." The same guidance warns that simulations "erode trust between employees and security" and that "employees who are afraid for their jobs will not report mistakes". A dashboard built on click rate, shown to managers with names attached, is the mechanism by which that happens. The metric set is not a neutral reporting choice; it is the programme’s incentive design.

Reporting to the board#

Under NIS2 the board is both the audience and part of the subject. Article 21(2)(g) makes "basic cyber hygiene practices and cybersecurity training" one of the ten mandatory risk-management measures for essential and important entities. Article 21(2)(f) is a separate obligation – "policies and procedures to assess the effectiveness of cybersecurity risk-management measures" – and a recorded, comparable simulation series can contribute evidence about relevant training outcomes. It does not replace role-specific training or assessment of other measures. Article 20(2) then requires members of management bodies to follow training themselves and asks Member States to encourage the same for employees.

Commission Implementing Regulation (EU) 2024/2690 shows what an assessor expects to see. It binds a defined list of digital entity types – DNS providers, TLD registries, cloud, data centre, CDN, managed service and managed security service providers, online marketplaces, search engines, social networking platforms and trust service providers – and can be a reference for other organisations after checking their applicable requirements. Its Annex section 7 implements Article 21(2)(f); section 8 implements 21(2)(g) and splits into 8.1 awareness raising and basic cyber hygiene practices and 8.2 security training. Map relevant results to those headings alongside policies, training records and follow-up actions. A metric series alone does not establish compliance.

Mention the consequence once and move on: Article 34(4) and (5) require national maximum fines of at least EUR 10 000 000 or 2% of total worldwide annual turnover for essential entities, and EUR 7 000 000 or 1.4% for important entities, whichever is higher. These are minimum levels for national maxima, not fixed EU-wide caps. That is why the board asks; the metrics are how you answer.

What to put in the report#

• Report rate and reporting speed first, as the headline, with trends across enough comparable campaigns to support a useful interpretation.
• The resilience ratio and its direction, with the lure difficulty for each campaign shown alongside it.
• Click and submission rates as context, never as the score, and never broken down by named individual.
• Coverage: who has been trained and how recently, given the DBIR’s 30-day recency finding.
• What changed operationally as a result – a control tightened, a reporting queue resourced, a process fixed.
• Which NIS2 obligation each figure helps assess: Article 21(2)(g) for the training itself, Article 21(2)(f) for the assessment of its effectiveness.