Measuring Cybersecurity Awareness Training: Methods and Motivations

I am sure we are all familiar with the scenario: You want more funding for a project, funding that will contribute to a positive outcome for the company if invested correctly, but you face stiff resistance from the powers that be. It comes down to money, pure and simple. How much do they have to put in, and what exactly are they getting back, i.e., the return on investment (ROI). But how exactly do we measure ROI with something a bit more abstract like cybersecurity awareness training?

A typical ROI equation would be:

ROI = R/I, where R = Return (Benefit) and I = Investment (Cost).

So, you take your costs and benefits, and you have an answer to your question – it may cost X, but the return will be Y. Sounds simple enough, right? The problem with an ROI calculation with something like cybersecurity awareness training is that there are many variables; consequently, it is not uncommon for companies to add an element of probability to calculate risk percentage.¹ There has been more attention given to ROI regarding awareness training in recent years, with one notable report from Osterman Research producing some interesting statistical findings and reaching some exciting conclusions. These included:

• That security budgets are increasing.
• Security awareness training budgets are increasing at a faster rate.
• Training dramatically improves users' ability to recognise threats.
• The ROI for security awareness training is significant.

If we look in particular at the last point, the report found that, on average, smaller organisations (50-99 employees) can achieve an ROI of 69%, and larger organisations (1,000+) can achieve an ROI of 562%. Furthermore, the report concluded that "when security awareness training is implemented, the costs of disinfecting workstations and remediating malware/ransomware attacks goes down dramatically, resulting in a significant ROI for both small and large organisations."²

So, ROI is a good barometer for calculating investment into cybersecurity awareness training. Still, as we have mentioned, the intangible nature of measuring awareness training means there is room for interpretation and conjecture. It can be difficult to put precise values on "what if" scenarios, especially when other external factors come into play.

Another more straightforward method by which you can tangibly present actual data is by measuring your organisation's Phishing Susceptibility Rate (PSR), incorporating a practical element into the evaluation instead of relying on abstract "what if" scenarios. The statistics do not lie; the best way to determine employees' awareness in the face of a cyberattack is to phish them. Conducting a simulated phishing test gives you a baseline from which to work and the ability to see in real-time how susceptible your company is to a real email cyberattack. Simulating a phishing or spear phishing attack in a safe and controlled environment provides you with a solid reference point that can bolster your argument when looking for further investment for security awareness training. Not only that, but a phishing simulation also offers further insight into the success of your current security programme. After all, the best time to spot weaknesses within your company is before a successful attack has taken place and not afterwards.

To calculate your organisations current PSR, use the following formula:

Phishing Susceptibility Rate = Total Number of Failures Divided by Total Number of Performed Tests

For example, if your IT team sends 100 phishing emails to the users within your organisation and results in 60 click-throughs, the PSR would be 60 ÷ 100 = 60%. As you conduct further tests and your users become more familiar with spotting phishing and spear phishing emails, you would then see your PSR drop. You can then focus on repeat clickers with further training or restricting access to company data where required. Changing behaviours and improving awareness is, of course, best achieved with measurable results, and tests like this also seek to highlight how successful your current awareness programme is, compared to your expectations.

Source: How to Calculate ROI for Security Awareness Training | PhishLabs

Interestingly, within the last few years, we have seen a paradigm shift in how companies invest in their security infrastructure. This has meant moving away from throwing money directly at technology (whether anti-virus, firewalls, etc.) to investing more and more into security awareness training to mitigate attacks against the weakest entry point into most organisations, which remains the human being. A common consensus has been reached within the cybersecurity and information security industry that awareness training is the most cost-effective investment in defending companies from malicious attacks.

After all, having the best and most up-to-date technological framework will do very little to prevent an employee from clicking on that phishing email or opening that file attachment filled with malware. Let us not forget that around 90% of cyberattacks begin via email.

But even though security awareness training is growing in popularity as IT professionals and business leaders grasp its growing importance in the changing dynamic in attacks facing companies, we often find that many potential clients are either:

• Unaware of the benefits of conducting awareness training, or,
• Face a reluctance of investment from C-level executives, unwilling to divert users' time from daily tasks.

Whilst it may seem more straightforward to initiate multi-factor authentication or encourage the use of password managers to help boost internal company security measures, this does not get to the heart of the issue. It is certainly not a more cost-effective method to secure company data. The growing awareness industry offers a myriad of training material that can be completed quickly, with time and location independence being only two of the benefits they afford. It is not about making employees IT experts, rather about teaching your employees to be "good enough"³ and empower them to help protect your company's assets. With phishing attacks having surged 63% since COVID-19 began and employees clicking on three times more malicious emails than before⁴, it is not a matter of if you get phished, but when.

_{¹ Morrow, S. (2019, November 18). Is It Possible to Calculate an ROI for Security Awareness Training? The Defence Works. thedefenceworks.com/blog/is-it-possible-to-calculate-an-roi-for-security-awareness-training/
² The ROI of Security Awareness Training. (2019). Osterman Research. www.infosecinstitute.com/wp-content/uploads/2021/03/IQ-Whitepaper-The-ROI-of-Security-Awareness-Training.pdf
³ Stamps, M. (n.d.).& The ROI of Security Awareness Training. TECHGUARD Security. Retrieved 12 July 2021, from blog.techguard.com/the-roi-of-security-awareness-training
⁴ Lynch, K. (2021, April 14). ROI Analysis: Improving Resilience with Cybersecurity Awareness Training | Mimecast Blog. Mimecast. www.mimecast.com/blog/roi-analysis-improving-resilience-with-cybersecurity-awareness-training/}