A crime against statistics that is probably worse than the cyber attacks faced in County Durham

Posted by   Martijn Grooten on   Feb 21, 2018

Tomorrow, I will give a talk entitled "Don't know much about security" at the offices of ENISA, the EU's agency for network and information security.

Despite the title, the talk will not be about myself; rather, it will be about the fact that there are many aspects of security that we simply don't understand well – from the size of many botnets and attacks to the attribution of prominent threats.

What is worse is that we often claim to know more about things than we really do. A typical, if fairly extreme, example of this is a recent report by UK organisation Big Brother Watch that claims that local councils in the UK "have been subjected to over 98 million cyber attacks in the past five years".

That is a very large number, and anyone working in security will immediately understand that the authors of the report probably used a very weak definition of what counts as an attack.

Curious, I started reading the report (pdf), and I learned that the researchers had simply asked all local councils how many attacks they had faced in the past five years. Many reported zero or one attack, while a few reported numbers that were significantly larger.

For example, sysadmins at Somerset County Council said they received some 30,000 attacks a month, in Portsmouth the total for the period was 463,000 attacks, while in Essex they have recorded an incredibly precise number of 731,910 attacks in five years.

But all of that is trivial compared to what the sysadmins at Durham County Council have had to endure: they reported no fewer than 1.3 million attacks per month. This adds up to 78 million attacks in a five-year-period, and thus accounts for more than 80% of the attacks reportedly faced by UK local authorities.


Now, it is possible that Durham County Council is of particular interest to attackers. Far more likely, however, is that the question was simply interpreted differently by the different respondents and that, for County Durham, this number included spam emails or port scans. To say that it is a bit odd simply to add up all these numbers is an understatement.

Of course, these kinds of 'crimes against statistics' are hardly unique to security, and I am not too concerned about the fact that the number might be wrong: what matters is that the number is useless.

Local authorities, like all organisations, suffer cyber attacks, and one might wonder whether they are being attacked more than the average organisation. One might also wonder whether there is a particular trend over time, or whether certain local authorities (such as the larger or more prosperous ones) suffer more attacks than others.

However, none of these questions are addressed in the report, which merely produces a big number. Security reports often contain such big numbers, and even when the methodology used to produce them is sound, it is rare for these numbers to have any meaning at all. All they do is scare people into believing that the state of security is bad. We can do much better than that, and we ought to.

As for Durham County Council, there is a silver lining: despite the 78 million attacks the authority faced, its sysadmins also reported that they didn't experience a single cybersecurity incident. Their neighbours in Hartlepool, who were only attacked twice, did have an incident to report.




Latest posts:

VB2017 paper: The life story of an IPT - Inept Persistent Threat actor

At VB2017 in Madrid, Polish security researcher and journalist Adam Haertlé presented a paper about a very inept persistent threat. Today, we publish both the paper and the recording of Adam's presentation.

Five reasons to submit a VB2018 paper this weekend

The call for papers for VB2018 closes on 18 March, and while we've already received many great submissions, we still want more! Here are five reasons why you should submit a paper this weekend.

First partners of VB2018 announced

We are excited to announce the first six companies to partner with VB2018.

VB2018: looking for technical and non-technical talks

We like to pick good, solid technical talks for the VB conference programme, but good talks don't have to be technical and we welcome less technical submissions just as much.

Partner with VB2018 for extra visibility among industry peers

Partnering with the VB conference links your company to a successful and well-established event, demonstrates your commitment to moving the industry forward, allows you to meet potential clients, be visible to industry peers and build lasting…