VBSpam Email Security Comparative Review - December 2021

Ionuţ Răileanu & Adrian Luca

Virus Bulletin

Copyright © 2021 Virus Bulletin


Introduction

In this test – which forms part of Virus Bulletin’s continuously running security product test suite – nine full email security solutions, one custom configured solution1, one open-source solution and one blocklist were assembled on the test bench to measure their performance against various streams of wanted, unwanted and malicious emails. In this round of testing we welcome two new participants to the public VBSpam test: Check Point and Zoho Mail.

With the email threat landscape constantly changing, and with the aim of diversifying the Malware corpus, this test includes emails from MXMailData, an Australia-based company. Of all the samples received from MXMailData, for the test we selected and sent to the products only those containing attachments that had a high possibility of being malicious.

For some additional background to this report, the table and map below show the geographical distribution (based on sender IP address) of the spam emails seen in the test. (Note: these statistics are relevant only to the spam samples we received during the test period.)

# Sender's IP country Percentage of spam
1 United States 12.18%
2 India 7.80%
3 China 6.82%
4 Vietnam 6.50%
5 Brazil 5.29%
6 Japan 5.22%
7 Indonesia 3.91%
8 Pakistan 3.03%
9 Russian Federation 2.71%
10 Colombia 1.84%

Top 10 countries from which spam was sent.

map-dec21.pngGeographical distribution of spam based on sender IP address.

 

Malware and phishing

The majority of the samples from these two categories were successfully blocked by the security solutions we tested. However, there were some samples that stood out and, at least initially, managed to evade the filters. In the following sections we describe some of them.

 

Bazar Loader2

Active during 9 and 10 November, and again briefly on 15 November, this malspam campaign was the most challenging for the solutions participating in the test to block. The format is something we have seen in the past: a short text and a password-protected attachment. The ‘.doc’ file extracted from the attached archive contains a short text written in white that is saved on the computer to an ‘.hta’ file. Through mshta.exe, the ‘.hta’ file runs and connects to the attacker’s server (which, in one of the samples, was shoulderelliottd[.]com).

Attachment SHA256:

7a9b3945f42e8e3ee3272501bfa9ebf842140ca740f0698c56e0d3f90445eb4a

0c00919c17f1fd52d965b933e46d80e2d25ae4d162920fda8a9772763e07224b

70cfd7872c56ccb75008b2db299179fa3ee3293ca910b5582966f97db759b45b

Name:

Info.zip

request.zip

bazar-loader.png Email from Bazar Loader campaign.

Dridex

This was a large malspam campaign (making up 11% of the Malware corpus) that we observed on 10 November and again on 15 November. Even though the majority of the emails were blocked by the tested solutions, there were a few that managed to bypass their filters. The attached ‘.xll’ files were reported3 to be linked to Dridex.

dridex.pngEmail with attachment linked to Dridex.

 

German phishing campaign4

This was the largest phishing campaign we saw during the test period. It targeted German speakers and contained a URL leading to a PHP page that redirected to the actual malicious link. The only such redirect that we managed to catch was hxxps://spkfinanzverifikation[.]com/OIYZ6MP95A. At the time of writing this report, the URLs were inactive.

german-phish.pngGerman phishing email.

 

Netflix phishing

We highlight these phishing emails because they were the ones that evaded most of the security solutions filters. They were spotted on 19 November, being sent from the same IP address (217[.]66[.]226[.]61) and contained the URL hxxps://www[.]pun-entertainment[.]com/app/login/nz/.

netflix-phis.pngNetflix phishing email.

 

Results

All of the tested solutions managed to block more than 98% of the spam samples, with Cleanmail and Libraesva being the only solutions that blocked 100% of the malware samples. Libraesva also had the best phishing catch rate.

Of the participating full solutions, five achieved a VBSpam award – Axway, Check Point, Cleanmail, Libraesva and N-able Mail Assure – as did the custom-configured solution Spamhaus Data Query Service (DQS) + SpamAssassin. A further three achieved a VBSpam+ award: Bitdefender, Fortinet and Net at Work.

 

Axway MailGate 5.6

SC rate: 99.88%
FP rate: 0.06%
Final score: 99.56
Malware catch rate: 96.65%
Phishing catch rate: 99.47%
Project Honey Pot SC rate: 99.75%
Abusix SC rate: 99.91%
MXMailData SC rate: 95.99%
Newsletters FP rate: 0.9%

Speed:   10% GREEN 50% GREEN 95% GREEN 98% GREEN
vbspam-verified-1221.jpg

Axway has performed well in previous tests and continues to produce spam catch rates that exceed 99.5%. With only 11 samples missed from the phishing corpus, Axway achieved the third highest phishing catch rate in this test. While two false positives stood in the way of a VBSpam+ award, Axway earns a VBSpam award with ease.

 

Bitdefender Security for Mail Servers 3.1.7

SC rate: 99.97%
FP rate: 0.00%
Final score: 99.97
Malware catch rate: 99.38%
Phishing catch rate: 99.56%
Project Honey Pot SC rate: 99.98%
Abusix SC rate: 99.97%
MXMailData SC rate: 99.67%
Newsletters FP rate: 0.0%

Speed:   10% GREEN 50% GREEN 95% GREEN 98% GREEN
vbspam-plus-1221.jpg

Bitdefender has participated in every VBSpam test since they began in May 2009 and has performed well in every one of them – and it’s the only product that has earned a VBSpam+ award in each of the four VBSpam tests in 2021. With impressive catch rates of more than 99% not only in the spam corpus but also in the malware and phishing samples, combined with a lack of false positives on both ham and newsletters, Bitdefender proves itself once again to a dependable solution. With the highest final score in this test, Bitdefender’s developers earn another VBSpam+ award to add to their collection.

 

Check Point - Harmony Email & Office

SC rate: 99.25%
FP rate: 0.21%
Final score: 98.19
Malware catch rate: 98.40%
Phishing catch rate: 98.74%
Project Honey Pot SC rate: 98.66%
Abusix SC rate: 99.30%
MXMailData SC rate: 98.66%
Newsletters FP rate: 0.9%

Speed:   10% GREEN 50% GREEN 95% speed-colour-blobs-YELLOW.jpg 98% speed-colour-blobs-YELLOW.jpg
vbspam-verified-1221.jpg

 One of the new entries in this test, Check Point makes a great impression on its debut. With a steady and well adjusted performance, blocking more than 99% of the spam samples Check Point earns a VBSpam award.

 

Cleanmail Domain Gateway

SC rate: 99.95%
FP rate: 0.15%
Final score: 99.04
Malware catch rate: 100.00%
Phishing catch rate: 99.27%
Project Honey Pot SC rate: 99.85%
Abusix SC rate: 99.95%
MXMailData SC rate: 100.00%
Newsletters FP rate: 5.7%

Speed:   10% GREEN 50% GREEN 95% GREEN 98% GREEN
vbspam-verified-1221.jpg

Only two of the participating solutions managed to block all the malware samples in this test, Cleanmail being one of them. To add to this impressive feat, the product achieved a higher than 99% phishing catch rate. In this test Cleanmail earns another VBSpam award, its fourth in 2021.

 

Fortinet FortiMail

SC rate: 99.90%
FP rate: 0.00%
Final score: 99.90
Malware catch rate: 99.12%
Phishing catch rate: 95.49%
Project Honey Pot SC rate: 99.80%
Abusix SC rate: 99.91%
MXMailData SC rate: 99.33%
Newsletters FP rate: 0.0%

Speed:   10% GREEN 50% GREEN 95% GREEN 98% GREEN
vbspam-plus-1221.jpg

Fortinet’s FortiMail appliance has long performed well in our tests. Its final scores have exceeded 99.60 in each of this year’s tests. On this occasion it was one of only two solutions with no false positives in either the ham or the newsletter corpus, and to add to this impressive performance it achieved a 99.90% spam catch rate. These results earn the product another VBSpam+ award, its third in a row this year.

 

Libraesva ESG v.4.7

SC rate: 99.88%
FP rate: 0.03%
Final score: 99.70
Malware catch rate: 100.00%
Phishing catch rate: 99.71%
Project Honey Pot SC rate: 99.83%
Abusix SC rate: 99.88%
MXMailData SC rate: 100.00%
Newsletters FP rate: 0.9%

Speed:   10% GREEN 50% GREEN 95% GREEN 98% speed-colour-blobs-YELLOW.jpg
vbspam-verified-1221.jpg

Not only was Libraesva one of the two solutions that blocked 100% of the malware samples in this test, but it also achieved the highest phishing catch rate, 99.71%. The Italian solution has had a good year in the VBSpam test, having been awarded a VBSpam+ in two of the four quarterly tests. In this, the last test of the year, Libraesva is awarded a VBSpam certification.

 

N-able Mail Assure

SC rate: 99.90%
FP rate: 0.03%
Final score: 99.72
Malware catch rate: 99.95%
Phishing catch rate: 98.74%
Project Honey Pot SC rate: 99.79%
Abusix SC rate: 99.91%
MXMailData SC rate: 99.93%
Newsletters FP rate: 0.9%

Speed:   10% GREEN 50% GREEN 95% GREEN 98% GREEN
vbspam-verified-1221.jpg

This is the second time N-able Mail Assure has participated in the VBSpam test, having earned a VBSpam+ award on its debut. On this occasion the product’s performance on the malware corpus stands out, with only one sample missed. With a spam catch rate of 99.90% and a final score of 99.72, N-able Mail Assure is awarded a VBSpam certification in this test.

 

Net at Work NoSpamProxy

SC rate: 99.79%
FP rate: 0.00%
Final score: 99.76
Malware catch rate: 99.90%
Phishing catch rate: 98.06%
Project Honey Pot SC rate: 99.07%
Abusix SC rate: 99.84%
MXMailData SC rate: 99.93%
Newsletters FP rate: 0.9%

Speed:   10% GREEN 50% GREEN 95% GREEN 98% GREEN
vbspam-plus-1221.jpg

Net at Work’s developers have reason to be proud of their product’s performance in this test, given the excellent spam, phishing and malware catch rates, combined with zero false positives in the ham corpus. The product earns a VBSpam+ award on this occasion.

 

Rspamd

SC rate: 98.43%
FP rate: 0.74%
Final score: 94.68
Malware catch rate: 80.69%
Phishing catch rate: 90.01%
Project Honey Pot SC rate: 92.48%
Abusix SC rate: 99.00%
MXMailData SC rate: 80.11%
Newsletters FP rate: 2.8%

Speed:   10% GREEN 50% GREEN 95% GREEN 98% GREEN

Rspamd, a free and open-source solution, joined the VBSpam test in March 2021, and although the product has yet to achieve VBSpam certification, its performance is improving with each test. Worthy of note on this occasion are the 98.43% spam catch rate and the 90.01% phishing catch rate.

 

Spamhaus Data Query Service (DQS) + SpamAssassin

SC rate: 99.56%
FP rate: 0.00%
Final score: 99.45
Malware catch rate: 98.04%
Phishing catch rate: 98.25%
Project Honey Pot SC rate: 99.11%
Abusix SC rate: 99.61%
MXMailData SC rate: 98.40%
Newsletters FP rate: 3.8%

Speed:   10% GREEN 50% GREEN 95% GREEN 98% GREEN
vbspam-verified-1221.jpg

Spamhaus Data Query Service (DQS) + SpamAssassin is a custom configured solution that integrates the Spamhaus DQS DNSBL service with the free open-source SpamAssassin solution. In this test the product impressed with a 99.56% spam catch rate and no false positives in the ham corpus. It just misses out on a VBSpam+ award due to a small number of newsletter false positives, but it earns VBSpam certification nevertheless.

 

Zoho Mail

SC rate: 99.40%
FP rate: 0.24%
Final score: 97.88
Malware catch rate: 98.46%
Phishing catch rate: 98.84%
Project Honey Pot SC rate: 98.24%
Abusix SC rate: 99.50%
MXMailData SC rate: 98.66%
Newsletters FP rate: 11.3%

Speed:   10% GREEN 50% GREEN 95% speed-colour-blobs-YELLOW.jpg 98% speed-colour-blobs-ORANGE.jpg

Zoho Mail is the second new entry in this test. Achieving an impressive 99.40% spam catch rate, and with phishing and malware catch rates higher than 98%, Zoho Mail was on track to earn VBSpam certification. However, a number of misclassified ham and newsletter samples lowered the product’s final score to 97.88, just beneath the threshold for certification on this occasion.

 

Abusix Mail Intelligence

SC rate: 99.69%
FP rate: 0.00%
Final score: 99.66
Malware catch rate: 87.08%
Phishing catch rate: 98.79%
Project Honey Pot SC rate: 98.35%
Abusix SC rate: 99.86%
MXMailData SC rate: 85.38%
Newsletters FP rate: 0.9%

Abusix Mail Intelligence is a set of blocklists that is tested as a partial solution because it has access only to parts of the emails (IP addresses, domains, URLs), which are queried as to their DNS zones. With this setup, the very high (99.69%) spam catch rate and the lack of false positives in the ham corpus is very impressive. The reason there is no award for this product is that the criteria for VBSpam certification include ham delivery speed values, which are calculated only for full solutions.

 

Results tables

  True negatives False positives FP rate False negatives True positives SC rate Final score VBSpam
Axway 3383 2 0.06% 429.4 353224 99.88% 99.56 vbantispam-pass.gif
Bitdefender 3385 0 0.00% 95 353558.4 99.97% 99.97 vbantispam-plus.gif
Check Point 3378  7 0.21%  2653.8  350970.6  99.25%  98.19 vbantispam-pass.gif
Cleanmail Domain Gateway  3380 5 0.15%  189 353464.4  99.95% 99.04 vbantispam-pass.gif
FortiMail 3385  0 0.00% 352.8  353300.6  99.90%  99.90 vbantispam-plus.gif
Libraesva  3384 1 0.03%  440 353213.4  99.88%  99.70 vbantispam-pass.gif
N-able Mail Assure  3384 1 0.03% 350 353303.4  99.90%  99.72 vbantispam-pass.gif
NoSpamProxy  3385 0 0.00% 756 352897.4  99.79%  99.76 vbantispam-plus.gif
Rspamd  3360 25 0.74%  5540.6  348112.8  98.43%  94.68  
Spamhaus Data Query Service (DQS) + SpamAssassin  3385 0 0.00% 1541.8  352111.6  99.56%  99.45 vbantispam-pass.gif
Zoho Mail  3377 8 0.24%  2109.8  351543.6  99.40%  97.88   
Abusix Mail Intelligence*  3385 0 0.00% 1111.2  352542.2  99.69% 99.66  

*This product is a partial solution and its performance should not be compared with that of other products.
Spamhaus Data Query Service (DQS) + SpamAssassin is a fully configured solution that integrates Spamhaus DQS on top of SpamAssassin. Spamhaus DQS is not a stand-alone solution but rather a DNSBL service that can be added to MTAs and email security solutions such as SpamAssasssin. The test set up reflects the real-life performance expected from this combined production deployment, not as individual product elements.
(Please refer to the text for full product names and details.)

 

  Newsletters Malware Phishing Project Honey Pot Abusix MXMailData STDev
  False positives FP rate False negatives SC rate False negatives SC rate False negatives SC rate False negatives SC rate False negatives SC rate
Axway 1 0.9% 65 96.65% 11 99.47% 67.2 99.75% 302.2 99.91% 5 99.67% 0.22
Bitdefender 0 0.0% 12 99.38% 9 99.56% 5 99.98% 85 99.97% 298 80.11% 0.08
Check Point 1 0.9% 31 98.40% 26 98.74% 355.8 98.66% 2278 99.30% 0 100.00% 2.5
Cleanmail Domain Gateway 6 5.7% 0 100.00% 15 99.27% 38.8 99.85% 150.2 99.95% 60 95.99% 0.12
FortiMail 0 0.0% 17 99.12% 93 95.49% 54.2 99.80% 288.6 99.91% 10 99.33% 0.15
Libraesva 1 0.9% 0 100.00% 6 99.71% 45.2 99.83% 394.8 99.88% 1 99.93% 0.24
N-able Mail Assure 1 0.9% 1 99.95% 26 98.74% 57 99.79% 292 99.91% 24 98.40% 0.19
NoSpamProxy 1 0.9% 2 99.90% 40 98.06% 247.2 99.07% 507.8 99.84% 0 100.00% 0.28
Rspamd 3 2.8% 375 80.69% 206 90.01% 2003.4 92.48% 3239.2 99.00% 20 98.66% 1.2
Spamhaus Data Query Service (DQS) + SpamAssassin 4 3.8% 38 98.04% 36 98.25% 236.8 99.11% 1281 99.61% 1 99.93% 0.47
Zoho Mail 12 11.3% 30 98.46% 24 98.84% 469.2 98.24% 1620.6 99.50% 20 98.66% 0.53
Abusix Mail Intelligence* 1 0.9% 251 87.08% 25 98.79% 440 98.35% 452.2 99.86% 219 85.38% 0.53

*This product is a partial solution and its performance should not be compared with that of other products. None of the queries to the IP blocklist included any information on the attachments; hence its performance on the malware corpus is added purely for information.
Spamhaus Data Query Service (DQS) + SpamAssassin is a fully configured solution that integrates Spamhaus DQS on top of SpamAssassin. Spamhaus DQS is not a stand-alone solution but rather a DNSBL service that can be added to MTAs and email security solutions such as SpamAssasssin. The test set up reflects the real-life performance expected from this combined production deployment, not as individual product elements.
The standard deviation of a product is calculated using the set of its hourly spam catch rates.
(Please refer to the text for full product names and details.)

 

  Speed
  10% 50% 95% 98%
Axway speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg
Bitdefender speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg
Check Point speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-YELLOW.jpg speed-colour-blobs-YELLOW.jpg
Cleanmail Domain Gateway speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg
FortiMail speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg
Libraesva speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-YELLOW.jpg
N-able Mail Assure speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg
NoSpamProxy speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg
Rspamd speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg
Spamhaus Data Query Service (DQS) + SpamAssassin speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg
Zoho Mail speed-colour-blobs-GREEN.jpg speed-colour-blobs-GREEN.jpg speed-colour-blobs-YELLOW.jpg speed-colour-blobs-ORANGE.jpg

Spamhaus Data Query Service (DQS) + SpamAssassin is a fully configured solution that integrates Spamhaus DQS on top of SpamAssassin. Spamhaus DQS is not a stand-alone solution but rather a DNSBL service that can be added to MTAs and email security solutions such as SpamAssasssin. The test set up reflects the real-life performance expected from this combined production deployment, not as individual product elements.

 

Products ranked by final score
Bitdefender 99.97
FortiMail 99.90
NoSpamProxy 99.76
N-able Mail Assure 99.72
Libraesva 99.70
Abusix Mail Intelligence* 99.66
Axway 99.56
Spamhaus Data Query Service (DQS) + SpamAssassin 99.45
Cleanmail Domain Gateway 99.04
Check Point 98.19
Zoho Mail 97.88
Rspamd 94.65

*This product is a partial solution and its performance should not be compared with that of other products. None of the queries to the IP blocklist included any information on the attachments; hence its performance on the malware corpus is added purely for information.
Spamhaus Data Query Service (DQS) + SpamAssassin is a fully configured solution that integrates Spamhaus DQS on top of SpamAssassin. Spamhaus DQS is not a stand-alone solution but rather a DNSBL service that can be added to MTAs and email security solutions such as SpamAssasssin. The test set up reflects the real-life performance expected from this combined production deployment, not as individual product elements.
(Please refer to the text for full product names and details.)

 

Hosted solutions Anti-malware IPv6 DKIM SPF DMARC Multiple MX-records Multiple locations
Check Point Threat Emulation    √ √   
 
 
Cleanmail Domain Gateway Cleanmail    
N-able Mail Assure N-able Mail Assure    
NoSpamProxy Cyren & NoSpamProxy      
Zoho Mail Zoho    √ √ 

(Please refer to the text for full product names and details.)

 

Local solutions Anti-malware IPv6 DKIM SPF DMARC Interface
CLI GUI Web GUI API
Axway Kaspersky, McAfee        
Bitdefender Bitdefender        
FortiMail Fortinet  
Libraesva ClamAV; others optional        
Rspamd None              
Spamhaus Data Query Service (DQS) + SpamAssassin Optional        

Spamhaus Data Query Service (DQS) + SpamAssassin is a fully configured solution that integrates Spamhaus DQS on top of SpamAssassin. Spamhaus DQS is not a stand-alone solution but rather a DNSBL service that can be added to MTAs and email security solutions such as SpamAssasssin. The test set up reflects the real-life performance expected from this combined production deployment, not as individual product elements.

(Please refer to the text for full product names and details.)

 

VBSpam-quadrant-Dec2021.jpg(Please refer to the text for full product names and details.)

 

 

Appendix: set-up, methodology and email corpora

The full VBSpam test methodology can be found at https://www.virusbulletin.com/testing/vbspam/vbspam-methodology/vbspam-methodology-ver20.

The test ran for 16 days, from 12am on 6 November to 12am on 22 November 2021 (GMT).

The test corpus consisted of 357,210 emails. 353,719 of these were spam, 26,643 of which were provided by Project Honey Pot, 325,578 were provided by Abusix with the remaining 1,498 spam emails provided by MXMailData. There were 3,385 legitimate emails (‘ham’) and 106 newsletters, a category that includes various kinds of commercial and non-commercial opt-in mailings.

82 emails in the spam corpus were considered ‘unwanted’ (see the June 2018 report5) and were included with a weight of 0.2; this explains the non-integer numbers in some of the tables.

Moreover, 1,942 emails from the spam corpus were found to contain a malicious attachment while 2,063 contained a link to a phishing or malware site; though we report separate performance metrics on these corpora, it should be noted that these emails were also counted as part of the spam corpus.

Emails were sent to the products in real time and in parallel. Though products received the email from a fixed IP address, all products had been set up to read the original sender’s IP address as well as the EHLO/HELO domain sent during the SMTP transaction, either from the email headers or through an optional XCLIENT SMTP command6.

For those products running in our lab, we all ran them as virtual machines on a VMware ESXi cluster. As different products have different hardware requirements – not to mention those running on their own hardware, or those running in the cloud – there is little point comparing the memory, processing power or hardware the products were provided with; we followed the developers’ requirements and note that the amount of email we receive is representative of that received by a small organization.

Although we stress that different customers have different needs and priorities, and thus different preferences when it comes to the ideal ratio of false positive to false negatives, we created a one-dimensional ‘final score’ to compare products. This is defined as the spam catch (SC) rate minus five times the weighted false positive (WFP) rate. The WFP rate is defined as the false positive rate of the ham and newsletter corpora taken together, with emails from the latter corpus having a weight of 0.2:

WFP rate = (#false positives + 0.2 * min(#newsletter false positives , 0.2 * #newsletters)) / (#ham + 0.2 * #newsletters)

while in the spam catch rate (SC), emails considered ‘unwanted’ (see above) are included with a weight of 0.2.

The final score is then defined as:

Final score = SC - (5 x WFP)

In addition, for each product, we measure how long it takes to deliver emails from the ham corpus (excluding false positives) and, after ordering these emails by this time, we colour-code the emails at the 10th, 50th, 95th and 98th percentiles:

speed-colour-blobs-GREEN.jpg (green) = up to 30 seconds
YELLOW (yellow) = 30 seconds to two minutes
speed-colour-blobs-ORANGE.jpg (orange) = two to ten minutes
speed-colour-blobs-RED.jpg (red) = more than ten minutes

 

Products earn VBSpam certification if the value of the final score is at least 98 and the ‘delivery speed colours’ at 10 and 50 per cent are green or yellow and that at 95 per cent is green, yellow or orange.

Meanwhile, products that combine a spam catch rate of 99.5% or higher with a lack of false positives, no more than 2.5% false positives among the newsletters and ‘delivery speed colours’ of green at 10 and 50 per cent and green or yellow at 95 and 98 per cent earn a VBSpam+ award.

 

Footnotes

Spamhaus Data Query Service (DQS) + SpamAssassin is a custom solution built on top of the SpamAssassin open-source anti-spam platform.

2 https://bazaar.abuse.ch/sample/79cf54cb43eb9c510c7ce3271962f8d5befd28628688da4f625b01d9b52d5030/.

3 https://bazaar.abuse.ch/sample/2ad9563f018dcfe967225314cce0b6d9d3d6e1b7d8da1828ef9350c6f270d28c/.

4 https://www.proofpoint.com/us/blog/threat-insight/high-volume-german-phishing-campaign-aims-steal-banking-credentials.

5 https://www.virusbulletin.com/virusbulletin/2018/06/vbspam-comparative-review.

6 http://www.postfix.org/XCLIENT_README.html.

 

Download PDF

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest reviews:

VBSpam comparative review

In the Q3 2022 VBSpam test seven full email security solutions, one custom configured solution, one open-source solution and one blocklist were assembled on the test bench to measure their performance against various streams of wanted, unwanted and…

VBSpam Comparative Review

In the Q2 2022 VBSpam Test seven full email security solutions, one custom configured solution, one open-source solution and two blocklists were assembled on the test bench to measure their performance against various streams of wanted, unwanted and…

VBSpam Comparative Review

In the Q1 2022 VBSpam Test six full email security solutions, one custom configured solution, one open-source solution and two blocklists were assembled on the test bench to measure their performance against various streams of wanted, unwanted and…

VBSpam Comparative Review

In the Q4 2021 VBSpam test the performance of nine full email security solutions, one custom configured solution, one open-source solution and one blocklist was measured against various streams of wanted, unwanted and malicious emails.

VBSpam Comparative Review

In this test the performance of eight full email security solutions, one custom configured solution, one open-source solution and one blocklist was measured against various streams of wanted, unwanted and malicious emails.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.