VB ESA - M365 Comparative Test Report

Email Security Add-ons for Microsoft 365

Test period: 2026-08-01 to 2026-08-16

Test methodology: https://www.virusbulletin.com/testing/vb-esa-m365/vb-esa-m365-methodology/

Copyright © 2026 Virus Bulletin

Coro Email Security
Elpha Secure

Foreword

This report presents the results of the VB ESA - M365 comparative test covering the period from 1 to 16 August 2026. Whilst the test programme runs continuously, results from official test periods such as this one form the basis of public VB ESA - M365 certification.

Two products participated in this test period under public participation terms: Coro Email Security and Elpha Secure, integrated via API as integrated cloud email security (ICES) solutions. The spam corpus in this period was dominated by Project Honey Pot samples, which accounted for approximately 71% of total spam volume. Readers should bear this in mind when interpreting aggregate spam IDR figures, as overall scores are weighted heavily towards performance on that feed.

Elpha Secure achieved VB ESA - M365 certification; Coro Email Security achieved VB ESA - M365+ certification in this period. Coro Email Security is awarded the Top Performer badge for this period.

We intend to publish VB ESA - M365 results on a regular cadence going forward, building a longitudinal picture of how add-on products perform in the real-world Microsoft 365 environment over time.

— Virus Bulletin Team, September 2026

1. Executive summary

Two products completed the Q3 2026 VB ESA - M365 test period under public participation terms: Coro Email Security and Elpha Secure.

Both products are VB ESA - M365 certified in this period.

Results at a glance

Coro Email Security: Spam IDR 96% | False positives 0 | ✓ VB ESA - M365+ | ✓ Malware 100 | ✓ Phishing 100

Elpha Secure: Spam IDR 94% | False positives 0 | ✓ VB ESA - M365 | ✓ Malware 100

Microsoft 365 (Exchange Online Protection alone) passed 131 of 93,734 spam-feed messages, four of 9,725 phishing samples, and 21 of 1,013 malware samples through to the inbox in this period.

Full per-product results, the combined comparative table, and the test methodology follow in Sections 3–6 of this report.

Note on Microsoft 365: This report does not evaluate Microsoft 365 or Exchange Online Protection. Microsoft 365 is used solely as a comparative base to isolate the incremental value each tested add-on provides on top of native filtering. Results for Microsoft 365 reflect the specific tenant configuration, licensing tier, and sample mix used in this test instance and cannot be generalized to other deployments. No conclusions about Microsoft 365’s general effectiveness should be drawn from this report.

2. The VB ESA - M365 TEST

VB ESA - M365 is Virus Bulletin’s continuously running test programme for products that supplement Microsoft 365’s native email security – Exchange Online Protection (EOP) – with an additional detection layer. It covers integrated cloud email security (ICES) solutions, which connect via API without changing MX records.

Tested products are exposed to a continuous, live stream of real-world email: spam drawn from third-party spam feed providers including Abusix and Project Honey Pot; phishing emails containing credential-theft or malware-delivery links; malware‑bearing attachments; and legitimate ham and newsletter traffic. No email is artificially constructed for the test – all test cases are in the wild.

2.1 What is actually being measured

The test is built around a Microsoft 365 comparative base consisting of the set of samples that Microsoft 365 itself disposed of (marking them as ‘Failed’ or ‘FilteredAsSpam’ in every test instance, or ‘Quarantined’ in at least one) and which therefore fall outside the scope of evaluation for the tested add-on products. Samples in this base are excluded from product scoring entirely. The samples that remain (the ‘residual sample set’) are what actually reached each product, and it is on this residual set alone that detection and false-positive performance is measured.

The three status values used to exclude samples are native Microsoft 365 / Exchange Online Protection labels. Each exclusion rule reflects what actually happens to that message in production:

• Failed: The message never leaves Microsoft 365’s infrastructure; it is rejected or bounced before delivery. No add-on ever processes it.
• FilteredAsSpam: Microsoft 365 consistently diverts the message to the Junk Email folder. In a real deployment the add‑on only sees mail that EOP passes, not mail that it catches, so these samples are excluded. The ‘every instance’ threshold avoids penalizing products for occasional EOP inconsistency.
• Quarantined: Microsoft 365 places the message under administrative hold, removing it from user reach entirely. The ‘at least once’ condition (versus ‘every instance’ for junk) reflects that quarantining a message is a strong and deliberate action, unlikely to be accidental.

In any of these cases, scoring an add-on against these samples would measure something that doesn’t happen in a live environment. The residual set is therefore the most accurate proxy for what a product actually sees, and must handle, when deployed alongside Microsoft 365.

2.2 Incremental detection rate (IDR)

The primary detection metric is the incremental detection rate, or IDR:

IDR = (number of malicious samples detected by the add-on) ÷ (number of malicious samples that passed Microsoft 365 filtering) 

‘Detected’ means any add-on action that prevents delivery or places the message under administrative control (typically a block or quarantine).

2.3 Certifications and badges 

The VB ESA - M365 test programme recognizes exceptional performance through a series of awards and badges. 

3. Test environment

Testing was conducted against a Microsoft 365 tenant licensed on Microsoft 365 Business Basic / Business Standard – Exchange Online Protection only, with no Defender for Office 365 overlay, so that measured results are attributable to the tested add-ons rather than to a richer native Microsoft stack. Each product received live email continuously via a dedicated Exchange admin centre connector for the duration of the test period. The test period ran for 16 days (2026-08-01 to 2026-08-16), with an average of 5,858 spam samples per day.

3.1 Sample volumes

The following tables show the total test-case volumes for the test period, and the number of each that passed Microsoft 365 filtering into the residual sample set.   

Malicious email 

Legitimate email

Microsoft 365 itself generated 386 ham false positives this period – blocking 386 legitimate messages before the add-ons were involved. Coro Email Security and Elpha Secure each preserved all 150 ham messages correctly passed by M365, adding zero further false positives.

Spam feed corpus composition.

Threat corpus composition.

Note on the newsletter feed

No newsletter samples were submitted during this test period. The newsletter feed is operated on a best-effort basis and may be empty in some test periods, depending on availability. False-positive scoring for newsletters is therefore not applicable for this test.

3.2 Geographic origin of spam samples

The following table shows the number of spam samples in the residual set by country of origin*, as determined by GeoIP lookup on the sending IP address.

4. Overall results

The table that follows summarizes the participating products’ performance on the residual sample set for this test period.

Incremental value over M365 alone – spam

The following table outlines how much each product adds on top of native Microsoft 365 filtering.

Incremental detection rates across spam feeds.

Spam detection rate heat map.

Block rate of M365 alone vs M365 + product across spam feeds.

Incremental detection rates across malware and phishing feeds.

Malware and phishing detection rate heat map.

Block rate of M365 alone vs M365 + product across malware and phishing feeds.

5. Product results

5.1 Coro Email Security

• Website: https://www.coro.net
• How to integrate with Microsoft 365: https://docs.coro.net/protection/connecting-to-m365/

 ✓ VB ESA - M365+ | ✓ Top Performer | ✓ Phishing 100 | ✓ Malware 100 

Spam IDR 96.2% | False positives (Ham + Newsletter) 0

Coro Email Security detected 126 of the 131 residual spam samples submitted to it, missing five messages. It correctly allowed through all ham traffic. 

• False positives against ham traffic: 0
• Phishing false negatives: 0 of 4 residual
• Malware false negatives: 0 of 21 residual

Detection performance

Note: The Abusix IDR (92.7%) and the Project Honey Pot IDR (98.7%) diverge by 6.0 percentage points. Both are third‑party spam feeds of the same type, but the Abusix residual is smaller (55 samples) – each missed message on Abusix costs 1.8 percentage points, amplifying the visible gap.

Legitimate mail handling

Message disposition – all feeds

Spam detection by email language

Combined performance – M365 + Coro Email Security

The following table shows the combined effect of Microsoft 365 native filtering and the product acting together. For malicious feeds, a higher combined block rate is better. For legitimate feeds, a lower combined block rate is better (blocked = false positive).

5.2 Elpha Secure

• Website: https://www.elphasecure.com   
• How to integrate with Microsoft 365: https://www.elphasecure.com/technology/how-it-works

✓ VB ESA - M365 | — Top Performer | — Phishing 100 | ✓ Malware 100  

Spam IDR 93.9% | False positives (Ham + Newsletter) 0

Elpha Secure detected 123 of the 131 residual spam samples submitted to it, missing eight messages. It correctly allowed through all ham traffic.

• False positives against ham traffic: 0
• Phishing false negatives: 1 of 4 residual
• Malware false negatives: 0 of 21 residual

Detection performance

Note: The Abusix IDR (85.5%) and the Project Honey Pot IDR (100.0%) diverge by 14.5 percentage points. Both are third‑party spam feeds of the same type, but the Abusix residual is smaller (55 samples) – each missed message on Abusix costs 1.8 percentage points, amplifying the visible gap.

Legitimate mail handling

Message disposition — all feeds

Spam detection by email language

Combined performance – M365 + Elpha Secure

The following table shows the combined effect of Microsoft 365 native filtering and the product acting together. For malicious feeds, a higher combined block rate is better. For legitimate feeds, a lower combined block rate is better (blocked = false positive).

6. Methodology summary

6.1 Full methodology and procedure

The full methodology, award criteria, and test procedure can be found at: https://www.virusbulletin.com/testing/vb-esa-m365/vb-esa-m365-methodology/.

6.2 Message trace field definitions

All per-message status values in this report are extracted from the Microsoft 365 message trace facility. Message traces can be accessed by Exchange administrators at:

• https://admin.cloud.microsoft/exchange#/messagetrace

Each message submitted to the test is injected into a dedicated Microsoft 365 tenant and its fate recorded by querying the message trace API. The fields below correspond to the status values returned and used throughout this report.

7. About VB ESA - M365

VB ESA - M365 is Virus Bulletin’s comparative test programme for products that supplement Microsoft 365 email security. Products may participate publicly, with results published as in this report, or privately, receiving only non-comparative feedback. Public test participants are tested over a continuous one-year cycle, within which four periods are designated as official test periods; results from official periods form the basis of public certification.

Vendors wishing to enrol a product in VB ESA - M365 can find participation details at https://www.virusbulletin.com/testing/vb-esa-m365/vb-esa-m365-vendors/.

Latest reviews: