Yahoo's DMARC policy wreaks havoc among mailing lists

Posted by   Virus Bulletin on   Apr 15, 2014

Collateral damage in instruction to reject emails with invalid DKIM signatures.

A change in Yahoo's DMARC policy has caused frustration among operators of many mailing lists and their subscribers.

On its official website, DMARC is described as standardizing "how email receivers perform email authentication using the well-known SPF and DKIM mechanisms". It was set up in 2012 by some of the bigger players in the email field, including Yahoo, Gmail, Facebook and PayPal.

Both SPF and DKIM are used primarily to authenticate email and to help ensure its correct delivery: SPF can confirm that the mail was sent from one of the mail servers belonging to the sender's domain, while DKIM adds a domain-based signature to the email. If an email was sent from a mail server belonging to example.com and signed with example.com's DKIM signature, it is very likely that it was indeed sent by example.com and not by someone spoofing that domain.

Although neither standard makes a statement on whether authenticated emails are spam, when used together with domain-based reputation, the standards can help with the correct delivery of emails.

What DMARC adds to DKIM and SPF is to give domain owners a say over what receivers should do with emails that claim to have been sent from their domain, but which fail both SPF and DKIM checks. Depending on how much they trust their own records and how important they consider the risk of phishing compared with the risk of legitimate emails not being delivered, they can tell receivers to accept the emails anyway, to turn up the filters, or to reject the emails.

A recent survey among participants of our VBSpam tests showed that very few spam filters are currently checking the DMARC status of emails. Yet its adoption by many of the bigger players means that DMARC cannot be ignored in the current email landscape.

Yahoo's DMARC changes

Yahoo recently changed its DMARC policy to address a security problem. But, as John Levine has pointed out, in doing so it broke every mailing list in the world.

Yahoo apparently saw many phishing emails that forged the From: address to appear to have been sent from its servers. They weren't, and they did indeed lack a valid DKIM signature. Yahoo's decision to add a p=reject flag to its DMARC policy thus may seem reasonable: it told receivers to reject emails that claim to come from Yahoo yet which lack a valid DKIM signature.

However, there is one important case in which emails lack such a signature for a good reason: email discussion lists, more commonly known as mailing lists.

Mailing lists

Popular since the early days of the Internet, and still used by millions, mailing lists involve a central server that forwards emails from one subscriber to the list address to all other subscribers. While forwarding, most list servers modify the subject line of the email to include a 'list tag' and add a footer containing unsubscribe information to the bottom of the email. In doing so, they break any existing DKIM signatures.

To understand why this matters, imagine that Alice sends an email from her Yahoo address to a list server. The server adds a footer, modifies the subject and sends the email to the list's subscribers, including Bob's Gmail address.

Gmail's mail servers notice that the email sent to Bob has an invalid DKIM signature, yet it claims to come from Alice's Yahoo address (list servers typically don't modify the From: header). Following Yahoo's p=reject DMARC flag, rather than deliver the message to Bob, they bounce it back to the list server. The server notices the bounce and does what has long been best practice when messages bounce: it unsubscribes Bob from the list.

This issue isn't new: the same problem arose with the optional and now historic DKIM extension ADSP. Yet, while smaller domains caused problems for a few mailing lists, Yahoo is one of the largest email senders and there are very few lists that don't have any subscribers with a Yahoo address.

The issue has long divided the email community. Some argue that mailing lists have worked this way for decades and it is unreasonable to ask them to change because the relatively recent addition of DKIM has started to cause issues. Others say that leaving the From: address unchanged, while modifying subject and body, simply isn't desirable any more in today's email landscape. Indeed, this is what Yahoo said in defence of its policy.

I have a lot of sympathy for the former argument, which tends to be backed up by the RFCs. Yet Yahoo's changes (assuming they don't change their policy) may see the balance shift the other way. And perhaps that will be better in the long run.

The times are a-changin?

So why did Yahoo decide to ignore those of its users who subscribe to mailing lists?

Some have argued that it did so because it runs its own mailing lists (Yahoo Groups). I personally think the answer is more mundane. As Al Iverson suggests, it is likely that there are simply too few email users subscribing to mailing lists today to make them worth caring about in the face of significant abuse of their domain.


  Al Iverson's speculation on Yahoo's reason for changing its DMARC policy.

It is unlikely that the last words on DKIM and DMARC have been said. To find out more about the method's strengths and weaknesses, an article written by the aforementioned John Levine for Virus Bulletin provides a good introduction. At VB2014, Microsoft's Terry Zink will give a presentation on DMARC.

Posted on 15 April 2014 by Martijn Grooten

twitter.png
fb.png
linkedin.png
googleplus.png
reddit.png

 

Latest posts:

VB2016 video: Last-minute paper: A malicious OS X cocktail served from a tainted bottle

In a VB2016 last-minute presentation, ESET researchers Peter Kalnai and Martin Jirkal looked at the OS X malware threats KeRanger and Keydnap, that both spread through a compromised BitTorrent client. A recording of their presentation is now…

Consumer spyware: a serious threat with a different threat model

Consumer spyware is a growing issue and one that can have serious consequences: its use is increasingly common in domestic violence. But do our threat models consider the attacker with physical access to, and inside knowledge of the victim?

VB2016 paper: Debugging and monitoring malware network activities with Haka

In their VB2016 paper, Stormshield researchers Benoît Ancel and Mehdi Talbi introduced Haka, an open-source language to monitor, debug and control malicious network traffic. Both their paper and the video recording of their presentation are now…

VB2017: a wide ranging and international conference programme

We are proud to announce a very broad and very international programme for VB2017, which will take place in Madrid, 4-6 October 2017.

John Graham-Cumming and Brian Honan to deliver keynote addresses at VB2017

Virus Bulletin is excited to announce John-Graham Cumming and Brian Honan as the two keynote speakers for VB2017 in Madrid.