Yahoo's DMARC policy wreaks havoc among mailing lists

Posted by   Virus Bulletin on   Apr 15, 2014

Collateral damage in instruction to reject emails with invalid DKIM signatures.

A change in Yahoo's DMARC policy has caused frustration among operators of many mailing lists and their subscribers.

On its official website, DMARC is described as standardizing "how email receivers perform email authentication using the well-known SPF and DKIM mechanisms". It was set up in 2012 by some of the bigger players in the email field, including Yahoo, Gmail, Facebook and PayPal.

Both SPF and DKIM are used primarily to authenticate email and to help ensure its correct delivery: SPF can confirm that the mail was sent from one of the mail servers belonging to the sender's domain, while DKIM adds a domain-based signature to the email. If an email was sent from a mail server belonging to and signed with's DKIM signature, it is very likely that it was indeed sent by and not by someone spoofing that domain.

Although neither standard makes a statement on whether authenticated emails are spam, when used together with domain-based reputation, the standards can help with the correct delivery of emails.

What DMARC adds to DKIM and SPF is to give domain owners a say over what receivers should do with emails that claim to have been sent from their domain, but which fail both SPF and DKIM checks. Depending on how much they trust their own records and how important they consider the risk of phishing compared with the risk of legitimate emails not being delivered, they can tell receivers to accept the emails anyway, to turn up the filters, or to reject the emails.

A recent survey among participants of our VBSpam tests showed that very few spam filters are currently checking the DMARC status of emails. Yet its adoption by many of the bigger players means that DMARC cannot be ignored in the current email landscape.

Yahoo's DMARC changes

Yahoo recently changed its DMARC policy to address a security problem. But, as John Levine has pointed out, in doing so it broke every mailing list in the world.

Yahoo apparently saw many phishing emails that forged the From: address to appear to have been sent from its servers. They weren't, and they did indeed lack a valid DKIM signature. Yahoo's decision to add a p=reject flag to its DMARC policy thus may seem reasonable: it told receivers to reject emails that claim to come from Yahoo yet which lack a valid DKIM signature.

However, there is one important case in which emails lack such a signature for a good reason: email discussion lists, more commonly known as mailing lists.

Mailing lists

Popular since the early days of the Internet, and still used by millions, mailing lists involve a central server that forwards emails from one subscriber to the list address to all other subscribers. While forwarding, most list servers modify the subject line of the email to include a 'list tag' and add a footer containing unsubscribe information to the bottom of the email. In doing so, they break any existing DKIM signatures.

To understand why this matters, imagine that Alice sends an email from her Yahoo address to a list server. The server adds a footer, modifies the subject and sends the email to the list's subscribers, including Bob's Gmail address.

Gmail's mail servers notice that the email sent to Bob has an invalid DKIM signature, yet it claims to come from Alice's Yahoo address (list servers typically don't modify the From: header). Following Yahoo's p=reject DMARC flag, rather than deliver the message to Bob, they bounce it back to the list server. The server notices the bounce and does what has long been best practice when messages bounce: it unsubscribes Bob from the list.

This issue isn't new: the same problem arose with the optional and now historic DKIM extension ADSP. Yet, while smaller domains caused problems for a few mailing lists, Yahoo is one of the largest email senders and there are very few lists that don't have any subscribers with a Yahoo address.

The issue has long divided the email community. Some argue that mailing lists have worked this way for decades and it is unreasonable to ask them to change because the relatively recent addition of DKIM has started to cause issues. Others say that leaving the From: address unchanged, while modifying subject and body, simply isn't desirable any more in today's email landscape. Indeed, this is what Yahoo said in defence of its policy.

I have a lot of sympathy for the former argument, which tends to be backed up by the RFCs. Yet Yahoo's changes (assuming they don't change their policy) may see the balance shift the other way. And perhaps that will be better in the long run.

The times are a-changin?

So why did Yahoo decide to ignore those of its users who subscribe to mailing lists?

Some have argued that it did so because it runs its own mailing lists (Yahoo Groups). I personally think the answer is more mundane. As Al Iverson suggests, it is likely that there are simply too few email users subscribing to mailing lists today to make them worth caring about in the face of significant abuse of their domain.

  Al Iverson's speculation on Yahoo's reason for changing its DMARC policy.

It is unlikely that the last words on DKIM and DMARC have been said. To find out more about the method's strengths and weaknesses, an article written by the aforementioned John Levine for Virus Bulletin provides a good introduction. At VB2014, Microsoft's Terry Zink will give a presentation on DMARC.

Posted on 15 April 2014 by Martijn Grooten



Latest posts:

Ransomware not a problem for half of businesses

According to a report by IBM Security, 70 per cent of businesses that are the victim of a ransomware attack end up paying the ransom. However, the report also suggests that a little over half of businesses manage to avoid getting infected at all,…

Ransomware would be much worse if it wasn't for email security solutions

The latest VBSpam test brings good news: at least 199 out of every 200 emails containing a malicious attachment were blocked by email security solutions. All of the full solutions tested achieved a VBSpam award, with five earning a VBSpam+ award.

Throwback Thursday: The malware battle: reflections and forecasts

"Another year has come to its end and the malware battle still rages on." In January 2004, Jamz Yaneza reflected on the year just ended and pondered what the coming year would have in store for the AV industry.

VB2016 paper: Open Source Malware Lab

At VB2016, ThreatConnect Director of Research Innovation Robert Simmons presented a paper on setting up an open source malware lab. Today, we share the accompanying paper and video.

A Christmas present for the security community

As a Christmas present for the security community, we have uploaded most of the papers and videos from the VB2015 conference which took place in Prague almost 15 months ago. The Virus Bulletin crew wishes you all the best for 2017!