Subspace mailing list server

We have recently announced the availability of our new mailing list platform that will eventually take on the duties currently performed by vger. Off the bat, there were a few questions about how it works under the hood — especially regarding DMARC-friendly cofiguration.

Under the hood

There is nothing fancy about the setup — mailing lists are managed by mlmmj, while all delivery operations are handled by Postfix. All outgoing mail is delivered via kernel.org mirror edge nodes (see the output of dig -t txt +short _spf.kernel.org if you are curious what they are), which is mostly done to speed up delivery by spreading out the queue across several systems.

When mlmmj writes to the archive directory of the mailing list, the message is immediately picked up by public-inbox-watch and appended to the public-inbox archive for that list. The archive is then replicated to lore.kernel.org (using grokmirror integration), which usually happens within 60 seconds. This replication is parallel to Postfix delivering mail to list subscribers, so is not dependent on the size of the mail queue. In theory, it shouldn't take longer than a few minutes for a message sent to a lists.linux.dev address to show up on lore.kernel.org. Similarly, messages should never go missing from the public-inbox archive if they got accepted by mlmmj for delivery (I know, famous last words).

Appeasing DMARC

It's a common misconception that mailing lists are somehow incompatible with DMARC. There are two key principles to follow:

  1. The Envelope-From should be that of the mailing list domain. For example, if I send an email to linux-staging, the envelope-from of the outgoing message will be changed to linux-staging+bounces-x@lists.linux.dev, with some subscriber bounce tracking information in place of x. This way, when MTAs are looking at DMARC for “konstantin@linuxfoundation.org”, the SPF check will be performed against “lists.linux.dev” instead of the domain of the original sender (“linuxfoundation.org”).

  2. There should be no changes to any of the existing message headers and no modifications to the message body. This is actually the part that generally trips up mailing list operators, as it is a long-standing practice to do two things when it comes to mailing lists: modify the subject to insert a terse list identifier (e.g. Subject: [linux-staging] [PATCH] ...) and append a footer to the message body with mailing list administrative info. Doing either of these things will almost certainly invalidate DKIM signatures and therefore cause the message to fail the DMARC check. It is correct to add proper List-Id/List-Subscribe/etc headers, though — and hopefully the domain of the original sender isn't misconfigured to include these headers into the DKIM signature (true story, saw this happen).

Following the above advice will work for nearly all cases except where a domain sets a DMARC policy, but the message is sent without a DKIM signature. If this happens, DMARC validators are supposed to use a kludgy “alignment” check where the envelope-from must match the From: header. In that particular case the messages we send out will fail DMARC checks, unfortunately. As far as I'm concerned, this is the fault of domain owners and is properly fixed by setting up proper DKIM signing and giving users a way to send outgoing mail via proper SMTP gateways.

(There is a way to work around this by rewriting the “From: “ header so that it matches the list domain as well, but let's just not go there, as rewriting the From: header is not an acceptable solution for lists working with code reviews.)

Here's a write-up I randomly found while writing this post that goes into some more detail regarding DMARC and mailing lists.

Why no ARC headers?

We don't currently add ARC headers — as far as I can tell, they aren't required for operating a mailing list that properly sets the envelope-from. In theory, using ARC signing may help with the “DMARC with no DKIM” corner-case above, but I'm not convinced this is worth the crazy header bloat. Who knows, I may change my mind about this in the future.

Parting words

In short, the best way to assure that a message sent via subspace.kernel.org is delivered to all subscribers is to send it from a domain that properly DKIM-signs all mail. If you run your own server, you can either set up OpenDKIM on your own (it's not complicated, honest), or you can pay some money to a company like Mailgun to do it for you.