Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 541216

Summary: Support newer SPAM fighting tools on Eclipse.org mailing lists
Product: Community Reporter: Eclipse Webmaster <webmaster>
Component: MailingListsAssignee: Eclipse Webmaster <webmaster>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: bernd.hufmann, christian.dietrich.opensource, daniel_megert, denis.roy, gael.lalire, gunnar, jonah
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Whiteboard:
Bug Depends on:    
Bug Blocks: 522316, 558237    

Description Eclipse Webmaster CLA 2018-11-15 15:51:08 EST
It's time to upgrade our Mailman install to provide support for sites using  technologies like DKIM,DMARC and SPF, however enabling these options may break some traditional/expected mailing list behaviours.

The obvious fix is to change our lists so that messages are no longer sent out as if they were from the 'sending' user, but rather so that all messages originate from the list itself. We would also need to remove any existing DKIM headers and generate new ones.  

Depending on the feedback to this bug I'll look at implementing the above change in Q1.

-M.
Comment 1 Bernd Hufmann CLA 2019-01-10 13:31:00 EST
My company Ericsson will soon activate DMARC. So, we won't see emails from Ericsson employees to mailing lists. The activation supposed to happen in the month of January 2019. Is there any chance to have this bug fixed in the early Q1?

Thanks
Bernd
Comment 2 Eclipse Webmaster CLA 2019-01-14 14:42:43 EST
It's on the list for Q1, but right now I'm not sure when it will actually happen.

-M.
Comment 3 Eclipse Webmaster CLA 2020-04-22 12:03:53 EDT
*** Bug 562403 has been marked as a duplicate of this bug. ***
Comment 4 Gael Lalire CLA 2020-04-22 12:14:19 EDT
As 562403 was marked as duplicate of this one, please also add TLS support.
Even if TLS is not a SPAM fighting tool, but a confidentiality (encryption) tool.
Comment 5 Jonah Graham CLA 2020-06-05 15:38:10 EDT
Hi Webmasters - a gentle ping on this topic. The number of mailing list emails that get marked as spam by gmail with messages like below continues to rise.

Gmail could not verify that it actually came from *****. Avoid clicking links, downloading attachments or replying with personal information.
Comment 6 Eclipse Webmaster CLA 2020-10-08 17:07:13 EDT
Our new MTA has been created, and is currently being tested.  Once we're convinced it's ready for production we'll make it live.

-M.
Comment 7 Jonah Graham CLA 2020-12-04 14:50:31 EST
(In reply to Eclipse Webmaster from comment #0)
> The obvious fix is to change our lists so that messages are no longer sent
> out as if they were from the 'sending' user, but rather so that all messages
> originate from the list itself. We would also need to remove any existing
> DKIM headers and generate new ones.  

The email that was just sent on eclipse.org-committers brought up a question for me.

The change to the sending user seems unsatisfying. I assume it won't be as bad as the current tracecompass-dev workaround. However I am subscribed to numerous lists such as gdb-patches@sourceware.org and emails sent by that list don't fail DMARC. 

Can you do what sourceware.org is doing?

I can send you full headers for some of the messages if it is helpful, but here is the summary from gmail for one of them:

Message ID	<1063873041.7900814.1594230222719@mail.yahoo.com>
Created on:	8 July 2020 at 13:43 (Delivered after 12 seconds)
From:	Hannes Domani <ssbssa@yahoo.de> Using WebService/1.1.16197 YMailNorrin Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:76.0) Gecko/20100101 Firefox/76.0
To:	Gdb-patches <gdb-patches@sourceware.org>
Subject:	Re: [PATCH 7/7] Reset Windows hardware breakpoints on executable's entry point
SPF:	PASS with IP 173.228.157.41 Learn more
DKIM:	'PASS' with domain yahoo.de Learn more
DMARC:	'PASS' Learn more
Comment 8 Jonah Graham CLA 2020-12-04 14:59:33 EST
(In reply to Jonah Graham from comment #7)
> I assume it won't be as
> bad as the current tracecompass-dev workaround.

For those not on tracecompass-dev, see Bug 558237 Comment 5
Comment 9 Eclipse Webmaster CLA 2020-12-04 16:56:35 EST
(In reply to Jonah Graham from comment #7)

> I assume it won't be as bad as the current tracecompass-dev workaround.

Broadly I suspect it will be similar. When I've tested without it, invariably one of the 'spam fighters'(spf/dkim/dmarc) complains and fixing that simply causes another one of them to whine, at which point Gmail at least starts putting things in the spam folder.

I don't particularly like this change either, but it seems unavoidable to me.  However I'm willing to keep testing while the new mx is coming online to see if there is a way to avoid it.

> Can you do what sourceware.org is doing?

I have no insight into how their mail infrastructure is setup, so if you can provide more specific technical details I can try and answer that question.

-M.
Comment 10 Gunnar Wagenknecht CLA 2020-12-05 10:23:06 EST
(In reply to Eclipse Webmaster from comment #9)
> I don't particularly like this change either, but it seems unavoidable to
> me.  However I'm willing to keep testing while the new mx is coming online
> to see if there is a way to avoid it.

Is it the workaround mentioned here?
https://doc.coker.com.au/internet/dkim-and-mailing-lists/

It really seems unavoidable with Mailman

> > Can you do what sourceware.org is doing?
> 
> I have no insight into how their mail infrastructure is setup, so if you can
> provide more specific technical details I can try and answer that question.

I found this one to:
https://begriffs.com/posts/2018-09-18-dmarc-mailing-list.html

I also found the use of Google Groups to be usually flawless. Not sure if this could be a solution for managed lists.

FWIW, it seems to be that there are multiple issues at play here. It's not just the mailing list software but also all the spam related settings for eclipse.org domains etc.

Thanks for digging into the issue, though!
Comment 11 Jonah Graham CLA 2020-12-05 10:38:02 EST
(In reply to Gunnar Wagenknecht from comment #10)
> I found this one to:
> https://begriffs.com/posts/2018-09-18-dmarc-mailing-list.html

This one has the following recommendation:

Lists should keep the From address, the Subject, and the Message totally unchanged. They should add a Sender header to indicate their relay role, and set at least the List-Id and List-Unsubscribe headers for mailbox rules and subscription management.


FWIW the sourceware mailing lists mentioned in Comment 7 appear to follow this recommendation.
Comment 12 Eclipse Webmaster CLA 2021-02-18 15:24:18 EST
It's taken a while but we are now publishing SPF/DMARC/DKIM records and signing outbound mail.  Hopefully this will help improve delivery rates.

I'm going to close this as 'fixed', and if there are still issues with the mailing lists let's open another bug.

-M.
Comment 13 Denis Roy CLA 2021-02-18 15:35:54 EST
This was a bear of a task; thanks for seeing it through, Matt.
Comment 14 Jonah Graham CLA 2021-02-22 10:37:47 EST
Thanks Matt for your work on this. I am very happy that the end result was not that we all end up with a mailing list like tracecompass!

There are still some issues and as requested in Comment 12 I have created Bug 571408 for the follow up.