Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[15.0][16.0] Odoo not following RFC 8058 - mass e-mails will go to spam #165169

Open
ossimantylahti opened this issue May 12, 2024 · 11 comments
Open
Labels
15.0 16.0 Marketing CRM, mail, event, livechat, mass mailing, online appointments, ...

Comments

@ossimantylahti
Copy link
Contributor

ossimantylahti commented May 12, 2024

It seems that Odoo's mass e-mail marketing does not follow RFC 8058. This RFC dictates that unsubcribing from a mass e-mail list should happen with one click only. There cannot be any other pages after clicking on the link.

This is time critical defect, since from 1st June 2024 onwards Google, Yahoo! and Microsoft start to automatically deliver mass e-mails that do not follow RFC 8058 to spam box.

*Impacted versions:
[15.0][16.0] are impacted.

[17.0] works ok.
*Steps to reproduce:

Send an e-mail using Odoo's e-mail mass marketing tool
Try to unsubscribe from the list by clicking on the Unsubscribe button

*Current behavior:

After clicking on the link, I'm getting 403: Forbidden error
After clicking on the link, the email still remains in the mass e-mail list

  • Expected behavior:

I expect that Odoo should follow RFC 8058 and let user unsubcribe himself from the list by one click. Right now 1) the unsubcribe does not work at all due to 403. 2) It should be ONE CLICK only without any additional confirmations.

  • Video/Screenshot link (optional):
    Screencaps:
  1. Testing with a simple plaintext mass e-mail template:

image

  1. Testing with another template
    image

  2. Email arrives and the unsubcribe link is below
    image

  3. But clicking on the link gives 403 forbidden error
    image

4b. Same thing with another browser and incognito mode:
image

  1. Email still remains in the mass e-mail list and is not even blacklisted.
    image
  • Support ticket number submitted via odoo.com/help (optional):
    #3924174
@ossimantylahti
Copy link
Contributor Author

Here is Mailgun's statement about RFC 8058 enforcement.

https://www.mailgun.com/blog/deliverability/what-is-rfc-8058/#chapter-3

“Bulk senders have until June 1 to implement one-click unsubscribe on the Google front in all of your commercial and promotional messages.”
Anu Yamunan, Director of Product for Anti-Abuse & Safety at Google

@ossimantylahti
Copy link
Contributor Author

ossimantylahti commented May 12, 2024

To make things worse, Odoo does not generate List-Unsubcribe headers. And is passing session token data. RFC states that this cannot be done:

"Also, the request MUST NOT include cookies or other context information to prevent the server from associating the request with previous web requests."

Additionally, Odoo does not check if the email sender domain DKIM is configured correctly. This is a requirement to comply with the RFC.

Requirements for unsubscribing in plain English: If you manage your own email program, or even just your unsubscribes, you will have to manually implement a one-click unsubscribe process.

Senders must include one List-Unsubscribe header field and one List-Unsubscribe-Post header field in their message.

The List-Unsubscribe header field must contain one HTTPS URI.

The List-Unsubscribe-Post header must contain the value “List-Unsubscribe=One-Click”.

The message MUST have a valid DKIM signature to cover the List-Unsubscribe, and List-Unsubscribe-Post headers.

The URI must include sufficient information to identify the mail recipient and the list from which they are to be removed.

The post request MUST NOT include cookies, HTTP auth, or any other identifying data that might link the unsubscribe action to any previous web activity.

@jorv-odoo
Copy link
Contributor

jorv-odoo commented May 13, 2024

HI @ossimantylahti ,
Just to clarify, as it is not clear from your reproduction steps, did you send the email using the

  1. "Test" button and wait for the email to reach your inbox -> click unsubscribe link
  2. Using the proper flow of Send -> In Queue (Wait for CRON) -> Receive the email then click the button

Note that in scenario 1, the "Test" button email mostly serves for visualizing the rendered template but will not have the specific unsubscribe URLs generated and linked to the respective models.
Did you test it using scenario 2, and if so do you still receive an 403 error?

Thank you in advance.

@jorv-odoo jorv-odoo added Marketing CRM, mail, event, livechat, mass mailing, online appointments, ... 15.0 16.0 labels May 13, 2024
@tde-banana-odoo
Copy link
Contributor

Hello,

Indeed the test button is in a flow where it is difficult to have the unsubcribe links working. However standard marketing emails effectively have those headers set, depending on the version / module (mass mailing, digest, mailing lists). We plan to try to backport improvements in 15.0 so that all major versions have those headers.

Cheers,

@IT-Ideas
Copy link
Contributor

IT-Ideas commented May 16, 2024

Hello,

Indeed the test button is in a flow where it is difficult to have the unsubcribe links working. However standard marketing emails effectively have those headers set, depending on the version / module (mass mailing, digest, mailing lists). We plan to try to backport improvements in 15.0 so that all major versions have those headers.

Cheers,

Hello @tde-banana-odoo !

Hope all is going well at the farm 😄

Once the fix is ready, could you link the PR to the issue so that we will be able to get the fix code asap. I personally need to back port it in v13.0 😅

Thanks !

@ossimantylahti
Copy link
Contributor Author

@jorv-odoo I can confirm that the 403 error is indeed due to sending the messages with "Test" button. When clicking on the unsubscribe on the actual scheduled e-mail, that unsubscribe works.

However, the message is still missing List-Unsubscribe header fields and one List-Unsubscribe-Post header field s required by the RFC. Those fields should be added in order to avoid sending the e-mails directly to the spam folder.

@ossimantylahti
Copy link
Contributor Author

However standard marketing emails effectively have those headers set, depending on the version / module (mass mailing, digest, mailing lists).

They are not set. Here are raw headers from Odoo 16 EE test from day before yesterday.

Received: from AS2PR08MB9047.eurprd08.prod.outlook.com (2603:10a6:20b:5ff::10)
 by VE1PR08MB5792.eurprd08.prod.outlook.com with HTTPS; Mon, 13 May 2024
 20:07:24 +0000
Received: from DB8PR09CA0003.eurprd09.prod.outlook.com (2603:10a6:10:a0::16)
 by AS2PR08MB9047.eurprd08.prod.outlook.com (2603:10a6:20b:5ff::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Mon, 13 May
 2024 20:07:22 +0000
Received: from DB5PEPF00014B8B.eurprd02.prod.outlook.com
 (2603:10a6:10:a0:cafe::54) by DB8PR09CA0003.outlook.office365.com
 (2603:10a6:10:a0::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Mon, 13 May 2024 20:07:22 +0000
Authentication-Results: spf=pass (sender IP is 51.15.253.103)
 smtp.mailfrom=signdemo.odoo.com; dkim=pass (signature was verified)
 header.d=odoo.com;dmarc=pass action=none
 header.from=signdemo.odoo.com;compauth=pass reason=100
Received-SPF: Pass (protection.outlook.com: domain of signdemo.odoo.com
 designates 51.15.253.103 as permitted sender)
 receiver=protection.outlook.com; client-ip=51.15.253.103;
 helo=mailsaas10a.odoo.com; pr=C
Received: from mailsaas10a.odoo.com (51.15.253.103) by
 DB5PEPF00014B8B.mail.protection.outlook.com (10.167.8.199) with Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7587.21
 via Frontend Transport; Mon, 13 May 2024 20:07:20 +0000
Received: from 79.164.79.34.bc.googleusercontent.com (79.164.79.34.bc.googleusercontent.com [34.79.164.79])
	by mailsaas10a.odoo.com (Postfix) with ESMTPS id 2BFEB17D7B4
	for <info@obs-solutions.fi>; Mon, 13 May 2024 22:07:20 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=odoo.com; s=mail;
	t=1715630840; bh=uEOLEICP1j5QUaONmcAMg8aOEnLkmCXOi9ClDe6TnDo=;
	h=Subject:Reply-To:To:Date:From:From;
	b=UyTR/30BojL4hr8ehDuRRftwBfRehXZ9H1TpD17Qa9ACJT3JJ6hwLm6772weRYKue
	 D6nJlSPh8UNPRBscW1pUTy1pFATOTgwRfG3D1GReypkP4bFa7QtGiUzZFsxlEgo/EZ
	 1k36kAfkoMfuvtLJtmRFGFtKmpIgE2wPZ7PdjMWk=
Message-Id: <785354336216087.1715630839.699951171875000-openerp-reply_to@eupq02>
Subject: 2nd test e-mail to Los Ossis
Reply-To: "Administrator" <admin@signdemo.odoo.com>
To: info@obs-solutions.fi
Date: Mon, 13 May 2024 20:07:20 -0000
X-Odoo-Objects: mailing.contact-4
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="===============8995063412556372739=="
From: "Administrator" <admin@signdemo.odoo.com>
Return-Path: bounce@signdemo.odoo.com
X-MS-Exchange-Organization-ExpirationStartTime: 13 May 2024 20:07:20.7114
 (UTC)
X-MS-Exchange-Organization-ExpirationStartTimeReason: OriginalSubmit
X-MS-Exchange-Organization-ExpirationInterval: 1:00:00:00.0000000
X-MS-Exchange-Organization-ExpirationIntervalReason: OriginalSubmit
X-MS-Exchange-Organization-Network-Message-Id:
 cdef1601-e85a-441e-d961-08dc73884bf8
X-EOPAttributedMessage: 0
X-EOPTenantAttributedMessage: 9f8541fa-0a22-45df-98fc-ac9745b88200:0
X-MS-Exchange-Organization-MessageDirectionality: Incoming
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic:
 DB5PEPF00014B8B:EE_|AS2PR08MB9047:EE_|VE1PR08MB5792:EE_
X-MS-Exchange-Organization-AuthSource:
 DB5PEPF00014B8B.eurprd02.prod.outlook.com
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Office365-Filtering-Correlation-Id: cdef1601-e85a-441e-d961-08dc73884bf8
X-MS-Exchange-AtpMessageProperties: SA|SL
X-MS-Exchange-Organization-SCL: 1
X-Microsoft-Antispam: BCL:0;ARA:13230031|5073199003;
X-Forefront-Antispam-Report:
 CIP:51.15.253.103;CTRY:FR;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mailsaas10a.odoo.com;PTR:mailsaas10a.odoo.com;CAT:NONE;SFS:(13230031)(5073199003);DIR:INB;
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 May 2024 20:07:20.4770
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: cdef1601-e85a-441e-d961-08dc73884bf8
X-MS-Exchange-CrossTenant-Id: 9f8541fa-0a22-45df-98fc-ac9745b88200
X-MS-Exchange-CrossTenant-AuthSource:
 DB5PEPF00014B8B.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: Internet
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB9047
X-MS-Exchange-Transport-EndToEndLatency: 00:00:03.9184956
X-MS-Exchange-Processed-By-BccFoldering: 15.20.7544.049
X-Microsoft-Antispam-Mailbox-Delivery:
	ucf:0;jmr:0;auth:0;dest:I;ENG:(910001)(944506478)(944626604)(920097)(930097)(140003)(1420198);
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?1/HP2ASU1uqYqk8nTULufiqtrrBknpflmPAkLQmIaNRIE6FOY5neqVJPu5EO?=
 =?us-ascii?Q?6uw/Yt+hyckzYoeqs8cwMHuzHKqp836sYIZodCGakSiPs5joD4GjRkiqnYDN?=
 =?us-ascii?Q?aiyuPkxR/xn68jbglKyMzCRtIxuksKtni4v1CSc/ETsuG8o36Lm4ThGYgQFn?=
 =?us-ascii?Q?aW09ogYoEdArdfVTbQWpXiT/p1Wpba0Leo76ckio0jhYt4vtPJ9a8v1kxcwp?=
 =?us-ascii?Q?nQKZ6aWsewQ2XAG7nLm1lnAcf6DqESNXnGu9y6ke3Yt35hP2R2NZ31/WAlNH?=
 =?us-ascii?Q?kTrpCOhv7loWxTuMlQLpHLic/dEgOyVS6ZSL5H4lNVHK98h1ntNZuoz7S/rh?=
 =?us-ascii?Q?cSw1tzvIZtKaOOGTjxYIehsJXOIQqWzJCqRvrd+argIj+lsdLdDowpg6Z4dm?=
 =?us-ascii?Q?ZbEssYOaKWskVBmtpf8QcPPhIygwWxm3bguADkkQ/TohhvsoQJlFF0nzMMbo?=
 =?us-ascii?Q?iGLUqGDa551+wF4dnkhEUYEaRNZomY6QYYBfu/gW/vJD2fGZjWmzHBmlhkcY?=
 =?us-ascii?Q?SH0cdOM7HcgGVWlRYf2lqPpJ9jLLdt7KY0qDl1HMa6hV+flrTvguSU6+e1fm?=
 =?us-ascii?Q?5WGqI79Cp1pWEhUTtBha3eGcgg3fu3ox4/qkJ3FOY8d3YLY2Miiwn1TjFL9a?=
 =?us-ascii?Q?11yHNXanaojlAHSVgHjoLR+9wagrqNWFE5rQgkG1pRniTOdgz+jnIXGvK7rg?=
 =?us-ascii?Q?hknW9KYVUCPZ0ECgthMQGbh3nmD+N4ytn3K37D6ka+p8+iBEGl9P5+QpjaTN?=
 =?us-ascii?Q?Oik0O/aOh090q6ykXGetgw1SWVLi0InUk4IqDCo/rPvLlyUXhuhmlMrUbVJJ?=
 =?us-ascii?Q?2ZFetfOonpIKML7plfDUZIiJNKOiKljWrL5ksI9ZGMSk19p15Q7mdV05mEeT?=
 =?us-ascii?Q?/n2WyFELvnQhTUObyU/60SWLfMlbQ2t3QogsyN/n6+srbevp4JJS9GOSLzqy?=
 =?us-ascii?Q?mOgpd2v7nAnTIkXiimAe225k6lSMXxyvzxI6YjiVEiXylg8YZGV5kBfAwhIg?=
 =?us-ascii?Q?fTnIVFwUIU+tiD9vHdPzRhaFJclqjVFeuSmEpDtvLgJVlFdguH1niUkUMs/G?=
 =?us-ascii?Q?QJ/3/THFxAFtvh8uZuxyXWnqE4w5G2ei6YEFezRqfWG+TBYmFCeV+7Lr9vbK?=
 =?us-ascii?Q?WD1W6ioc2k4k8WO8f5rSsoCKMcY6c9fVSgagTOk8U1maMmb136LJWZbZCtB9?=
 =?us-ascii?Q?r5/5NqMtiSjT409yekA1GN4F+cA5CyY0ooCcO3rWT1DLHKuG9xrXQbNSg/Yh?=
 =?us-ascii?Q?dfC8+pBDn9tyXRQR963STlJU37ql3s4rS6J+nLgoaOff63EO16iuh2R83R30?=
 =?us-ascii?Q?ca03jWOxQiLtBQHVYWO5lBnv8T7KtY7CDcFxNKqcFcufyOyGIKkdbFwmFi4K?=
 =?us-ascii?Q?CKjv1c7y80MhPOiMr+fMXsErfbey7ROYI4UDrsDWlRj1TwdAuZiQC/y52Muz?=
 =?us-ascii?Q?W+ZwUe+dH7Ocrrasih8AaMLK1zKR5WL77QaWspsZdKSg6x/EW99CyVyKO4LJ?=
 =?us-ascii?Q?IjreY2/AKUoXz4cVF+lWPU6jIYDGmc7zdN4WqqRtMFqoLtydh0NxvM9z59rM?=
 =?us-ascii?Q?Bljx/gXRnIoc9H9LY6G+HIys85nnE3OlFFbuO0CqmpJ3uN5z5yJZE2Xxo8FQ?=
 =?us-ascii?Q?ImBZq2iiLJPaYpg3oomn7GQHAKQm7Zfc7jCVU6jw9b7Bj62jRM0KmrYz8qh9?=
 =?us-ascii?Q?nWOpNvdToB7oejb9uJV5Iy+XPZcjltme6yxcdrNN3gbW3XrrMcZPgOj5jM2q?=
 =?us-ascii?Q?5cBr4+Mq4cxhyeGk+htEsk2IEW1LO0xuS0pgGVdqWLSQMGMhWCmfhD2N7mHO?=
 =?us-ascii?Q?zk33O9b+88jynQ75rlx/TDbvN3a/sJu0qSlGXaF/3VpblTp2P+DH7Wc6lr74?=
 =?us-ascii?Q?VYgn0mqo/MCfnXX0muwQ19PVtITrIXHDxVR2K7b8Ro38hkASmXDTWumjrlVL?=
 =?us-ascii?Q?QOzXel8jk5wwr67/eaP7hpsb8v760w5ibNllTinAAzw=3D?=

@IT-Ideas
Copy link
Contributor

IT-Ideas commented May 16, 2024

@jorv-odoo I can confirm that the 403 error is indeed due to sending the messages with "Test" button. When clicking on the unsubscribe on the actual scheduled e-mail, that unsubscribe works.

Yes and the reason behind is quite legit as the access token is not yet generated, which makes totally sense to me as its role is to provide public access, which is to me not suitable until the mailing is actually sent.

@jorv-odoo
Copy link
Contributor

@ossimantylahti thx for confirming.
As stated by TDE, Odoo 17.0+ is actually compliant, we are currently checking if we can easily backport the solution to 15+ to make them compliant as well. Hopefully we can find an adequate solution asap

jorv-odoo added a commit to odoo-dev/odoo that referenced this issue May 16, 2024
Starting from 1st June 2024, most major email provivders will start
enforcing compliance with RFC 8085 dictating easy unsubscription
for marketing emails.

While Odoo 17.0+ is compliant, previous versions did not generate
the relevant email headers for outgoing emails (`List-Unsubscribe` and
`List-Unsubscribe-Post`).

This commit tries to backport the used approach in 17.0 to be
functionally equivalent in versions 15+.

Related github issue odoo#165169
jorv-odoo added a commit to odoo-dev/odoo that referenced this issue May 16, 2024
Starting from 1st June 2024, most major email provivders will start
enforcing compliance with RFC 8085 dictating easy unsubscription
for marketing emails.

While Odoo 17.0+ is compliant, previous versions did not generate
the relevant email headers for outgoing emails (`List-Unsubscribe` and
`List-Unsubscribe-Post`).

This commit tries to backport the used approach in 17.0 to be
functionally equivalent in versions 15+.

Related github issue odoo#165169
jorv-odoo added a commit to odoo-dev/odoo that referenced this issue May 16, 2024
Starting from 1st June 2024, most major email proivders will start
enforcing compliance with RFC 8085 dictating easy unsubscription
for marketing emails.

While Odoo 17.0+ is compliant, previous versions did not generate
the relevant email headers for outgoing emails (`List-Unsubscribe` and
`List-Unsubscribe-Post`).

This commit tries to backport the used approach in 17.0 to be
functionally equivalent in versions 15+.

Related github issue odoo#165169
@ossimantylahti ossimantylahti changed the title [15.0][16.0][17.0] Odoo not following RFC 8058 - mass e-mails will go to spam [15.0][16.0] Odoo not following RFC 8058 - mass e-mails will go to spam May 17, 2024
@IT-Ideas
Copy link
Contributor

@jorv-odoo any update regarding this issue?

Thanks and have a great day!

@jorv-odoo
Copy link
Contributor

@IT-Ideas a RnD commit is on the way. Hopefully it will be merged asap, but there is still some validation and testing to do. I will try to update this thread once things are more definite.

Have a great day!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
15.0 16.0 Marketing CRM, mail, event, livechat, mass mailing, online appointments, ...
Projects
None yet
Development

No branches or pull requests

4 participants