Failed to send longer email

[rslenshub]

I am using phpmailer to send email via SMTP.
It was working fine for the first year.
Suddenly one day it has started to failed, after going through some test, i found out that it is only failing on longer content email.
After i restarted the php server it is able to send out the longer content email again until around a month later, it is started to failed again but the problem goes away after the server restarted.
Until recently, the server need to restart every week in order for the email to send out without failure.
I have tried to upgrade the phpmailer to version 6.5.2. The problem still persist.
The exact same email will be able to send out after restarting the php server.
I tried to point to different SMTP service, it seems to have the same behavior.
I can see from the debug log that after sending "." to indicate the end of data sending, phpmailer wait for around 20 seconds and error out. This behavior stayed the same regardless of which version of phpmailer i use.

Now, i have to restart the server every 3 days in order for the longer email to be able to send out.

Please let me know if more information is needed to check what is the issue.

Code to reproduce

$phpmailer = new PHPMailer();
$phpmailer->IsSMTP();
$phpmailer->IsHTML(true);
$phpmailer->BounceEmail = 'bounce@' . $MAILER_DOMAIN;
$phpmailer->addReplyTo('donotreply@' . $MAILER_DOMAIN, 'DoNotReply');
$phpmailer->Sender = $sender;
$phpmailer->setFrom($sender , $sender_name);
$phpmailer->AddCustomHeader("Return-Path: admin@" . $MAILER_DOMAIN);
//$this->Helo = $MAILER_DOMAIN;
$phpmailer->SMTPAuth = true;
$phpmailer->Host = $host;
$phpmailer->Port = $port;
$phpmailer->Username = $user;
$phpmailer->Password = $pass;
$phpmailer->Subject = $subject;
$phpmailer->msgHTML($body);
$phpmailer->SMTPDebug = 2;
foreach($to AS $email_addr){
	$phpmailer->AddAddress($email_addr);
}
$ok = $phpmailer->Send();
var_dump($ok);

Debug output

Error case:

----AFTER CONNECTION OK----

2022-01-06 02:01:17 SMTP INBOUND: "250 2.1.5 Recipient OK"
2022-01-06 02:01:17 SERVER -> CLIENT: 250 2.1.5 Recipient OK
2022-01-06 02:01:17 CLIENT -> SERVER: DATA
2022-01-06 02:01:17 SMTP INBOUND: "354 Start mail input; end with ."
2022-01-06 02:01:17 SERVER -> CLIENT: 354 Start mail input; end with .
2022-01-06 02:01:17 CLIENT -> SERVER: Date: Thu, 6 Jan 2022 02:01:15 +0000

----SENDING VERY LONG EMAIL CONTENT OVER HERE-----

2022-01-06 02:01:17 CLIENT -> SERVER:
2022-01-06 02:01:17 CLIENT -> SERVER: --b1_sxphHvbLKDJudEnI72ShDxLhkjvJrSSsE3bKsssi42Q--
2022-01-06 02:01:17 CLIENT -> SERVER:
2022-01-06 02:01:17 CLIENT -> SERVER: .
2022-01-06 02:01:36 SMTP INBOUND: ""
2022-01-06 02:01:36 SERVER -> CLIENT:
2022-01-06 02:01:36 SMTP ERROR: DATA END command failed:
SMTP Error: data not accepted.
2022-01-06 02:01:36 SMTP NOTICE: EOF caught while checking if connected
2022-01-06 02:01:36 Connection: closed

string(72) "SMTP Error: data not accepted.SMTP server error: DATA END command failed"

Success Case

When sending out shorter messages it will be successful. Successful sending looks like below:
2022-01-06 01:04:28 CLIENT -> SERVER:
2022-01-06 01:04:28 CLIENT -> SERVER: --b1_D2lAxpFGaljjVZZbLe38evPd0mIlkCBLnKxJDNsw--
2022-01-06 01:04:28 CLIENT -> SERVER:
2022-01-06 01:04:28 CLIENT -> SERVER: .
2022-01-06 01:04:28 SMTP INBOUND: "250 2.6.0 D2lAxpFGaljjVZZbLe38evPd0mIlkCBLnKxJDNsw@XXXXXX [InternalId=996432413492, Hostname=EXG16MAIL11.XXXX.XXXX.com] 1867 bytes in 0.429, 4.246 KB/sec Queued mail for delivery"
2022-01-06 01:04:28 SERVER -> CLIENT: 250 2.6.0 D2lAxpFGaljjVZZbLe38evPd0mIlkCBLnKxJDNsw@XXXXXX [InternalId=996432413492, Hostname=EXG16MAIL11.XXXX.XXXX.com] 1867 bytes in 0.429, 4.246 KB/sec Queued mail for delivery
2022-01-06 01:04:28 CLIENT -> SERVER: QUIT
2022-01-06 01:04:28 SMTP INBOUND: "221 2.0.0 Service closing transmission channel"
2022-01-06 01:04:28 SERVER -> CLIENT: 221 2.0.0 Service closing transmission channel
2022-01-06 01:04:28 Connection: closed

[Synchro]

The post-DATA delay is entirely down to the server; the timeout from the RFC on that is 10 minutes. This point is often where a receiving server will perform SPF and DKIM checks, virus scanning, etc, so it's often slow. If the server drops the connection or returns an invalid response (as in the empty response you can see in your transcript), then there's nothing PHPMailer can do except report it as an error, as it's doing.

[rslenshub]

Thanks Synchro for your reply. A few syndrome i notice.

After restarted, it seems to release the threshold and the email is able to send out again, but the threshold seems to be filling up by each email sending out.
The whole process from connection initiated until connection close is around 20 seconds for the failure case.
Mainly spent at this few lines, after issuing the "." for ending the DATA send, there are around 20 seconds before server replying an empty reply. All the others process including connection and sending contents to server took less than 1 second to complete.
2022-01-06 02:01:17 CLIENT -> SERVER: .
2022-01-06 02:01:36 SMTP INBOUND: ""
2022-01-06 02:01:36 SERVER -> CLIENT:

I will try to use the telnet command and send the email manually when phpmailer is failing to see if it can go through. Will update once the test done. Do let me know if there is any others suggestion that is able to identify or help on the issue.

Thanks

[rslenshub]

Done the test using telnet. Putting the content output from phpmailer on those CLIENT -> SERVER: lines.
It is able to send out the long email successfully. Actually not that long, total is 18737 bytes in 36.642, 0.499KB/sec. Queued mail for delivery.
So only have issue sending via php.

The server is running on Microsoft Windows Server 2012 (64-bit) using PHP Version 7.4.24 and Apache 2.4.51

[rslenshub]

Hi Synchro, thanks a lot for your helps and advice. I have finally able to resolved the issue.
I am using wireshark to trace the network flow between my application server and the SMTP server and found out that it is because of the IP-Sec encryption between the 2 server that is causing the issue. By limiting the MTU to 1400, now all email able to send out successfully. There is nothing wrong with the phpmailer package. Thanks again for all your helps.

I have put in the details of my tracing and resolving steps as reference in case anyone else that is encountering the same issue.
https://tech--about.blogspot.com/2022/02/smtp-mail-sending-failure-connection.html

Thanks again for all your helps and finally i am having a sounds sleep now.

[Synchro]

Well done, sounds like that was tricky to track down, and thanks for posting your solution.