Sending emails using Google Workspace "service account" with domain-wide delegation.

[td234]

I need a little help with my setup. Any help will be greatly appreciated.

I am trying to send emails from my website on behalf of any/all users on my domain in my Workspace account. Some of these are forms and some are cron jobs. Note: This is a "service account" so this does not require individual users to authenticate using a consent screen.

Here's what I have done:

1. I have created the projects in Google Cloud and set up the service account and enabled the API Gmail API. I have created the key and downloaded the JSON file.

2. In Workspace Admin I have given domain wide access to service account's email address with the scope https:/www.googleapis.com/auth/gmail.send

Now I am trying to figure out how to set up phpMailer to use send mails using this information. I have read so many "suggestions" and nothing has worked. I keep getting connection errors.

2024-02-10 15:52:10 SERVER -> CLIENT: 220 smtp.gmail.com ESMTP n5-20020a4a6105000000b0059cc37c8cdcsm767246ooc.46 - gsmtp
2024-02-10 15:52:10 CLIENT -> SERVER: EHLO localhost
2024-02-10 15:52:10 SERVER -> CLIENT: 250-smtp.gmail.com at your service, [216.177.181.65]250-SIZE 35882577250-8BITMIME250-STARTTLS250-ENHANCEDSTATUSCODES250-PIPELINING250-CHUNKING250 SMTPUTF8
2024-02-10 15:52:10 CLIENT -> SERVER: STARTTLS
2024-02-10 15:52:10 SERVER -> CLIENT: 220 2.0.0 Ready to start TLS
2024-02-10 15:52:10 CLIENT -> SERVER: EHLO localhost
2024-02-10 15:52:10 SERVER -> CLIENT: 250-smtp.gmail.com at your service, [216.177.181.65]250-SIZE 35882577250-8BITMIME250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH250-ENHANCEDSTATUSCODES250-PIPELINING250-CHUNKING250 SMTPUTF8
2024-02-10 15:52:10 CLIENT -> SERVER: QUIT
2024-02-10 15:52:10 SERVER -> CLIENT: 221 2.0.0 closing connection n5-20020a4a6105000000b0059cc37c8cdcsm767246ooc.46 - gsmtp

I have tried methods passing the JSON file, getting a token from the file and passing that, and several others.

I would greatly appreciate any input from someone who has gotten this to work using a service account.

[Synchro]

This is not getting as far as attempting to authenticate; something is going wrong prior to that. I suggest you enable low-level debug output, which should show you more info about what's going on. At a guess, I'd say you're selecting an invalid auth mechanism, but I can't tell without seeing your code and then debug detail.

As far as I'm aware, Gmail service accounts use "app passwords", which work just like regular passwords. There isn't a way to pass a JSON file to SMTP login mechanisms.

[decomplexity]

I assume that in your example you are using normal client_credentials flow and not a JWT signed by your client (both are valid).

When your client received the offer of the six authentication methods (LOGIN PLAIN XOAUTH2, etc), it should have responded:

2024-02-10 15:52:10 CLIENT -> SERVER: AUTH XOAUTH2

followed by a base64 access token (see below)
rather than:

2024-02-10 15:52:10 CLIENT -> SERVER: QUIT

Your access token is set up when you initially call Google API, e.g.

$this->provider  = new Client();
$this->provider -> setScopes([Gmail::MAIL_GOOGLE_COM]); // must be set before setAuthConfig
$this->provider -> setAuthConfig(self::GMAIL_XOUTH2_CREDENTIALS);
$this->provider -> useApplicationDefaultCredentials();
$this->provider -> setApplicationName(self::GOOGLEAPI_APPLICATION_NAME);

and returned when you use e.g.

$token_array = $this->provider->fetchAccessTokenWithAssertion();

and then select the token itself from this four-element array with:

$token_array['access_token'];	

GMAIL_XOUTH2_CREDENTIALS (above) is typically your JSON's 'gmail-xoauth2-credentials.json', and mail.google.com is probably the only scope value you need (too ‘wide’ perhaps, but you can always refine it later).

[td234]

Hi.

Thanks for your reply.

Here is what I have. The google/client is working and giving me a token.

			$client = new Google\Client();
			$client->setApplicationName('Admin SMTP Login');
			$client->setScopes(['https://www.googleapis.com/auth/gmail.send']);
			$client->setAuthConfig($serviceAccountPath);
			$client->setSubject($from_email); // User to impersonate

			$mail->AuthType = 'XOAUTH2';
			$mail->SMTPDebug = SMTP::DEBUG_LOWLEVEL;
			$mail->SMTPSecure = PHPMailer::ENCRYPTION_STARTTLS;  
			$mail->Port = 587;

			$accessToken = $client->fetchAccessTokenWithAssertion();

			$mail->oauthUserEmail = $from_email;
			$mail->oauthClientId = $client->getClientId();
			$mail->oauthClientSecret = $client->getClientSecret();
			$mail->oauthRefreshToken = $accessToken['access_token'];

2024-02-15 02:55:29 Connection: opening to smtp.gmail.com:587, timeout=300, options=array()
2024-02-15 02:55:29 Connection: opened
2024-02-15 02:55:29 SMTP INBOUND: "220 smtp.gmail.com ESMTP k13-20020a9d7dcd000000b006e12023de5fsm84648otn.46 - gsmtp"
2024-02-15 02:55:29 SERVER -> CLIENT: 220 smtp.gmail.com ESMTP k13-20020a9d7dcd000000b006e12023de5fsm84648otn.46 - gsmtp
2024-02-15 02:55:29 CLIENT -> SERVER: EHLO localhost
2024-02-15 02:55:30 SMTP INBOUND: "250-smtp.gmail.com at your service, [216.177.181.65]"
2024-02-15 02:55:30 SMTP INBOUND: "250-SIZE 35882577"
2024-02-15 02:55:30 SMTP INBOUND: "250-8BITMIME"
2024-02-15 02:55:30 SMTP INBOUND: "250-STARTTLS"
2024-02-15 02:55:30 SMTP INBOUND: "250-ENHANCEDSTATUSCODES"
2024-02-15 02:55:30 SMTP INBOUND: "250-PIPELINING"
2024-02-15 02:55:30 SMTP INBOUND: "250-CHUNKING"
2024-02-15 02:55:30 SMTP INBOUND: "250 SMTPUTF8"
2024-02-15 02:55:30 SERVER -> CLIENT: 250-smtp.gmail.com at your service, [216.177.181.65]250-SIZE 35882577250-8BITMIME250-STARTTLS250-ENHANCEDSTATUSCODES250-PIPELINING250-CHUNKING250 SMTPUTF8
2024-02-15 02:55:30 CLIENT -> SERVER: STARTTLS
2024-02-15 02:55:30 SMTP INBOUND: "220 2.0.0 Ready to start TLS"
2024-02-15 02:55:30 SERVER -> CLIENT: 220 2.0.0 Ready to start TLS
2024-02-15 02:55:30 CLIENT -> SERVER: EHLO localhost
2024-02-15 02:55:30 SMTP INBOUND: "250-smtp.gmail.com at your service, [216.177.181.65]"
2024-02-15 02:55:30 SMTP INBOUND: "250-SIZE 35882577"
2024-02-15 02:55:30 SMTP INBOUND: "250-8BITMIME"
2024-02-15 02:55:30 SMTP INBOUND: "250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH"
2024-02-15 02:55:30 SMTP INBOUND: "250-ENHANCEDSTATUSCODES"
2024-02-15 02:55:30 SMTP INBOUND: "250-PIPELINING"
2024-02-15 02:55:30 SMTP INBOUND: "250-CHUNKING"
2024-02-15 02:55:30 SMTP INBOUND: "250 SMTPUTF8"
2024-02-15 02:55:30 SERVER -> CLIENT: 250-smtp.gmail.com at your service, [216.177.181.65]250-SIZE 35882577250-8BITMIME250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH250-ENHANCEDSTATUSCODES250-PIPELINING250-CHUNKING250 SMTPUTF8
2024-02-15 02:55:30 Auth method requested: XOAUTH2
2024-02-15 02:55:30 Auth methods available on the server: LOGIN,PLAIN,XOAUTH2,PLAIN-CLIENTTOKEN,OAUTHBEARER,XOAUTH
SMTP Error: Could not authenticate.
2024-02-15 02:55:30 CLIENT -> SERVER: QUIT
2024-02-15 02:55:30 SMTP INBOUND: "221 2.0.0 closing connection k13-20020a9d7dcd000000b006e12023de5fsm84648otn.46 - gsmtp"
2024-02-15 02:55:30 SERVER -> CLIENT: 221 2.0.0 closing connection k13-20020a9d7dcd000000b006e12023de5fsm84648otn.46 - gsmtp
2024-02-15 02:55:30 Connection: closed
SMTP Error: Could not authenticate.
Mailer Error: SMTP Error: Could not authenticate.

[td234]

Okay, solved this. Going to leave this here for anyone else who is looking to make this work.

Using the Gmail API directly with PHPMailer is not the typical approach because PHPMailer is primarily designed for SMTP communication, while the Gmail API uses HTTP requests to interact with Gmail. However, you can still leverage PHPMailer's capabilities for building MIME email messages, which can be useful if you're comfortable with PHPMailer's API for composing emails but want to send them through the Gmail API.

Here's the approach on how you might combine PHPMailer for creating the email message and the Gmail API for sending the message:

Compose the email with PHPMailer: Use PHPMailer to create your MIME message. This is helpful if you want to take advantage of PHPMailer's features, like adding attachments or using HTML templates.

`$mail = new PHPMailer();
$mail->isSMTP(); // Though we are not sending via SMTP, this configures the message format correctly
$mail->CharSet = PHPMailer::CHARSET_UTF8;

$mail->setFrom('from@example.com', 'From Name');
$mail->addAddress('to@example.com', 'To Name');
$mail->Subject = 'Here is the subject';
$mail->Body = 'This is the HTML Message Body in bold!';
$mail->AltBody = 'This is the body in plain text for non-HTML mail clients';

// PHPMailer 6.5.0 introduced getSentMIMEMessage() method to retrieve the MIME message
// For older versions, use the preSend() and getSentMIMEMessage() combination
if (!$mail->preSend()) {
throw new Exception('Mailer Error: ' . $mail->ErrorInfo);
}
$mimeMessage = $mail->getSentMIMEMessage();`

Convert the PHPMailer-generated email to a format suitable for the Gmail API: Extract the MIME string from PHPMailer without actually sending it via SMTP. PHPMailer generates the email as a MIME string, which is exactly what the Gmail API expects. However, the Gmail API requires the message to be base64url encoded.

$encodedMessage = base64_encode($mimeMessage); $encodedMessage = str_replace(['+', '/', '='], ['-', '_', ''], $encodedMessage); // Base64url encoding

Send the email using the Gmail API: Use the Gmail API to send the MIME message.

`$client = new Client();
$client->setAuthConfig('/path/to/service-account.json');
$client->setScopes(Gmail::GMAIL_SEND);
$client->setSubject('user@example.com'); // User to impersonate

$service = new Gmail($client);

$message = new Message();
$message->setRaw($encodedMessage);

try {
$service->users_messages->send('me', $message);
echo 'Message sent successfully';
} catch (Exception $e) {
echo 'An error occurred: ' . $e->getMessage();
}`

Hope this helps someone.

[decomplexity]

Your post above of Thu 15/02/2024 15:20 UTC just beat the one I was composing, but I though it worth posting anyway as your post and (just possibly) mine may help others.

A couple of quick points (I am travelling at the moment):

• assuming that Google are following RFC 6749 4.4.3 for client_credentials grant, even with user impersonation, there should be no need to pass a refresh token, nor should it need a client secret as the json is presenting the token server with a private key, although I would hope that Google API ignored both when sees the json saying type =>‘service account’.

• the usual reasons for authentication failure are:

a. the ‘aud’ claim in the token referring to something other the resource to be used

b. the scope not allowing SMTP sending (which may not be the same thing as when sending direct from the account – it isn’t in the Microsoft world where the scope operands can even determine the ‘aud’!) or being ‘wider’ in permission terms than that registered centrally for the app.

It is also worth noting that using PHPMAiler with Google authorisation code grant and Gmail seems straightforward and works out of the box. I have however experienced a number of oddball problems with service accounts: although obtaining access tokens has never been an issue, Gmail SMTP seems to be a delicate flower when accepting a proffered access token and returning ‘authorised’ status.