MSFT forcibly expires refresh tokens

[decomplexity]

When azure.php acquires an access token, the MSFT endpoint can optionally also return a new refresh token (RFC 6749 10.4). What please is the easiest way to access (in order to subsequently store) this new refresh token?

I ask because in January 2021 MSFT changed the rules for refresh token lifetimes, and there is now a maximum lifetime (90 days). Service applications such as PHPMailer that are not logged into by a user but only at the outset by admin are thus forced to obtain and store a new refresh token (provided along with each new access token) at each invocation, but were not written to do this, probably because Gmail does not life-expire refresh tokens and MSFT itself hitherto had more relaxed rules.
We are currently forced to run an offline ‘authorization code flow’ every 89 days or so and ‘plug in’ the new refresh tokens. We could, I guess, run a full authorization code flow online at the conclusion of each PHPMailer invocation, but this seems wasteful if the new refresh token is available.

[hajekj]

If a refresh token has been invalidated, you need to do a full new authentication cycle - eg. interactive or something. I suggest using client_credentials flow (application identity) - https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow. You can also scope the permissions of the app in Exchange to limit the send permission to only specific mailboxes: https://docs.microsoft.com/en-us/graph/auth-limit-mailbox-access

[decomplexity]

Agreed, Jan.
I am trying, however, to address the situation when the current refresh token is still valid but want to obtain a fresh one each time I acquire an access token (per RFC 6749 10.4) using thephpleague/oauth2-client and your oauth2-azure as provider. I have tried to follow the flow of theleague/oauth2-client to pinpoint where and how it uses oauth2-azure to obtain an access token (and then, with luck, for me to add a request for a new refresh token also) but have failed!

[hajekj]

I am not really sure right now, and will have to look into the RFC and docs, but afaik, providing a new refresh token is solely at the discretion of the authorization server. Maybe I misunderstood it, so please correct me if I am wrong.

[decomplexity]

Tnx Jan.

In https://docs.microsoft.com/en-us/azure/active-directory/develop/refresh-tokens
MSFT says:

“Refresh tokens replace themselves with a fresh token upon every use. The Microsoft identity platform doesn't revoke old refresh tokens when used to fetch new access tokens. Securely delete the old refresh token after acquiring a new one. “

It would be interesting to find how application frameworks such as Laravel handle refreshing a (currently valid) MSFT refresh token: whether they log an expiration date for each token and shortly before that use an ‘authorization flow’ (like yours) to obtain a new one, or – at least for MSFT endpoints – get and store a new one each time they request an access token.

[hajekj]

In my opinion, whenever you get a new refresh_token, you should just delete the previous one, that way, you can partially avoid it expiring. Generally, there are following scenarios:

1. access_token expires and application refreshes it for user within the refresh_token expiration window (default 90 days), you will get a new refresh_token, which you then persist instead of the old one.
2. Both access_token and refresh_token expire (for example, user didn't log into the app for over 90 days), in that case, you will get an error when attempting to perform a refresh and you will have to do the interactive authorization flow again.
3. refresh_token has been revoked (user reset their password, blocked in tenant etc.) - refresh attempt will throw an error and you should attempt to do interactive authorization flow.

So whenever you do a successful refresh, you should throwaway your old refresh_token and replace it with a new one. I just the League's underlying code and it sould replace the old refresh_token with a new one. I would have to test to see if it works for real tho.

Either way, I still think, that you shouldn't use interactive authentication for PHPMailer etc. and instead stick to client_credentials grant as mentioned previously because you will mostlikely run into issues with interactive auth being required at some points (eg. conditional access policy change).

I can however try to make some test scenario to see if the refresh_token gets replaced when replaced correctly.

[decomplexity]

Thanks for taking the time to find the relevant line in thephpleague/oauth2-client’s Token/AccessToken. I had seen this line when scanning the code, but was unclear whether this was retrieving a new refresh token along with the access token or retrieving the existing refresh token (which would then be used then to acquire an access token).
I have now tested it and it does retrieve a new working refresh token.
Thank you.

Since $refreshToken (in oauth2-client’s Token/AccessToken) is defined as a protected string, is the easiest way to access the value – without changing the code – to create e.g. a getNew RefreshToken subclass that extends AccessToken ?

Re revoking or deleting a superseded refresh token, when I first cut code for using OAuth2 for PHPMailer (for both MSFT and Google), there wasn’t an easy ‘revoke’ API call, e.g. a MSFT revocation needed Powershell. So when later I wrote the decomplexity/SendOauth2 wrapper for PHPMailer, which naturally needs only a single ‘client’ refresh token, I stored the token centrally along clientId, clientSecret et al in one file that services the umpteen points in a website that want to send mail. So, given a new refresh token, replacing (i.e. deleting but not revoking) the old one it with the wrapper is simple.

This usage does, as you say, properly need a client credentials grant rather than an interactive authorization code grant. The only reason I currently use the latter is that it is the way OAuth2 has been implemented in PHPMailer. But your point about MSFT Azure security policy changes forcing manual reauthorization is well made, not least because MSFT themselves have a nasty habit of making unilateral changes at the user principal or tenant level.
I will revisit to see if I can force a client credentials grant without amending PHPMailer code.

[hajekj]

Also, I wasn't able to find the original article, but back in 2016, Vittorio wrote a blog bost about why acessing a refresh token directly was a bad practice and why they disabled access to it from ADAL back then. I will see if I can find it when I get to my PC.

[decomplexity]

Thanks - would be interesting to read. I guess it depends on what accessing directly meant. Refresh tokens clearly need to be moved and stored securely, and the decomplexity wrapper has optional encryption/decryption hook-points for the configuration file. Perhaps Vittorio's post pre-dated MSFT's imposition of the 90-day blanket revocation.

[hajekj]

I was able to find the article via Wayback Machine - https://web.archive.org/web/20160404074010/http://www.cloudidentity.com/blog/2015/08/13/adal-3-didnt-return-refresh-tokens-for-5-months-and-nobody-noticed/

[decomplexity]

An interesting read, especially the further links (vide 'The New Token Cache in ADAL v2') and the ensuing comments. (By ‘ADAL V2’, does Vittorio means ADAL accessing a V2 endpoint as well as a V1 one (?), or was this a stop-gap release before the announcement of MSAL). Since ADAL is deprecated anyway, it is unclear how much applies to the latest MSAL and MSFT Graph and to those doing ‘native’ calls rather than going via ADAL or MSAL in a .NET environment. But I had never even thought of MSAL caching!

[hajekj]

I think it refers to V1 endpoint, since V2 wasn't available at that time. I generally wanted to turn this into MSAL for PHP, but I figured I just wouldn't be able to keep up the pace with Microsoft's improvements and changes. But I have a few ideas on implementing the token cache, right now, I just use $_SESSION for it. There were some implementations back then and attempts by official MS vendors (https://github.com/infoxchange/oidc-aad-php-library - for Moodle), but it's pretty out of date as well.

[decomplexity]

Re the line in TheLeague’s \AccessToken you kindly pointed me to and which does give a new refresh token, attempts to read protected $this->refreshToken within e.g. a child class that extends \AccessToken or in other ways I've tried give $this->refreshToken as null. There has to be an easier way!

[decomplexity]

We have done a BAD thing: we have had to add a line to our copies of TheLeague’s \AccessToken (which support our production PHPMailer wrappers) that writes the new refresh token to $_SESSION; this is then picked up elsewhere in the wrapper.
The new tokens are now stored and used automatically, and the old tokens are deleted but not revoked (MSFT will revoke them within 90 days max).
If you know of any way to copy the token from TheLeague’s \AccessToken (or elsewhere) without necessitating such a code change, I would be grateful as I am reluctant to publish the updated wrapper on Github and Packagist when it needs a change to TheLeague’s code - albeit only one line.

[decomplexity]

Hi Jan - with 'refreshed' refresh tokens now working fine with with authorization_code grants (with the one line addition mentioned above), I mentioned in an earlier post that "I will revisit to see if I can force a client credentials grant without amending PHPMailer code".
This has proved impossible, so as an experiment I rewrote PHPMailer's Oauth2 interface to accept client_credentials as well as authorization_code grants. This produces an access token successfully but the token was bounced with an authorization error by Exchange Online.
Since client_credentials grant support was announced by MSFT in early June, this seemed odd, possibly an SMTP.Send scope error since permissions have to come from the API without scope overrides from the client. However, reading MSFT's announcement more closely, client_credentials grant is supported only for POP (!!) and IMAP. I queried this with Greg Taylor of the Exchange Team who confirmed that client_credentials grant for SMTP AUTH was still work-in-progress.