I'm going to try a spike of integrating passport.js into our authentication code and hopefully will implement this along the way.

Potential advantages of passport.js:

• test coverage from a library widely used
• adopt passport's modular strategy since we have a variety of oAuth services we may be looking to authenticate with

Similarly, I may find other libraries that would beneficial for these reasons.

@louh correct me if I'm wrong, but I believe if we make this an option, we need to add a sign up page.
These are some implications I'm seeing:

• We'll need to have a sign up form where users can specify a password
• Existing users don't have a password
• Therefore users interested in this option may end up with duplicate accounts, which Auth0 appears to have some settings to mitigate
• However the syncing of the users back to our db of users may get wonky
  • for example, I tested one of the Database connections and was able to create a user via signing up with user and password (first ever!) despite the fact that that email is actually already used by a user in auth0. Furthermore, that new user has not shown up in the streetmix user db yet (though maybe that syncs with Auth0 on some sort of regular basis? I'm not sure)

https://auth0.com/docs/users/user-account-linking

Account linking is not allowed in the free plan. However, it might still be worth implementing anyways. We might have 'duplicate' accounts, but not sure if that is effecting our target users who are behind firewalls, since the whole problem is they don't have an account in the first place.

its old but I saw some inspiration here:
jaredhanson/passport#81 (comment)
but I think its not quite as straightforward for us as we're not using passport.js for the general login stuff either? not really sure, still trying to feel like I have more of a solid understanding of the pieces

I had opened a couple of threads over on the Auth0 support forums to try and gather more information about the issues we're running into. Linking here for additional context. (cc @reefdog, welcome to this issue!)

First post on Auth0 support forums: "Passwordless login breaks for many users on corporate e-mail domains"

The response from their support team:

This may be an issue with the email client opening the link. I have seen that mentioned before, although I can’t find a solution for it here.

The helpful bit is the reference to the "email client", and the users I've talked to all use Microsoft Outlook. So far that may be a common culprit. Unfortunately, they were not helpful about next steps, and we certainly have no control over how organizations implement and deploy their email clients.

I followed up with a post asking about how to migrate from passwordless to another option.

Second post on Auth0 support forums: "Is it possible to migrate from a magic link to a username/password login?"

Hopefully this is useful!

Excellent! Yes, for the record my two opening questions were:

1. Have we roped Auth0 into helping us solve this? (Sounds like yes, though I'll shake this tree again.)
2. Have we explored if other heavy passwordless-login-using apps (like Slack) have run into this, and what their solutions were? (Sounds like yes also, from our off-GH chatter, but I'll sniff around this one too.)

I'm first gonna do some additional due diligence on solving this without totally reworking our auth stack, so we can treat the latter as its own weighted task. Failing that, I'll move on to exploring adding the username/password auth. (I'm assuming, until we run into Big Reasons otherwise, we're sticking with Auth0 as the auth stack, and so will look first at their Universal Login.)

Surfaced from one of those support threads and the Auth0 config: have we already explored/discarded using the emailed code instead of a magic link? Our current theory is that something about corporate email systems (or clients) is pre-loading/expiring the magic link; the code wouldn't have that problem.

It's a less-than-ideal UX since it requires more user interaction than just a click, and is less familiar than a traditional username/password login. But it would require very little config change on the Auth0 side, no change to our login infrastructure, and just (well… "just") a new screen and handler on the Streetmix side.

Seems like a pretty good cost-benefit, though ROI may still be lower than just converting to Universal Login.

Thanks for the overview, and I agree with this assessment. I have no problem making the simpler change and adopting magic codes, although I personally don't like the UX of it either.

If converting to Universal Login doesn't seem too too time-consuming (including the process to transition users into setting passwords), then a little more effort for higher ROI does seem like the right move.

After mulling it over a bit, reading up, and talking to @easherma, I'm no longer convinced the ROI on doing codes is worthwhile. It's fine and would "solve" the immediate problem, but I think we all agree it's a crummier UX than both magic links and traditional U/P.

I surfaced another recent Auth0 community thread with this issue, and it's getting a little more traction: “Link protect” software breaks passwordless auth. Hoping a solution comes out of that.

Theoretically, server-side URL sanitizing/checking would cause this problem, but it's almost quite likely to just be Outlook's Link Preview, which seems like a fantastically poorly-considered feature.

Shortest-term, if/when people write in with this issue, we can recommend they disable Link Preview long enough to receive the email and log in, and then re-enable Link Preview if they want. Not at all the permanent solution, but will at least let them authenticate with their preferred email address.

Okay, so!

With #2329 (essentially rewritten from #2035), we've got a proof of concept of linking our sign-in dialog to the Auth0 Universal Login screen. Right now we're serving up the Classic Experience login screen, but as soon as we resolve the issue with our social provider dev keys, we can flip on the superior New Experience. (Nicer interface, faster-loading, better i18n.)

For the R&D I just added a button to the bottom of the social providers; clicking it goes to the universal login screen:

However, we can do a little better. We can keep our existing sign-in dialog, but when a user enters their email address and clicks "Continue with email", we then go to the universal login screen, with the email pre-filled. If we go this route, we'll want to add explanatory text until users are fully on-boarded and comfortable with this change in authentication strategy:

One slight weirdness to this approach: the social connection logins will be present on the Universal Login screen, too, which feels a little weird. Ideally we'd just offer them here (on the dialog), and then if the user took the email path, they only see email-related functionality. But, I think that's acceptable.

For reference, here's the Universal Login screen itself, in Login mode:

And in Sign Up mode:

Also! We can hint that we want it to prefer Sign Up mode unless it recognizes that the machine has logged in before, in which case it will automatically present Login mode. I think that's a great flag we should enable. Edit: This isn't quite right. It will redirect them back to our callback URL (auto-logging them in) if there's an existing/active Auth0 session. But it won't do what I described, presenting the login screen if we recognize their machine and the sign up screen if we don't. (See this section.)

More interface notes on the above:

• Anybody using the social ID providers can continue clicking the "Continue to…" buttons straight from our sign-in dialog as they do today. That doesn't need to change. (They will also be able to use those buttons from the Universal Login screen if they end up there, but they don't have to.)
• Auth0 supports all our languages except Arabic, so the login panel will be presented in the user's language most of the time.
• We can customize all the strings in the Universal Login screen, though hilariously, we have to make manual calls to their API, as there's no management interface for it. And we'd have to provide our translations for any custom text, of course.

More infra notes on the above:

• Existing users who sign up with the email address they were using for passwordless will continue to use their existing account. No new account is created. 👍🏼 (We should definitely confirm this on staging.)
• As already happens today, social ID accounts and email-using accounts are distinct from each other. We can manually link these into a single user account on the Auth0 side (once we upgrade to a paid plan). However, we don't yet have a way to merge the users in our database.

After some more hashing out, we settled on something way cooler: providing Password login as an additional option without yet removing Magic Link!

Users who want to keep using magic links can do so, but now we've got the password option for those who can't use magic links. To gently push everyone toward passworded accounts, we've made this the default form submit behavior.

Users without a password who end up on the Universal Login flow can just click the Sign Up button on that screen, but we've also got the "sign up here" link in the help text that goes directly to that mode. (Both the login and sign up modes will prefill with whatever email the user entered, if any, but only the form-submit/login version actually performs validation.)

#2329 has been updated accordingly.

Blargh. We can't do this after all. Turns out that the New Universal Login experience… doesn't support Passwordless logins. (How did it pass my testing above? I think I was unintentionally triggering the classic experience under certain conditions. 🤦🏼)

So, our options:

1. Keep the new Universal Login experience: Gain the nicer UI, internationalization, and ability to link directly to Sign Up tab. Lose passwordless.
   • 
2. Keep the classic Universal Login experience: Keep the ability to offer passwordless. Lose the ability to link directly to Sign Up, plus other New Experience niceties.
   • 

We've decided on the new Universal Login flow / ditch magic link path. Implementing now.

I wrote this in Discord, should've written it here, copying over for posterity:

I discovered a hangup that should have us holding off on merging/deploying #2329 for now.

Essentially, automatic user-linking works already because of a rule y'all already had, but, as currently written it only works when the email address is already verified. This happens automatically for passwordless and for social connections that provide the user's email (Google definitely, unsure about FB, Twitter definitely not), but it doesn't happen automatically for u/p sign-ups; the user has to verify their email manually after signing up.

So the current experience for someone with an existing account (passworded or social-that-provides-email) who signs up for u/p is that they don't get linked on the Auth0 side, which triggers the User.create flow on our side, which fails because the email isn't unique in our database.

If I ignore that, finish verifying the email, and then log in again, the u/p identity at Auth0 gets merged into the previous one. So it all works!

But we do have to figure out whether to disable that "only merge accounts whose email address is verified" guard (which is in the rule's JS code, so it's technically simple to do), or recognize on our side that the email isn't verified and rather than try to User.create and all that, tell them "go check your email".

The good news is, I understand the problem and the routes to solving it.

So, as @louh surmised, removing the email_verified guard is a no-go, as it lets just anyone create a new username/password account and link it to an existing account that uses the same email (through passwordless or Facebook/Google, which both provide the user's email address) without proving they are the actual owner of the email.

That means we need to build a guard into our auth process that recognizes if the email is unverified and prevents authentication until the user verifies their email.

I've got most of the hooks for this written, and will commit it when I'm finished with some testing.

Latest update. 😮‍💨

How does an email address get verified?

• Explicitly: Auth0 emails the user a "verify email" link and the user clicks it.
• Implicitly: The user logs in with a social account that provides their email address during the authentication handshake. Google and Facebook do this, Twitter does not.
• Implicitly: The user chooses a Magic Link passwordless authentication option, Auth0 sends them a "click here to log in" link, and they click it. (This method is being removed, but a large number of existing users have this kind of account.)

What accounts do we want to merge?

• If accounts share the same email address, and the email address is verified on all the accounts, we merge. (This is current behavior.)
• If accounts share the same email address, and the newest account (looking at created_at) has verified that email address, we merge. (This will be new behavior.)
  • Use case: I sign up with my email address, but don't verify my email. Later, I authenticate using a social account linked to that same email account. Since that second one has a verified email address (see the Implicit verification flows above), then this newer account is de facto verification of the earlier account.

We will not merge accounts where the unverified email address is newer than the verified, or of course when none are verified (though this latter case has no user story I can think of).

What can people with unverified emails do in Streetmix?

• If you sign up with an unverified email we've never seen before, then there's no merge logic (yet), and we'll let you into Streetmix with a periodic prod to please verify your email.
  • This is because there's no risk of you hijacking an existing account, and we want to minimize friction between you and using Streetmix.
• If you sign up with an unverified email that we have seen before (via passwordless or social login), then we'll block you from using Streetmix as that new account until you manually verify the email.
  • This is so you can't hijack an existing Streetmix account just by entering an email linked to an existing account.

How does this translate to actual tasks?

Over in #2329, I'm blocking people with unverified emails from fully authenticating, instead telling them to go verify their emails. Still to do:

• Wire up a Resend Verification Email button
• Improve the Auth0 merge logic to also merge older unverified accounts with newer verified accounts
• Enforce a Streetmix-side "you must verify your email first" block on users who come over from Auth0 with an unverified email that matches a local Streetmix account with a different User ID
• Throw "please verify!" toasts at users who come over from Auth0 with an unverified email that matches a local Streetmix account with the same User ID