Passcodes | Set up passcodes for registration #2639

coldlink · 2024-03-21T15:02:47Z

What does this change?

We've now upgraded to Okta Identity Engine. This unlocks a number of new features which previously weren't feasible. One of these is "passwordless", or the idea of eliminating, or reducing the need for passwords.

We also currently have an issue within our OAuth flows where users don't always end up signed in the same browser context that they registered from. This can occur if a user clicks a reset password/registration verification link in their on another browser or device, or if deeplinking/https interception isn't working properly on the user's device.

One way of fixing both these issues it to use One Time Passcodes (OTPs), where instead of a user clicking a link in an email, they have to type a 6 digit number into an input field which performs the same action of verifying the user. This means that the user always stays within the same browser device/context, while also securely verifying the user.

To begin with we're going to only implement OTPs for registration only, with the user still having to set a password, this is the focus of this PR. After this we will being rolling out passcodes for sign in and password reset too, thus eliminating passwords altogether, or at lease making them optional.

This PR builds on the work previously done in #2625 where the initial work to set up the IDX API was done, and #2637 where most of the APIs for passcodes were set up. The focus of this PR is to wire up the registration functionality to use the passcode API, and set up the frontend to handle this.

This PR however does not enable passcodes directly in PROD, the usePasscodesRegistration query parameter flag must be manually set in order to opt in, and the switch has to be enabled in PROD, it is currently set to false. This allows us to slowly opt in and first test on the CODE environment, followed by PROD behind the flag, followed by enabling it without the flag.

We're currently only enabling passcodes for new users who are registering. Existing users that are attempting to go through the registration flow will fallback to the legacy/current flow without passcodes for now until full passwordless is implemented. The full flowchart can be seen in #2567.

The Journey

The steps to enable passcode registration are as follows:

User attempts to register with a new email, this calls POST /register
POST /register
1. Checks if registrationPasscodesEnabled feature switch enabled and usePasscodeRegistration query parameter is set (this step is temporary before rolling out to all users)
  1. If it is, attempt passcode registration flow
  2. Otherwise use classic flow
2. Assuming 1) is stratified, first start the interaction code flow, and get the interaction_handle + authState
  1. See Okta IDX API | API setup for passcode registration #2637 and IDX API / Interaction Code Flow | API Setup + Social Sign In #2625 for more details on specifics of what the interaction code flow/IDX API is doing
3. Introspect the interaction_handle to get the stateHandle
4. Encrypt any consents we need to preserve during the registration flow, and update the authState with the stateToken and encryptedRegistrationConsents
5. Check if we have the select-enroll-profile remediation property which means registration is allowed using the IDX API
6. Call the /enroll endpoint to attempt to start the registration process
7. Call the /enroll/new endpoint to attempt to register the user with email
8. If successful, the user will have been send an email with a passcode by Okta
9. Set the encrypted state cookie to persist the email and stateHandle through the flow/between pages
10. Finally redirect to the email sent page
11. If at any point there is an error, we log the error and fallback to the classic registration flow
User shown the Email Sent page, but it has a passcode input field instead! User grabs the 6 digit passcode from their email. They type it in and click "Submit Passcode". This calls POST /register/code
POST /register/code
1. Grab the code from the request body and read the stateHandle from the encrypted state cookie
2. Check if the stateHandle is valid with the introspect endpoint
3. Call the /challenge/answer endpoint with the code and stateHandle
  i. If valid continue
  ii. If invalid, we throw and handle a specific error case for invalid passcode: api.authn.error.PASSCODE_INVALID
4. We need the user to set a password, so we need to enroll them in the "password" factor, find the passwordAuthenticatorId
5. Call the /credentials/enroll method with the authenticator id and methodType set to password
6. Redirect the user to the /welcome/password page where they'll set a password
GET /welcome/password
1. This is intercepted as a GET /welcome/:token route, so we update the existing handler, which uses the checkTokenInOkta from checkPasswordToken.ts
2. Checks if registrationPasscodesEnabled feature switch enabled and usePasscodeRegistration query parameter is set (this step is temporary before rolling out to all users)
  1. If it is, attempt passcode registration flow
  2. Otherwise use classic flow, i.e assume the token parameter is a token
3. If it is, read the encrypted state cookie for the email and state handle
4. Check if the stateHandle is valid with the introspect endpoint and making sure it has the enroll-authenticator object so we know it's in the correct state
5. Show the set password page
User enters a new password and submits, calls POST /welcome/password
1. This is intercepted as a POST /welcome/:token route, so we update the existing handler, which uses the changePasswordInOkta from changePassword.ts
2. Checks if registrationPasscodesEnabled feature switch enabled and usePasscodeRegistration query parameter is set (this step is temporary before rolling out to all users)
  1. If it is, attempt passcode registration flow
  2. Otherwise use classic flow, i.e assume the token parameter is a token
3. If it is, read the encrypted state cookie for the email and state handle
4. Check if the stateHandle is valid with the introspect endpoint and making sure it has the enroll-authenticator object so we know it's in the correct state
5. Validate the password field and check if breached
6. Call the setPasswordAndRedirect function, which uses the /challenge/answer endpoint to set a password. This sets a global okta session, and completes the flow by redirecting the user to the interaction code callback endpoint, but with a code parameter (not an interaction_code) and the state.
In this callback endpoint, use a standard OAuth library that supports Authorization Code Flow with PKCE, and complete the flow. The user is authenticated at this point, and everything else remains the same as what we currently have
At any point if there is an error, we catch the error, log it, and fallback to the existing flow.

Demo

Screen.Recording.2024-05-22.at.15.59.22.mov

Plan

Merge this PR in behind feature flag and switch
Perform testing on CODE apps and web
- The query flag is usePasscodeRegistration=true
Enable switch on PROD
- PR required
Perform testing on PROD behind
Remove feature flag on PROD
- Big PR required

While improving/making refinements.

Tested?

raphaelkabo · 2024-05-24T13:59:20Z

Some questions/notes (I think I've answered the first two of these for myself by reading the code but I wanted to double check because they're edge cases):

As a user, what's the experience if there's a token expiry while I'm on the passcode field? What happens if I wait for ages there, then put in the right passcode? What if it's the wrong passcode?
Is there a situation where a token expiry can occur between the passcode input field and the password input field?
For simplicity, I would just make /welcome/password its own GET and POST routes, and fail with an error if the feature flag and query parameter aren't present. We wouldn't be letting real users use the query parameter on production anyway, and it'll avoid overloading the /welcome/:token routes with logic that doesn't belong in them ('password' can never be a valid token).

raphaelkabo

I'm very happy (and impressed) with this! I'm not approving it since it needs rebase with main anyway, and I've left some comments, but generally speaking I think it's very much ready for a CODE deploy behind a flag.

src/server/lib/okta/idx/challenge.ts

src/server/lib/okta/idx/introspect.ts

raphaelkabo · 2024-05-24T14:19:23Z

src/server/routes/register.ts

+			requestState: mergeRequestState(state, {
+				pageData: {
+					email: readEncryptedStateCookie(req)?.email,
+					timeUntilTokenExpiry: encryptedState.stateHandleExpiresAt


You could be pedantic/avoid confusion and call this msUntilTokenExpiry. Just because some OAuth stuff uses seconds instead.

I did it this way as timeUntilTokenExpiry was already a thing in Gateway, and wanted to avoid a refactor.

hopefully the new function name makes it clearer

raphaelkabo · 2024-05-24T14:21:32Z

src/server/routes/register.ts

+				const passwordAuthenticatorId =
+					challengeAnswerResponse.remediation.value
+						.flatMap((remediation) => {
+							if (remediation.name === 'select-authenticator-enroll') {
+								const parsedRemediation =
+									selectAuthenticationEnrollSchema.safeParse(remediation);
+
+								if (parsedRemediation.success) {
+									return parsedRemediation.data.value.flatMap((value) => {
+										if (value.name === 'authenticator') {
+											return value.options.flatMap((option) => {
+												if (option.label === 'Password') {
+													if (
+														option.value.form.value.some(
+															(v) => v.value === 'password',
+														)
+													) {
+														return [
+															option.value.form.value.find(
+																(v) => v.name === 'id',
+															)?.value,
+														];
+													}
+												}
+											});
+										}
+									});
+								}
+							}
+						})
+						.filter(
+							(id): id is string => typeof id === 'string' && id.length > 0,
+						);


This is terrifying. No suggestions to fix it, it just is. :(

A further thought: is there any way at all that we could leverage zod to handle this complex nested object? It feels like a maintenance nightmare otherwise.

this is leveraging Zod as best I can 😱

it's just annoying at there are so many states to what the IDX API returns, it's not consistent, so we always have to double check everything.

raphaelkabo · 2024-05-24T14:23:13Z

src/server/routes/register.ts

+							(id): id is string => typeof id === 'string' && id.length > 0,
+						);
+
+				if (!passwordAuthenticatorId.length) {


I'm surprised Typescript can promise that passwordAuthenticatorId is set to anything after all those nested flatMaps and ifs! Maybe passwordAuthenticatorId?.length is safer anyway?

if (!passwordAuthenticatorId.length) { is effectively the same as doing if (passwordAuthenticatorId?.length) { in this scenario?

raphaelkabo · 2024-05-24T14:34:03Z

src/client/pages/EmailSent.tsx

+				<>
+					{email ? (
+						<MainBodyText>
+							We’ve sent an email to <b>{email}</b> with verification


This is odd wording ('verification instructions and a verification code') - surely just 'with a verification code' is fine?

I'll speak to @nickhartyyy about this

raphaelkabo · 2024-05-24T14:34:13Z

src/client/pages/EmailSent.tsx

+						<MainBodyText>
+							We’ve sent you an email with verification instructions and a
+							verification code.


Similar odd wording here.

raphaelkabo · 2024-05-24T14:50:27Z

src/server/lib/okta/idx/challenge.ts

+	} else {
+		logger.error(
+			'Failed to set validation flags in Okta as there was no id',
+			undefined,
+			{
+				request_id,
+			},
+		);
+	}


Does anything rely on this? If the email isn't validated successfully, what happens to the short-lived user session next? What happens to any new user sessions? (Basically, are we happy leaving the error only logged?)

one of the cases where it should be theoretically impossible to get to, but helpful if we log an error message

on next sign in they'd be sent the "security email" if they were ever to end up in this scenario

cypress/integration/ete-okta/registration_2.6.cy.ts

coldlink · 2024-05-28T13:19:18Z

Some questions/notes (I think I've answered the first two of these for myself by reading the code but I wanted to double check because they're edge cases):

As a user, what's the experience if there's a token expiry while I'm on the passcode field? What happens if I wait for ages there, then put in the right passcode? What if it's the wrong passcode?

Is there a situation where a token expiry can occur between the passcode input field and the password input field?

For simplicity, I would just make /welcome/password its own GET and POST routes, and fail with an error if the feature flag and query parameter aren't present. We wouldn't be letting real users use the query parameter on production anyway, and it'll avoid overloading the /welcome/:token routes with logic that doesn't belong in them ('password' can never be a valid token).

As a user, what's the experience if there's a token expiry while I'm on the passcode field? What happens if I wait for ages there, then put in the right passcode? What if it's the wrong passcode?
- Everything should be handled correctly, if there is a passcode expiry at any point this would be caught and the user shown a "link expired" page, thanks to the timeUntilTokenExpiry code
- This also works because we introspect the stateHandle on every request to make sure it's still valid, regardless of where the user is within the process.
Is there a situation where a token expiry can occur between the passcode input field and the password input field?
- No, once the passcode is used, the stateHandle is updated to allow for additional time for the password field input (I think up to an hour), the timeUntilTokenExpiry is updated to reflect this
For simplicity, I would just make /welcome/password its own GET and POST routes, and fail with an error if the feature flag and query parameter aren't present. We wouldn't be letting real users use the query parameter on production anyway, and it'll avoid overloading the /welcome/:token routes with logic that doesn't belong in them ('password' can never be a valid token).
- I did it this way for two reasons.
- 1. To make it easier to migrate to passcodes in the future without having to rewrite any routes, and to reuse as much code as possible
- 1. To make it easier to fallback to the existing code if for some reason something goes wrong

…e use passcodes

…arams

…to Encrypted State

…des to register

…for passcode registration

…` flags when registering with passcodes

…d DRY

… passcode expiry

…nality

coldlink force-pushed the mm/idx-api-part-ii branch from 42cbf7b to ee5a834 Compare March 28, 2024 09:22

coldlink force-pushed the mm/passcodes-ete branch from 9e69a50 to add2649 Compare March 28, 2024 13:15

coldlink force-pushed the mm/idx-api-part-ii branch from ee5a834 to 1825c32 Compare March 28, 2024 14:43

coldlink force-pushed the mm/passcodes-ete branch 2 times, most recently from 9693763 to 40201c7 Compare March 28, 2024 17:12

coldlink force-pushed the mm/idx-api-part-ii branch from 1825c32 to f6036aa Compare April 2, 2024 10:14

coldlink force-pushed the mm/passcodes-ete branch from 40201c7 to b35c732 Compare April 2, 2024 10:23

coldlink force-pushed the mm/idx-api-part-ii branch from f6036aa to c195305 Compare April 2, 2024 15:51

coldlink force-pushed the mm/passcodes-ete branch 5 times, most recently from 619b9a9 to 3b6394c Compare April 3, 2024 09:21

raphaelkabo force-pushed the mm/idx-api-part-ii branch from b56e151 to 983a966 Compare April 8, 2024 12:53

Base automatically changed from mm/idx-api-part-ii to main April 8, 2024 13:37

coldlink force-pushed the mm/passcodes-ete branch 4 times, most recently from 09b3a5c to 50de391 Compare May 21, 2024 07:57

coldlink requested review from guardian-ci and removed request for guardian-ci May 21, 2024 12:43

coldlink force-pushed the mm/passcodes-ete branch 2 times, most recently from 2abba14 to 40406c7 Compare May 22, 2024 10:50

coldlink marked this pull request as ready for review May 22, 2024 10:52

coldlink requested a review from a team as a code owner May 22, 2024 10:52

coldlink force-pushed the mm/passcodes-ete branch from 40406c7 to f7caeba Compare May 22, 2024 10:54

coldlink requested review from guardian-ci and removed request for guardian-ci May 22, 2024 10:54

coldlink force-pushed the mm/passcodes-ete branch from f7caeba to ee9e89a Compare May 22, 2024 14:30

coldlink changed the title ~~Set up passcodes for registration~~ Passcodes | Set up passcodes for registration May 22, 2024

coldlink requested review from guardian-ci and removed request for guardian-ci May 22, 2024 15:11

raphaelkabo reviewed May 24, 2024

View reviewed changes

coldlink added 18 commits May 28, 2024 14:44

feat(passcodes): update email sent page with passcde field on cases w…

a7664dc

…e use passcodes

chore(social-sign-in): add request id

1416624

feat(passcodes): add usePasscodeRegistration to persistable query p…

b624125

…arams

chore(metrics): add OktaIDXRegister metric

2afb077

fix(idx): remove coerce expiresAt as date, as this caused some issues

c840f5d

chore(errors): add PASSCODE_INVALID message

9bef4af

chore(encrypted-state): add stateHandle and stateHandleExpiresAt …

cf9eb4f

…to Encrypted State

feat(PageData): add hasStateHandle parameter through request state

ed184f5

feat(passcodes): update registration methods to attempt to use passco…

b38278e

…des to register

feat(passcodes): update the check and change password flows to allow …

0416ff3

…for passcode registration

fix(passcodes): make sure to set `validateEmailAndPasswordSetSecurely…

cc26e96

…` flags when registering with passcodes

feat(passcodes): add resend functionality

63a158a

chore(passcodes): remove 418 call used for testing

ebbdcba

chore(passcodes): add cypress tests for the passcode registration flow

1bc19f6

chore(passcodes): additional code comments

eb31ffc

feat(passcodes): add validateIntrospectRemediation function to avoi…

ae54fae

…d DRY

chore: remove refs to consents after rebasing with main

980c142

refactor(idx): create convertExpiresAtToExpiryTimeInMs function for…

71c074b

… passcode expiry

coldlink force-pushed the mm/passcodes-ete branch from ee9e89a to 71c074b Compare May 28, 2024 13:45

coldlink requested review from a team and removed request for a team May 28, 2024 13:46

feat(passcodes): update setPasswordAndRedirect with missing functio…

5f7796a

…nality

coldlink mentioned this pull request May 29, 2024

Passcodes | Remove usePasscodeRegistration query parameter and make passcode registration default #2752

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Passcodes | Set up passcodes for registration #2639

Passcodes | Set up passcodes for registration #2639

coldlink commented Mar 21, 2024 •

edited

raphaelkabo commented May 24, 2024 •

edited

raphaelkabo left a comment

raphaelkabo May 24, 2024

coldlink May 28, 2024

coldlink May 28, 2024

raphaelkabo May 24, 2024

raphaelkabo May 24, 2024

coldlink May 28, 2024

raphaelkabo May 24, 2024

coldlink May 28, 2024

raphaelkabo May 24, 2024

coldlink May 28, 2024

raphaelkabo May 24, 2024

coldlink May 28, 2024

raphaelkabo May 24, 2024

coldlink May 28, 2024

coldlink commented May 28, 2024 •

edited

Passcodes | Set up passcodes for registration #2639

Are you sure you want to change the base?

Passcodes | Set up passcodes for registration #2639

Conversation

coldlink commented Mar 21, 2024 • edited

What does this change?

The Journey

Demo

Plan

Tested?

raphaelkabo commented May 24, 2024 • edited

raphaelkabo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coldlink commented May 28, 2024 • edited

coldlink commented Mar 21, 2024 •

edited

raphaelkabo commented May 24, 2024 •

edited

coldlink commented May 28, 2024 •

edited