Page MenuHomePhabricator

[keyserver] Fix sending a user cookie when olm session creation failed
ClosedPublic

Authored by inka on Apr 25 2024, 8:09 AM.
Tags
None
Referenced Files
F3385860: D11780.id39490.diff
Fri, Nov 29, 2:41 AM
F3385847: D11780.id39845.diff
Fri, Nov 29, 2:36 AM
F3385843: D11780.id39683.diff
Fri, Nov 29, 2:35 AM
F3385799: D11780.id39843.diff
Fri, Nov 29, 2:20 AM
F3385451: D11780.diff
Fri, Nov 29, 12:25 AM
F3382759: D11780.id39889.diff
Thu, Nov 28, 11:53 AM
F3382466: D11780.id39530.diff
Thu, Nov 28, 10:56 AM
Unknown Object (File)
Sun, Nov 24, 9:51 PM
Subscribers

Details

Summary

issue: ENG-7247
It used to be possible to have a user cookie even though the notif olm session creation failed. This was because the cookie was created before the session, so even though an error was thrown, the user cookie would be sent with it.

We want to create the user cookie once both olm sessions are created. The only thing I really did here is moving code around so that the cookie creation is called after both session are created

Test Plan

keyserverAuthResponder:
Tested that if an error if thrown from olmNotifSession, the user doesn't get a user cookie and is no longer logged in after refreshing the app
Tested this for both users how were already registered with the keyserver, and users who weren't
Tested that it is possible to login if olm sessions are created correctly
logInResponder:
Tested that it is possible to login if everything works. Tested that if an error if thrown from olmNotifSession, the user doesn't get a user cookie and is not logged in after refreshing the app
siweAuthResponder:
Tested registering. I am facing a problem, but I am also facing it on master. Please see ENG-8055. But same as on master, the user gets registered with the keyserver, just the UI hangs.
Tested logging in with SIWE auth - this worked without problems

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Address review - update comments and inline processOLMAccountCreation

inka planned changes to this revision.Apr 30 2024, 1:48 AM

This solution has a problem - the policies are not marked as accepted until processAccountCreationCommon. So for new users processSuccessfulLogin doesn't set a session to avoid sending user all the data before policies are accepted. But this results in errors when creating threads in processAccountCreationCommon -> createUpdates, because a session is missing.

Marking policies as accepted requires a cookie (we check if viewer.loggedIn).

Move marking policies accepted out of processAccountCreationCommon, before fetchNotAcknowledgedPolicies in processSuccessfulLogin

This revision is now accepted and ready to land.Apr 30 2024, 2:27 AM
inka requested review of this revision.Apr 30 2024, 2:27 AM

Hmm. Thanks for catching this error case and fixing up the diff, but I'm not sure I understand the explanation.

It sounds like you're suggesting that the following order is necessary:

  1. First, we have to call setNewCookie
  2. Then we can mark the policies as accepted, which requires viewer.loggedIn (changed as a result of setNewCookie)
  3. Then we can call setNewSession. This needs to come after marking the policies as accepted, in order to "avoid sending user all the data before policies are accepted" (not sure I understand this part very well)
  4. Finally, we can call processAccountCreationCommon, which requires a session since it calls createUpdates

However, in createAccount, we appear to call setNewSession right after setNewCookie, and before we call viewerAcknowledgmentUpdater. Based on your explanation, it seems like that logic is broken?

This diff is very confusing. It's doing way too many things at once, and the test plan is insufficient to cover all of the code that's being modified.

I would love if you could decompose this into several smaller diffs. If you feel strongly that you don't have time for that, then at the very least the test plan needs to be extended to cover all of the responders that you're modifying, including the legacy ones, and including testing siweAuthResponder and keyserverAuthResponder for both an account creation and a login

keyserver/src/responders/user-responders.js
298 ↗(On Diff #39683)

Why do we have an any type here? This should be typed better. Can you try this?

{
  +deviceTokenUpdateRequest?: ?DeviceTokenUpdateRequest,
  +platformDetails: PlatformDetails,
  +watchedIDs: $ReadOnlyArray<string>,
  ...
}

It would probably be better to actually just insert these three params directly into the ProcessSuccessfulLoginParams list

305 ↗(On Diff #39683)

We should be marking policies as accepted on registration, but not on login. I think you've made it too easy for somebody to make a mistake here. This parameter isn't named in a way where it's clear that it should be specified only for registration. See my shouldMarkPoliciesAsAcceptedAfterCookieCreation suggestion below

797–799 ↗(On Diff #39683)

Indentation is broken here

802 ↗(On Diff #39683)
  1. Why are we passing this in as a function? I think it would be more clear what's going on if we pass in shouldMarkPoliciesAsAcceptedAfterCookieCreation: boolean, and it also avoids some level of misdirection (it's confusing to see viewerAcknowledgmentUpdater called here when it's really being called in processSuccessfulLogin)
  2. It appears that you're always passing in markPoliciesAccepted here, but it should only be called on registration, not on login
ashoat requested changes to this revision.Apr 30 2024, 6:51 AM
This revision now requires changes to proceed.Apr 30 2024, 6:51 AM
keyserver/src/responders/user-responders.js
802 ↗(On Diff #39683)

Oops, I was wrong about point 2 (got confused by the broken indentation)

I spoke to @inka in our 1:1 today and it appears that some information was lost in communication between @inka and @kamil. In consultation with @inka, we determined that the order should be the following:

  1. First, we have to call setNewCookie
  2. Then, in the case of a registration, we should mark the policies as accepted. This can happen before setNewSession
  3. Then, in all cases, we should check if the policies are accepted. For registration we can assume they are, but for login we need to query the database. This should happen before setNewSession
  4. Then we call setNewSession, but only if the policies are accepted
  5. Finally, we can call processAccountCreationCommon, which requires a session since it calls createUpdates

We then went one-by-one through every single responder that has been modified in this diff, and analyzed whether the order is correct:

  1. createAccount – no changes have currently been made here, but I suggest a change below. If we make that change, I think we should update the test plan to test this responder.
  2. logInResponder – two issues described inline. Given the amount of changes, I think we should update the test plan to test this responder.
  3. siweAuthResponder – several changes, and I suggest more below. Given the amount of changes, I think we should update the test plan to test this responder. We should test this responder for both login and registration.
  4. keyserverAuthResponder – this one is obviously the one with the most changes, and it's already covered in the test plan.

Finally, a note about splitting up this diff. At this point I've spent a couple hours reviewing it, so I feel like I understand it, and I'm not sure how much would be gained by splitting it up. I feel strongly that it would have been easier to review if it had been split up earlier, but I'll leave it up to @inka to decide whether to split it up at this point, as it might be more work than is worth it at this point.

keyserver/src/creators/account-creator.js
129 ↗(On Diff #39683)

Don't we have the same problem here that you solved for processSuccessfulLogin? It seems like the notif session creation needs to be attempted before we do any writes to the database, so that if it fails we don't end up in an inconsistent state.

251 ↗(On Diff #39683)

I think it's confusing that siweAuthResponder calls processSuccessfulLogin, which has been modified to support handling this viewerAcknowledgmentUpdater call... but instead we handle it in a custom way here.

I think if we go with my shouldMarkPoliciesAsAcceptedAfterCookieCreation suggestion below, it should be clear that this corresponds to registration. So it makes sense for siweAuthResponder to set that in the registration case.

In general, I think it's better to have a consistent mechanisms for these things. Unifying codepaths makes it easier for the reader to understand how things work, and that makes it easier to update the code without making mistakes.

keyserver/src/responders/user-responders.js
340 ↗(On Diff #39683)

This async IIFE isn't doing anything anymore, and should have been removed. But this comment can be ignored because my next comment below suggests assigning the promise

340–358 ↗(On Diff #39683)

Discussed this with @inka in our 1:1 and I suggested to replace these lines with the following for two reasons:

  1. Slightly improves performance in the case where markPoliciesAccepted does not need to be called, by avoiding blocking the fetchNotAcknowledgedPolicies call on the cookie creation
  2. Makes more explicit the requirement that the cookie is created before markPoliciesAccepted is called, and that fetchNotAcknowledgedPolicies should be called after markPoliciesAccepted
  3. Lets us skip fetchNotAcknowledgedPolicies if we just marked the policies as acknowledged
const setNewCookiePromise = (async () => {
  const [userViewerData] = await Promise.all([
    createNewUserCookie(userID, {
      platformDetails: request.platformDetails,
      deviceToken,
      socialProof,
      signedIdentityKeysBlob,
    }),
    deleteCookie(viewer.cookieID),
  ]);
  viewer.setNewCookie(userViewerData);
})();
const policiesCheckAndUpdate = (async () => {
  if (markPoliciesAccepted) {
    await setNewCookiePromise;
    await markPoliciesAccepted(viewer);
    return [];
  }
  return await fetchNotAcknowledgedPolicies(
    userID,
    baseLegalPolicies,
  );
})();
const [notAcknowledgedPolicies] = await Promise.all([
  policiesCheckAndUpdate,
  setNewCookiePromise,
]);

Two quick notes on this logic:

  1. Note that this logic will have to be updated to reflect my shouldMarkPoliciesAsAcceptedAfterCookieCreation suggestion below.
  2. It would be good to make the relationship between baseLegalPolicies (passed to fetchNotAcknowledgedPolicies) and policyTypes.tosAndPrivacyPolicy (passed to viewerAcknowledgmentUpdater) more explicit. If somebody were to add another policy to baseLegalPolicies, it seems like this logic would break: we would be assuming it's okay to proceed because tosAndPrivacyPolicy has been acknowledged, but we're not checking about other policies
381 ↗(On Diff #39683)

This check used to happen earlier in some cases, such as logInResponder.

  1. Now the check doesn't even happen if the policies haven't been acknowledged
  2. If the check happens and fails, it can leave us in a weird state where a user has a cookie but no session

I initially thought we could just move this call to the start of processSuccessfulLogin, but then it presents a different risk: if called during registration, we might see a situation where a user is added to the users table, but then a crash occurs when verifyCalendarQueryThreadIDs is called. On subsequent calls we would think that the user is already created, and as a result the policies would never be set as acknowledged.

Instead, it seems like we will need to move this check to occur earlier, and probably it will need to be called separately for each responder before any writes occur.

688–696 ↗(On Diff #39683)

processSIWEAccountCreation handles cookie creation when siweAuthResponder is called for registration. But in that case, we don't seem to be doing anything to prevent processSuccessfulLogin from creating the cookie as well.

As a result, it looks like registration via siweAuthResponder results in two cookies being created. This issue was originally tracked in ENG-2767, but it looks like it was never fully fixed up. (cc @varun)

Prior to your changes, we could have passed cookieHasBeenSet to processSuccessfulLogin here to disable cookie creation. But now it seems like the best approach would be to do the same thing we did for processOLMAccountCreation:

  1. Get rid of processSIWEAccountCreation
  2. Inline the parts of processSIWEAccountCreation that handle creating the row in the users table
  3. Rely on processSuccessfulLogin to handle the rest (cookie creation and policy stuff)
  4. Have siweAuthResponder call createAndSendReservedUsernameMessage directly (only for registration)

Address review pt. 1:

  • use shouldMarkPoliciesAsAcceptedAfterCookieCreation flag
  • fix indentation
keyserver/src/responders/user-responders.js
298 ↗(On Diff #39683)

Addressed in a separate diff: D11895

inka planned changes to this revision.May 6 2024, 5:24 AM

Marking as changes planned this since I am still working on the rest of the review

keyserver/src/creators/account-creator.js
129 ↗(On Diff #39683)

Created a separate task to track, since this isn't directly related to the issue I am solving, which is It's possible to be logged in and not have a CSAT (ENG-7247): ENG-8046
Also this can be easily fixed in a separate diff, so this will help not make this one any larger

251 ↗(On Diff #39683)

Created ENG-8047 to track, since it can be done in a separate diff

keyserver/src/responders/user-responders.js
340–358 ↗(On Diff #39683)

I think the logic would be correct, in the sense the user is only asked to accept the TERMS_OF_USE_AND_PRIVACY_POLICY by the current UI, so this is the only policy which we should mark as accepted.
So I cannot mark any other policies as accepted. On the other hand, we seem to require that all baseLegalPolicies are accepted to proceed. This is probably an inconsistency that should be addressed, but I think this is outside of the scope of this task.
Created ENG-8049 to track. Please let me know if you think I should take care of this ASAP, but I don't think this is in scope of this issue

keyserver/src/responders/user-responders.js
340–358 ↗(On Diff #39683)

My comment is addressing 2.:

It would be good to make the relationship between baseLegalPolicies (passed to fetchNotAcknowledgedPolicies) and policyTypes.tosAndPrivacyPolicy (passed to viewerAcknowledgmentUpdater) more explicit. If somebody were to add another policy to baseLegalPolicies, it seems like this logic would break: we would be assuming it's okay to proceed because tosAndPrivacyPolicy has been acknowledged, but we're not checking about other policies

Optimize set new cookie - policies check and update relationship

inka planned changes to this revision.May 6 2024, 6:14 AM
inka added inline comments.
keyserver/src/responders/user-responders.js
361–367 ↗(On Diff #39843)

I skipped

return [];

because as explained here, the UI only requests that the user accepts policyTypes.tosAndPrivacyPolicy during registration.

Restore previous way of calling verifyCalendarQueryThreadIDs

keyserver/src/responders/user-responders.js
816 ↗(On Diff #39845)

This used to be called at the beginning of processOLMAccountCreation, so we just move it here

keyserver/src/responders/user-responders.js
688–696 ↗(On Diff #39683)

This can be handled in a separate diff, since this is not an issue introduced by this diff. Created ENG-8052 to track

@ashoat I think splitting this diff would only introduce more confusion at this point. But I created separate tasks for anything that still has to be implemented, but it can be done in a separate diff. And next time I will create simpler diffs from the start.

I will amend the test plan in a moment

createAccount will be updated in a separate diff, so not adding any test plan for it

ashoat requested changes to this revision.May 6 2024, 1:53 PM

Really close! Only one serious comment inline below.

In D11780#340802, @inka wrote:

@ashoat I think splitting this diff would only introduce more confusion at this point. But I created separate tasks for anything that still has to be implemented, but it can be done in a separate diff. And next time I will create simpler diffs from the start.

Thanks @inka. I think this decision makes sense, and I'm glad to see you splitting off some of your updates into separate diffs!

keyserver/src/responders/user-responders.js
351 ↗(On Diff #39848)

Nit

815 ↗(On Diff #39848)

I don't understand why you put this line here. It's problematic on several levels:

  1. It's confusing to see it called within olmAccountCreationPromise when it has nothing to do with Olm account creation
  2. Now the check doesn't even seem to occur if !existingUsername

I believe your intention is simply to make sure it's called before olmAccountCreationPromise, right? You should use the same format as the snippet I shared in my last review. Breaking it down:

const xNeedsToHappenBeforeYPromise = (async () => { ... })();
const yPromise = (async () => {
  if (earlyReturn) {
    return;
  }
  await xNeedsToHappenBeforeYPromise;
  ...
})();
await Promise.all([xNeedsToHappenBeforeYPromise, yPromise]);
340–358 ↗(On Diff #39683)

Thanks for creating ENG-8049. I agree that it's not a priority to address now, and I think your stopgap of omitting the return []; from my snippet makes sense

This revision now requires changes to proceed.May 6 2024, 1:53 PM
keyserver/src/responders/user-responders.js
815 ↗(On Diff #39848)

Actually, wouldn't it be way simpler to just call verifyCalendarQueryThreadIDs earlier, such as on line 756? (step 1)

keyserver/src/responders/user-responders.js
815 ↗(On Diff #39848)

My intention with this was imitate how the code worked before my changes. As I wrote in the inline below, this used to be called at the beginning of processOLMAccountCreation, which was called inside olmAccountCreationPromise.
I wanted to avoid introducing any changes to when verifyCalendarQueryThreadIDs is called, to reduce the complexity of this diff, and because that is not related to the issue I'm solving
Created ENG-8058 to track. Please let me know if you think this should be prioritised, but I don't think this is in scope of this issue

ashoat requested changes to this revision.May 7 2024, 6:43 AM

Two notes:

  1. When a diff reviewer requests changes with a simple, specific proposal, you should almost always make that change. It should only be skipped when you have a strong reason to, such as the proposed change is bad. Creating a follow-up task that you don't ever intend to address instead of making a simple change is not something we should be doing on this team. This applies to myself, @tomek, and everybody else on the team.
  2. Thanks for creating ENG-8058. I agree that it makes sense to evaluate this in a holistic way, across all of the auth responders. Let's prioritize this – I've set its parent to ENG-7247, and moved that parent back to "In Progress".
This revision now requires changes to proceed.May 7 2024, 6:43 AM

Move call to verifyCalendarQueryThreadIDs in keyserverAuthResponder. I will update other responders in https://linear.app/comm/issue/ENG-8058/review-verifycalendarquerythreadids-in-auth-responders

This revision is now accepted and ready to land.May 7 2024, 8:07 AM