Page MenuHomePhabricator

Implement notifications sessions storage on top of MMKV. Migrate existing notifs account to SQLite
ClosedPublic

Authored by marcin on Mar 4 2024, 5:01 AM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Dec 24, 2:04 AM
Unknown Object (File)
Tue, Dec 24, 2:04 AM
Unknown Object (File)
Tue, Dec 24, 2:04 AM
Unknown Object (File)
Tue, Dec 24, 2:04 AM
Unknown Object (File)
Fri, Dec 6, 8:37 AM
Unknown Object (File)
Sat, Nov 30, 12:08 PM
Unknown Object (File)
Nov 27 2024, 4:50 PM
Unknown Object (File)
Nov 25 2024, 10:26 AM
Subscribers

Details

Summary

This differential deprecates old way of keeping notifications sessions and builds new one on top of MMKV. Now sessions are kept in MMKV under keys that are
constructed from relevant keyserver id. Pickling key for sessions is also stored in MMKV. Additionally this differential implements SQLite migration that transfers
existing notifs account and session from old storage to SQLite and MMKV respectively.

Test Plan
  1. Build the app before applying this diff and log in to create session.
  2. Apply this diff and buld the app again.
  3. Ensure the notifs are still working. This tests migration.
  4. Log out and log in and ensure notifs are stil working.

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Important changes were made to: SQLiteQueryExecutor.cpp and NotificationsCryptoModule.{h, cpp} files. The rest are boilerplate and logical consequences.

Harbormaster returned this revision to the author for changes because remote builds failed.Mar 4 2024, 5:13 AM
Harbormaster failed remote builds in B27292: Diff 37769!
native/cpp/CommonCpp/DatabaseManagers/SQLiteQueryExecutor.cpp
518 ↗(On Diff #37769)

It isn't ideal but I don't think there is a better place for this migration.

520 ↗(On Diff #37769)

This is defined in CommCoreModule: https://github.com/CommE2E/comm/blob/master/native/cpp/CommonCpp/NativeModules/CommCoreModule.h#L27. The CommCoreModule uses this constant to store pickling key for notifs and content account. Ideally this constant should be removed from CommCoreModule and we should keep picling keys for accounts in SQLite in the same table where we keep olm accounts as we do on the keyserver. The fact that we keep picling keys for both accounts in Secure Store is an artifact since code that creates and persists crypto account in SQLite was implemented before we introduced SQLite encryption so it made sense. But now we need to migrate this pickling key to SQLite. I will create follow up task for this.

native/cpp/CommonCpp/Notifications/BackgroundDataStorage/NotificationsCryptoModule.cpp
76 ↗(On Diff #37769)

This implementation addresses the issue.

When we call clearSensitiveData then NOTIFS.PICKLING_KEY is removed from MMKV. When we call flushState we call getPicklingKey(false). Therefore if clearSensitiveData is called after statefulDecrypt and before flushState then flushState will fail since there is no pickling key in MMKV. This solves the race condition described in the issue linked above.

  1. Add missing newlines.
  2. Fix emscriptedn CI
marcin requested review of this revision.Mar 4 2024, 6:05 AM
michal added inline comments.
native/cpp/CommonCpp/Notifications/BackgroundDataStorage/NotificationsCryptoModule.cpp
25 ↗(On Diff #37794)

I think @inka is currently working on making this configurable on dev envs, so we probably can't just hardcode it.

native/cpp/CommonCpp/Notifications/BackgroundDataStorage/NotificationsCryptoModule.cpp
104 ↗(On Diff #37771)

I had a discussion with @tomek and was told that a good practice with key-value stores is to have a schema for keys in which the key begins with most general and ends with the most specific information. Therefore we begin with KEYSERVER. meaning that this key holds a value for some keyserver, then we have the keyserver id and finally we have information what kind of data we store. Similar scheme is adopted for unread count: KEYSERVER. + <id> + .UNREAD_COUNT.

I've just realized that there is a vulnerability with the design here. The JS promise that calls initializeNotificationsSession will create pickling key entry in MMKV if it hasn't been created yet. but what if we have two promises running simultaneously for two different keyservers? This entry might become corrupted then. I think it is better to keep separate pickling key for each session bundled together with the session itself as a JSON.

Store each session bundled with its pickling key in MMKV to avoid potential rece condition.

native/cpp/CommonCpp/Notifications/BackgroundDataStorage/NotificationsCryptoModule.cpp
24 ↗(On Diff #37896)

We can't just hardcode this. It needs to be configurable, as @michal mentioned

24 ↗(On Diff #37896)

(In particular, it should configurable via the same mechanisms that @inka is using on native... we should avoid the need for additional configuration)

native/android/app/src/main/java/app/comm/android/notifications/CommNotificationsHandler.java
91–92 ↗(On Diff #37896)

In cases of other errors, we're returning null, e.g. when Malformed notification JSON payload.. Why do we handle this case differently?

native/android/app/src/main/java/app/comm/android/notifications/CommNotificationsHandler.java
91–92 ↗(On Diff #37896)

I agree - throwing isn't particularly meaningful here. On the other hand logging and silently returning (as we do in case of Malformed notification JSON payload ) also isn't our best option since notification without keyserverID is more likely to occur than JSONException since it is new functionality that we introduce. I will display error message notification for staff users (we have done this already multiple times) and log + silent return for public users. @tomek tell me what you think.

native/cpp/CommonCpp/Notifications/BackgroundDataStorage/NotificationsCryptoModule.cpp
24 ↗(On Diff #37896)

This constant is used only when someone currently logged in updates/re-builds the app. Then session from old storage is transferred to MMKV and the key is created from value 256. That said it is correct to hardcode for production. It is also correct to hardcode for dev env if some one is logged to "old" keyserver. It will only fail if we are currently logged to "new" keyserver (launched according to @inka doc) and rebuild the app after landing my changes. That said I am wondering how bad is it to ask developers to log out before building the app with my changes and then log in after rebuilding the app? In such a case new session will be created with key created from actual keyserver id and this constant will never be used. The only alternative that I can think of is XCode build phase/build.gradle task that will read the content of @inka JSON and replace this constant with the value from the JSON. I am not sure if it is worth the hassle.
cc @ashoat @michal @inka what do you think?

native/android/app/src/main/java/app/comm/android/notifications/CommNotificationsHandler.java
91–92 ↗(On Diff #37896)

I will display error message notification for staff users (we have done this already multiple times) and log + silent return for public users.

I think this is a good approach

native/cpp/CommonCpp/Notifications/BackgroundDataStorage/NotificationsCryptoModule.cpp
24 ↗(On Diff #37896)

My opinion is that since this is a one time migration that works for production and for master, we shouldn't spend time on using the id from my config files (since I'm told it would take more than one day, so quite a lot of time).
If someone is using the real keyserver id already, they have to have a patched diff (D11141). This is basically trying to avoid breaking someones local changes, which I don't think we tend to do.
Especially that all they will need to do is log out, which probably isn't a big problem.

native/cpp/CommonCpp/Notifications/BackgroundDataStorage/NotificationsCryptoModule.cpp
24 ↗(On Diff #37896)

Thanks for explaining. Only one concern:

This constant is used only when someone currently logged in updates/re-builds the app.

How do we know that will be the case in the future? Introducing a constant like this is risky. Given how important it appears that this constant is not used elsewhere, it seems like you should rename it something like ashoatKeyserverIDUsedOnlyForMigrationFromLegacyNotifStorage, and add a lengthy comment above it warning against its use.

  1. Display error message notification on Android if received notification is missing keyserverID.
  2. Add explanatory comment regarding authoritativeKeyserverID constant in NotificationsCryptoModule.

Can you explain why you didn't rename the variable as requested? If you disagree with the feedback, I'm open to it, but you need to respond in some way to feedback... it should not simply be ignored.

Can you explain why you didn't rename the variable as requested? If you disagree with the feedback, I'm open to it, but you need to respond in some way to feedback... it should not simply be ignored.

It wasn't intentional - it was just my oversight.

Rename authoritativeKeyserverID to ashoatKeyserverIDUsedOnlyForMigrationFromLegacyNotifStorage.

michal added inline comments.
native/cpp/CommonCpp/Notifications/BackgroundDataStorage/NotificationsCryptoModule.h
13 ↗(On Diff #38019)

Can we also rename the method to show that it's legacy and for migration only, in addition to the comment?

This revision is now accepted and ready to land.Mar 12 2024, 8:54 AM