Page MenuHomePhabricator

[native] Fetch only a subset of messages from thick threads
ClosedPublic

Authored by tomek on Aug 22 2024, 4:39 AM.
Tags
None
Referenced Files
F3343526: D13137.id43686.diff
Fri, Nov 22, 2:59 AM
Unknown Object (File)
Sat, Nov 16, 6:07 AM
Unknown Object (File)
Sat, Nov 9, 6:23 AM
Unknown Object (File)
Sat, Nov 9, 5:27 AM
Unknown Object (File)
Fri, Nov 8, 10:37 PM
Unknown Object (File)
Fri, Nov 8, 1:44 PM
Unknown Object (File)
Oct 22 2024, 8:33 AM
Unknown Object (File)
Oct 15 2024, 5:55 PM
Subscribers

Details

Summary

For the thick threads we don't want to initially fetch all the messages. We're going to fetch them in batches while the user scrolls.

https://linear.app/comm/issue/ENG-8702/modify-nativedatasqlite-data-handlerjs-to-fetch-only
https://linear.app/comm/issue/ENG-8703/modify-webshared-workerutilsstorejs-to-fetch-only

Depends on D13133

Test Plan

On web, disable pruning completely, create a thick thread with 100 messages. Close and open the app and check if only 20 messages were fetched.
With the pruning enabled, modify the query to fetch 5 messages and check if thick threads don't contain more.

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Revert unnecessary changes

native/cpp/CommonCpp/DatabaseManagers/SQLiteQueryExecutor.cpp
1522–1527 ↗(On Diff #43571)

In getInitialMessages we have to explicitly specify the selected columns, and it doesn't make sense to include there fields that are skipped by processMessagesResults, so I modified the processMessagesResults function and all the places where it is used to skip the unused columns.

tomek requested review of this revision.Aug 22 2024, 4:58 AM
native/cpp/CommonCpp/NativeModules/PersistentStorageUtilities/ThreadOperationsUtilities/ThreadTypeEnum.h
5–20 ↗(On Diff #43583)

I wonder if it would be better to put this in a place where the web code could eventually access it. But not very important

native/cpp/CommonCpp/DatabaseManagers/SQLiteQueryExecutor.cpp
1500–1503 ↗(On Diff #43583)

It appears that it's critical for this to be synced with JS code

You don't appear to have added comments on both sides, which is the absolute minimum we need when we are doing something like this (expected two pieces of code to be synced)

Probably a better solution would be to have a collection in some place like ThreadTypeEnum.h... keeping SQL logic in sync with a list seems cumbersome

native/cpp/CommonCpp/NativeModules/PersistentStorageUtilities/ThreadOperationsUtilities/ThreadTypeEnum.h
5–20 ↗(On Diff #43583)

Probably need comments to keep this list in sync as well

native/cpp/CommonCpp/DatabaseManagers/SQLiteQueryExecutor.cpp
1500–1503 ↗(On Diff #43583)

Makes sense. Going to introduce a list to that file.

About sharing code with JS - going to add the comments. In the future, we can do https://linear.app/comm/issue/ENG-8995/shared-constant-between-web-and-native which will solve the issue completely.

native/cpp/CommonCpp/NativeModules/PersistentStorageUtilities/ThreadOperationsUtilities/ThreadTypeEnum.h
5–20 ↗(On Diff #43583)

It is already indirectly used by the web. SQLiteQueryExecutor can import from this file and use it through the WASM. The fact that it is placed inside native is unfortunate - we have a task https://linear.app/comm/issue/ENG-4311/move-database-code-to-shared that covers moving the whole shared code.

5–20 ↗(On Diff #43583)

Yeah, good call.

Extract thick thread types

kamil added inline comments.
native/cpp/CommonCpp/DatabaseManagers/SQLiteQueryExecutor.cpp
1494 ↗(On Diff #43620)
1522–1527 ↗(On Diff #43571)

Nice

This revision is now accepted and ready to land.Aug 26 2024, 2:09 AM
tomek marked an inline comment as done.

Format