HomePhabricator
Diffusion Comm 19613c3beb46

[keyserver] Add migration - process messages in our db for search

Description

[keyserver] Add migration - process messages in our db for search

Summary:
issue:https://linear.app/comm/issue/ENG-3316/implement-processing-messages-that-are-already-in-the-db-for-search
We need to process the messages that are already in our db for search.
pageSize is 1001 because I used the idea from https://mariadb.com/kb/en/pagination-optimization/ that is to fetch one more row than we intend to process. This allows us to know at each query whether it is the last query or not, and saves us from running a query that would return nothing, in case the number of elements is divisible by the page size we actually process.
This is also why if (messages.length === pageSize) I process a table that is one element shorter.
I use the id not the timestamp, because we don't have an index on the timestamp.

This should be landed after D7077

Test Plan: I checked that db_version in metadata table is 19, and run the keyserver. After each test I removed part, or all of the rows in the search table and changed db_version in metadata table to 19. I checked that the processed messages appear in the search table. I checked that no messages of type 0 or 20 are skipped (I artificially added some messages of type 20 to my messages table)

Reviewers: kamil, bartek, ashoat

Reviewed By: bartek, ashoat

Subscribers: kuba, ashoat, tomek, atul

Differential Revision: https://phab.comm.dev/D7117

Details

Provenance
InkaSokolowskaAuthored on Apr 7 2023, 6:08 AM
Reviewer
bartek
Differential Revision
D7117: [keyserver] Add migration - process messages in our db for search
Parents
rCOMM4ddc00224434: [keyserver] Process new messages for search
Branches
Unknown
Tags
Unknown