Change Details

issue:https://linear.app/comm/issue/ENG-3316/implement-processing-messages-that-are-already-in-the-db-for-search We need to process the messages that are already in our db for search. pageSize is 1001 because I used the idea from https://mariadb.com/kb/en/pagination-optimization/ that is to fetch one more row than we intend to process. This allows us to know at each query whether it is the last query or not, and saves us from running a query that would return nothing, in case the number of elements is divisible by the page size we actually process. This is also why `if (messages.length === pageSize)` I process a table that is one element shorter. I use the id not the timestamp, because we don't have an index on the timestamp. This should be landed after D7077

issue:https://linear.app/comm/issue/ENG-3316/implement-processing-messages-that-are-already-in-the-db-for-search We need to process the messages that are already in our db for search. pageSize is 1001 because I used the idea from https://mariadb.com/kb/en/pagination-optimization/ that is to fetch one more row than we intend to process. This allows us to know at each query whether it is the last query or not, and saves us from running a query that would return nothing, in case the number of elements is divisible by the page size we actually process. This is also why `if (messages.length === pageSize)` I process a table that is one element shorter. I use the id not the timestamp, because we don't have an index on the timestamp. This should be landed after D7077

issue:https://linear.app/comm/issue/ENG-3316/implement-processing-messages-that-are-already-in-the-db-for-search We need to process the messages that are already in our db for search. pageSize is 1001 because I used the idea from https://mariadb.com/kb/en/pagination-optimization/ that is to fetch one more row than we intend to process. This allows us to know at each query whether it is the last query or not, and saves us from running a query that would return nothing, in case the number of elements is divisible by the page size we actually process. This is also why `if (messages.length === pageSize)` I process a table that is one element shorter. I use the id not the timestamp, because we don't have an index on the timestamp. This should be landed after D7077