Page MenuHomePhabricator

[keyserver] Add fetch latest messages endpoint
AbandonedPublic

Authored by michal on Jun 2 2023, 6:12 AM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Mar 31, 1:20 PM
Unknown Object (File)
Sun, Mar 31, 1:20 PM
Unknown Object (File)
Sun, Mar 31, 1:17 PM
Unknown Object (File)
Mar 4 2024, 11:27 PM
Unknown Object (File)
Mar 4 2024, 11:27 PM
Unknown Object (File)
Mar 4 2024, 11:27 PM
Unknown Object (File)
Feb 23 2024, 12:19 AM
Unknown Object (File)
Feb 22 2024, 11:36 PM
Subscribers

Details

Summary

ENG-3972
Part of the fix for the "no messages" bug. This endpoint allows us to fetch the latest message for 25 threads, that we still don't have messages for (as an argument you pass the oldest message the client is aware of).

This is mostly based on @ashoat idea in the parent task

Test Plan

Tested with the later diffs if the messages are fetched correctly. Tests manully by making requests and checking if correct messages were returned.

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

michal requested review of this revision.Jun 2 2023, 6:30 AM
keyserver/src/fetchers/thread-fetchers.js
317–323 ↗(On Diff #27396)

It doesn't appear that you're using the threads table for anything. In my original query I was querying for the name column, but since you only need id, it appears that you should be able to do this query just on memberships, without a LEFT JOIN. Let me know if I'm missing something

lib/reducers/message-reducer.js
878–907 ↗(On Diff #27396)

I think we can merge these

kamil requested changes to this revision.Jun 6 2023, 4:17 AM

requesting changes to remove this from our queues as @michal is off this week and needs to respond to @ashoat's question

lib/actions/message-actions.js
318–319 ↗(On Diff #27396)

I think that because of the fact that response might return more fields, we typically explicitly return the object

This revision now requires changes to proceed.Jun 6 2023, 4:17 AM
  • Fixed small issues
  • Instead of passing joinedThreads to fetchMessageInfo I instead add a condition to the sql query role > 0 (more context here)
  • On the client threads are sorted according to lastUpdatedTimeIncludingSidebars. This means that:
    • we could fetch last message for a sidebar
    • client still doesn't have messages for the parent thread, so it's not displayed
    • later client scrolls down and fetches last message for the parent thread
    • parent thread in displayed, but it jumps up because it's sidebar has an earlier messages To fix that I also fetch the last message for parent threads of any sidebars. This complicates the lastMessage logic (I now have to filter the additional parent threads) so now it's done on the server and passed to the client.
keyserver/src/fetchers/thread-fetchers.js
311

Hmm... one concern here is that we're fetching the parents for a sidebar, but not including "sibling sidebars" that might be displayed in the UI. If the user scrolls down and we load another sibling sidebar, that sibling would be displayed above where the user scrolled to, so they will miss it. I wonder if we should do some work to make sure we include all of the sidebars that would normally be rendered in the UI today (there is an algorithm for determing which sidebars to show here)

keyserver/src/fetchers/thread-fetchers.js
311

It should be fine (but kinda brittle) in this case. The cutoff for displaying the sidebars is 3 days, while we stop sending messages after 2 weeks. So sidebars that would be fetched with this endpoint won't be displayed anyway.

keyserver/src/fetchers/thread-fetchers.js
311

That appears to only be true if the sidebars are marked as read. If the sidebars are unread, it looks like we show up to 5 of them indefinitely

michal added inline comments.
keyserver/src/fetchers/thread-fetchers.js
311

Yes, but unread status is kept in threadInfo which is always available. And there's no problem with displaying sidebars without messages in messageStore because they don't have a preview.

ashoat requested changes to this revision.Jun 28 2023, 3:14 PM
ashoat added inline comments.
keyserver/src/fetchers/thread-fetchers.js
311

Then why are we querying for sidebars at all?

It seems like what we want is a list of top-level chats, ordered by the most recent message in that chat or any of its sidebars. Instead, you're querying for a list of all chats (including sidebars), ordered by the most recent message in that chat.

If you could find a MySQL query that gives you the result you want directly, we'll see the following benefits:

  1. The code will be more readable. The reader doesn't have to ask "why do we return this pair here?", and they won't have to ask why we're adding parent thread IDs to the allThreads list
  2. The API exposed to clients will be more reasonable. By guaranteeing a top-level chat and its sidebars only appear once in the list returned by the API, we match more closely the behavior we need in the UI, and we avoid having to do any complicated post-processing on the client
  3. We can avoid unnecessarily fetching messageInfos for sidebars if this function only returns the list of top-level chats that we need previews for

I assume such a query would have to be more complicated, so it would be good if you can give me an example query I can run on my production keyserver, so we can audit the performance of the new query. What do you think?

keyserver/src/responders/message-responders.js
560

If we don't need any messages for sidebars (since the previews are not needed), then why are we querying for them here?

We are changing the approach for this task, so I'm abandoning this diff. More info in the linear task.