[keyserver] Test impact of ThreadStore reductions on Atul's ThreadStore
d2fdf5398d9e
Actions

Description

[keyserver] Test impact of ThreadStore reductions on Atul's ThreadStore

Summary:
This diff stack introduces a script that tests various different approaches to reducing the size of the ThreadStore. We may or may not land it... the goal is just to get some numbers.

My ThreadStore has some unique quirks, so I figured it would be good to test at least one other.

Depends on D9342

Test Plan:
Final results:

before reductions, Ashoat's ThreadStore is 74509866 bytes large (802748 bytes compressed)
after [A] filtering out old sidebars, Ashoat's ThreadStore is 28795130 bytes large (469736 bytes compressed)
after [B] improving encoding, Ashoat's ThreadStore is 11379070 bytes large (378324 bytes compressed)
after [C] introducing isAdminRole, Ashoat's ThreadStore is 14839491 bytes large (469148 bytes compressed)
after [D] introducing ThreadRolePermissionsStore, Ashoat's ThreadStore is 5640441 bytes large (322220 bytes compressed)
after [A] and [B], Ashoat's ThreadStore is 5711706 bytes large (232280 bytes compressed)
after [A] and [C], Ashoat's ThreadStore is 10047103 bytes large (287884 bytes compressed)
after [A] and [D], Ashoat's ThreadStore is 3064171 bytes large (199800 bytes compressed)
after [B] and [C], Ashoat's ThreadStore is 6993311 bytes large (346120 bytes compressed)
after [B] and [D], Ashoat's ThreadStore is 5632940 bytes large (323304 bytes compressed)
after [C] and [D], Ashoat's ThreadStore is 5640402 bytes large (322508 bytes compressed)
after [A], [B], and [C], Ashoat's ThreadStore is 4121983 bytes large (218956 bytes compressed)
after [A], [B], and [D], Ashoat's ThreadStore is 3058364 bytes large (200104 bytes compressed)
after [A], [C], and [D], Ashoat's ThreadStore is 3064132 bytes large (200072 bytes compressed)
after [B], [C], and [D], Ashoat's ThreadStore is 5632901 bytes large (323400 bytes compressed)
after [A], [B], [C], and [D], Ashoat's ThreadStore is 3058325 bytes large (200056 bytes compressed)
before reductions, Atul's ThreadStore is 45113868 bytes large (285084 bytes compressed)
after [A] filtering out old sidebars, Atul's ThreadStore is 4667972 bytes large (52036 bytes compressed)
after [B] improving encoding, Atul's ThreadStore is 5567741 bytes large (135580 bytes compressed)
after [C] introducing isAdminRole, Atul's ThreadStore is 4429463 bytes large (165248 bytes compressed)
after [D] introducing ThreadRolePermissionsStore, Atul's ThreadStore is 2460260 bytes large (115260 bytes compressed)
after [A] and [B], Atul's ThreadStore is 633244 bytes large (27396 bytes compressed)
after [A] and [C], Atul's ThreadStore is 684532 bytes large (37800 bytes compressed)
after [A] and [D], Atul's ThreadStore is 297540 bytes large (24240 bytes compressed)
after [B] and [C], Atul's ThreadStore is 2694710 bytes large (117604 bytes compressed)
after [B] and [D], Atul's ThreadStore is 2455381 bytes large (115264 bytes compressed)
after [C] and [D], Atul's ThreadStore is 2460251 bytes large (115628 bytes compressed)
after [A], [B], and [C], Atul's ThreadStore is 345579 bytes large (24436 bytes compressed)
after [A], [B], and [D], Atul's ThreadStore is 293864 bytes large (23964 bytes compressed)
after [A], [C], and [D], Atul's ThreadStore is 297531 bytes large (24172 bytes compressed)
after [B], [C], and [D], Atul's ThreadStore is 2455372 bytes large (115352 bytes compressed)
after [A], [B], [C], and [D], Atul's ThreadStore is 293855 bytes large (23996 bytes compressed)

NOTE: I've been tested this on a copy of my keyserver's production database from a couple days back. To make sure the data on approach A is accurate, I replaced Date.now() with the timestamp of the last message in the database. As a result, the numbers above might be slightly different from the numbers in earlier diffs.

Reviewers: atul, tomek

Reviewed By: atul

Differential Revision: https://phab.comm.dev/D9343