Page MenuHomePhabricator
Differential D7174

[keyserver] Fix account deletion performance
ClosedPublic
Actions

Authored by ashoat on Mar 24 2023, 5:04 PM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Mar 4, 1:15 PM
Unknown Object (File)
Tue, Mar 4, 1:15 PM
Unknown Object (File)
Tue, Mar 4, 1:15 PM
Unknown Object (File)
Tue, Mar 4, 1:15 PM
Unknown Object (File)
Tue, Mar 4, 1:14 PM
Unknown Object (File)
Tue, Mar 4, 1:06 PM
Unknown Object (File)
Feb 26 2025, 12:31 PM
Unknown Object (File)
Feb 25 2025, 9:49 AM
View All Files
Subscribers
None

Details

Reviewers
atul
tomek
• jon
Commits
rCOMM26c2d2f52e0b: [keyserver] Fix account deletion performance
Summary

Account deletion used to time out. After this it's basically instantaneous. The reports table is pretty huge...

More context here on Linear.

Test Plan

I ran it on prod and it fixed the issue

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

ashoat created this revision.Mar 24 2023, 5:04 PM
ashoat updated this revision to Diff 24105.Mar 24 2023, 5:15 PM

I analyzed the queries we run on the reports table and found that creating a report runs this query, so I refined this index so it can help there as well. No other queries on reports seemed to do a full scan

ashoat updated this revision to Diff 24106.Mar 24 2023, 5:16 PM

Forgot to update createTables in the last revision

ashoat added a reviewer: • jon.Mar 24 2023, 5:20 PM
ashoat added inline comments.
keyserver/src/cron/cron.js
30 ↗(On Diff #24106)

Not sure why I missed this one in D6334. The context is explained there

keyserver/src/database/migration-config.js
224 ↗(On Diff #24106)

I already created this index on prod to test the fix, before I realized it would be better to have a more specific index. This part of the query should only affect prod

Harbormaster completed remote builds in B17610: Diff 24104.Mar 24 2023, 5:20 PM
Harbormaster completed remote builds in B17611: Diff 24105.Mar 24 2023, 5:31 PM
Harbormaster completed remote builds in B17612: Diff 24106.
ashoat requested review of this revision.Mar 24 2023, 5:31 PM
tomek accepted this revision.Mar 27 2023, 2:10 AM
tomek added inline comments.
keyserver/src/cron/cron.js
30 ↗(On Diff #24106)

This is specific to a location of a keyserver, so we can also consider saying that it's in a keyserver's timezone instead of specifying a timezone explicitly.

This revision is now accepted and ready to land.Mar 27 2023, 2:10 AM
ashoat updated this revision to Diff 24196.Mar 27 2023, 7:06 AM

Clarify that it's in the timezone of the keyserver

This revision was landed with ongoing or failed builds.Mar 27 2023, 7:06 AM
Closed by commit rCOMM26c2d2f52e0b: [keyserver] Fix account deletion performance (authored by ashoat). · Explain Why
This revision was automatically updated to reflect the committed changes.
ashoat added a commit: rCOMM26c2d2f52e0b: [keyserver] Fix account deletion performance.
Harbormaster completed remote builds in B17675: Diff 24196.Mar 27 2023, 7:21 AM

Revision Contents
Changeset List

PathSize
keyserver/
src/
cron/
8 lines
database/
11 lines
4 lines

Diff 24197

View Options

keyserver/src/cron/cron.js

Show All 22 Linesimport {
type ServerThreadJoinRequest, type ServerThreadJoinRequest,
type ThreadJoinResult, type ThreadJoinResult,
type ThreadFetchMediaResult, type ThreadFetchMediaResult,
type ThreadFetchMediaRequest, type ThreadFetchMediaRequest,
type ToggleMessagePinRequest, type ToggleMessagePinRequest,
type ToggleMessagePinResult, type ToggleMessagePinResult,
type RoleModificationRequest, type RoleModificationRequest,
type RoleModificationResult, type RoleModificationResult,
type RoleDeletionRequest,
type RoleDeletionResult,
rawThreadInfoValidator, rawThreadInfoValidator,
} from 'lib/types/thread-types.js'; } from 'lib/types/thread-types.js';
import { serverUpdateInfoValidator } from 'lib/types/update-types.js'; import { serverUpdateInfoValidator } from 'lib/types/update-types.js';
import { userInfosValidator } from 'lib/types/user-types.js'; import { userInfosValidator } from 'lib/types/user-types.js';
import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js'; import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js';
import { values } from 'lib/utils/objects.js'; import { values } from 'lib/utils/objects.js';
import { import {
tShape, tShape,
tNumEnum, tNumEnum,
tColor, tColor,
tPassword, tPassword,
tID, tID,
} from 'lib/utils/validation-utils.js'; } from 'lib/utils/validation-utils.js';
import { import {
entryQueryInputValidator, entryQueryInputValidator,
verifyCalendarQueryThreadIDs, verifyCalendarQueryThreadIDs,
} from './entry-responders.js'; } from './entry-responders.js';
import { modifyRole } from '../creators/role-creator.js'; import { modifyRole } from '../creators/role-creator.js';
import { createThread } from '../creators/thread-creator.js'; import { createThread } from '../creators/thread-creator.js';
import { deleteRole } from '../deleters/role-deleters.js';
import { deleteThread } from '../deleters/thread-deleters.js'; import { deleteThread } from '../deleters/thread-deleters.js';
import { fetchMediaForThread } from '../fetchers/upload-fetchers.js'; import { fetchMediaForThread } from '../fetchers/upload-fetchers.js';
import type { Viewer } from '../session/viewer.js'; import type { Viewer } from '../session/viewer.js';
import { import {
updateRole, updateRole,
removeMembers, removeMembers,
leaveThread, leaveThread,
updateThread, updateThread,
▲ Show 20 Lines • Show All 320 Lines • ▼ Show 20 Lines): Promise<RoleModificationResult> {
const response = await modifyRole(viewer, request); const response = await modifyRole(viewer, request);
return validateOutput( return validateOutput(
viewer.platformDetails, viewer.platformDetails,
roleModificationResultValidator, roleModificationResultValidator,
response, response,
); );
} }
const roleDeletionRequestInputValidator = tShape<RoleDeletionRequest>({
community: tID,
roleID: tID,
});
export const roleDeletionResultValidator: TInterface<RoleDeletionResult> =
tShape<RoleDeletionResult>({
threadInfo: t.maybe(rawThreadInfoValidator),
updatesResult: tShape({
newUpdates: t.list(serverUpdateInfoValidator),
}),
});
async function roleDeletionResponder(
viewer: Viewer,
input: mixed,
): Promise<RoleDeletionResult> {
const request = await validateInput(
viewer,
roleDeletionRequestInputValidator,
input,
);
const response = await deleteRole(viewer, request);
return validateOutput(
viewer.platformDetails,
roleDeletionResultValidator,
response,
);
}
export { export {
threadDeletionResponder, threadDeletionResponder,
roleUpdateResponder, roleUpdateResponder,
memberRemovalResponder, memberRemovalResponder,
threadLeaveResponder, threadLeaveResponder,
threadUpdateResponder, threadUpdateResponder,
threadCreationResponder, threadCreationResponder,
threadJoinResponder, threadJoinResponder,
threadFetchMediaResponder, threadFetchMediaResponder,
newThreadRequestInputValidator, newThreadRequestInputValidator,
toggleMessagePinResponder, toggleMessagePinResponder,
roleModificationResponder, roleModificationResponder,
roleDeletionResponder,
}; };
View Options

keyserver/src/database/migration-config.js

Show All 22 Linesimport {
type ServerThreadJoinRequest, type ServerThreadJoinRequest,
type ThreadJoinResult, type ThreadJoinResult,
type ThreadFetchMediaResult, type ThreadFetchMediaResult,
type ThreadFetchMediaRequest, type ThreadFetchMediaRequest,
type ToggleMessagePinRequest, type ToggleMessagePinRequest,
type ToggleMessagePinResult, type ToggleMessagePinResult,
type RoleModificationRequest, type RoleModificationRequest,
type RoleModificationResult, type RoleModificationResult,
type RoleDeletionRequest,
type RoleDeletionResult,
rawThreadInfoValidator, rawThreadInfoValidator,
} from 'lib/types/thread-types.js'; } from 'lib/types/thread-types.js';
import { serverUpdateInfoValidator } from 'lib/types/update-types.js'; import { serverUpdateInfoValidator } from 'lib/types/update-types.js';
import { userInfosValidator } from 'lib/types/user-types.js'; import { userInfosValidator } from 'lib/types/user-types.js';
import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js'; import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js';
import { values } from 'lib/utils/objects.js'; import { values } from 'lib/utils/objects.js';
import { import {
tShape, tShape,
tNumEnum, tNumEnum,
tColor, tColor,
tPassword, tPassword,
tID, tID,
} from 'lib/utils/validation-utils.js'; } from 'lib/utils/validation-utils.js';
import { import {
entryQueryInputValidator, entryQueryInputValidator,
verifyCalendarQueryThreadIDs, verifyCalendarQueryThreadIDs,
} from './entry-responders.js'; } from './entry-responders.js';
import { modifyRole } from '../creators/role-creator.js'; import { modifyRole } from '../creators/role-creator.js';
import { createThread } from '../creators/thread-creator.js'; import { createThread } from '../creators/thread-creator.js';
import { deleteRole } from '../deleters/role-deleters.js';
import { deleteThread } from '../deleters/thread-deleters.js'; import { deleteThread } from '../deleters/thread-deleters.js';
import { fetchMediaForThread } from '../fetchers/upload-fetchers.js'; import { fetchMediaForThread } from '../fetchers/upload-fetchers.js';
import type { Viewer } from '../session/viewer.js'; import type { Viewer } from '../session/viewer.js';
import { import {
updateRole, updateRole,
removeMembers, removeMembers,
leaveThread, leaveThread,
updateThread, updateThread,
▲ Show 20 Lines • Show All 320 Lines • ▼ Show 20 Lines): Promise<RoleModificationResult> {
const response = await modifyRole(viewer, request); const response = await modifyRole(viewer, request);
return validateOutput( return validateOutput(
viewer.platformDetails, viewer.platformDetails,
roleModificationResultValidator, roleModificationResultValidator,
response, response,
); );
} }
const roleDeletionRequestInputValidator = tShape<RoleDeletionRequest>({
community: tID,
roleID: tID,
});
export const roleDeletionResultValidator: TInterface<RoleDeletionResult> =
tShape<RoleDeletionResult>({
threadInfo: t.maybe(rawThreadInfoValidator),
updatesResult: tShape({
newUpdates: t.list(serverUpdateInfoValidator),
}),
});
async function roleDeletionResponder(
viewer: Viewer,
input: mixed,
): Promise<RoleDeletionResult> {
const request = await validateInput(
viewer,
roleDeletionRequestInputValidator,
input,
);
const response = await deleteRole(viewer, request);
return validateOutput(
viewer.platformDetails,
roleDeletionResultValidator,
response,
);
}
export { export {
threadDeletionResponder, threadDeletionResponder,
roleUpdateResponder, roleUpdateResponder,
memberRemovalResponder, memberRemovalResponder,
threadLeaveResponder, threadLeaveResponder,
threadUpdateResponder, threadUpdateResponder,
threadCreationResponder, threadCreationResponder,
threadJoinResponder, threadJoinResponder,
threadFetchMediaResponder, threadFetchMediaResponder,
newThreadRequestInputValidator, newThreadRequestInputValidator,
toggleMessagePinResponder, toggleMessagePinResponder,
roleModificationResponder, roleModificationResponder,
roleDeletionResponder,
}; };
View Options

keyserver/src/database/setup-db.js

Show All 22 Linesimport {
type ServerThreadJoinRequest, type ServerThreadJoinRequest,
type ThreadJoinResult, type ThreadJoinResult,
type ThreadFetchMediaResult, type ThreadFetchMediaResult,
type ThreadFetchMediaRequest, type ThreadFetchMediaRequest,
type ToggleMessagePinRequest, type ToggleMessagePinRequest,
type ToggleMessagePinResult, type ToggleMessagePinResult,
type RoleModificationRequest, type RoleModificationRequest,
type RoleModificationResult, type RoleModificationResult,
type RoleDeletionRequest,
type RoleDeletionResult,
rawThreadInfoValidator, rawThreadInfoValidator,
} from 'lib/types/thread-types.js'; } from 'lib/types/thread-types.js';
import { serverUpdateInfoValidator } from 'lib/types/update-types.js'; import { serverUpdateInfoValidator } from 'lib/types/update-types.js';
import { userInfosValidator } from 'lib/types/user-types.js'; import { userInfosValidator } from 'lib/types/user-types.js';
import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js'; import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js';
import { values } from 'lib/utils/objects.js'; import { values } from 'lib/utils/objects.js';
import { import {
tShape, tShape,
tNumEnum, tNumEnum,
tColor, tColor,
tPassword, tPassword,
tID, tID,
} from 'lib/utils/validation-utils.js'; } from 'lib/utils/validation-utils.js';
import { import {
entryQueryInputValidator, entryQueryInputValidator,
verifyCalendarQueryThreadIDs, verifyCalendarQueryThreadIDs,
} from './entry-responders.js'; } from './entry-responders.js';
import { modifyRole } from '../creators/role-creator.js'; import { modifyRole } from '../creators/role-creator.js';
import { createThread } from '../creators/thread-creator.js'; import { createThread } from '../creators/thread-creator.js';
import { deleteRole } from '../deleters/role-deleters.js';
import { deleteThread } from '../deleters/thread-deleters.js'; import { deleteThread } from '../deleters/thread-deleters.js';
import { fetchMediaForThread } from '../fetchers/upload-fetchers.js'; import { fetchMediaForThread } from '../fetchers/upload-fetchers.js';
import type { Viewer } from '../session/viewer.js'; import type { Viewer } from '../session/viewer.js';
import { import {
updateRole, updateRole,
removeMembers, removeMembers,
leaveThread, leaveThread,
updateThread, updateThread,
▲ Show 20 Lines • Show All 320 Lines • ▼ Show 20 Lines): Promise<RoleModificationResult> {
const response = await modifyRole(viewer, request); const response = await modifyRole(viewer, request);
return validateOutput( return validateOutput(
viewer.platformDetails, viewer.platformDetails,
roleModificationResultValidator, roleModificationResultValidator,
response, response,
); );
} }
const roleDeletionRequestInputValidator = tShape<RoleDeletionRequest>({
community: tID,
roleID: tID,
});
export const roleDeletionResultValidator: TInterface<RoleDeletionResult> =
tShape<RoleDeletionResult>({
threadInfo: t.maybe(rawThreadInfoValidator),
updatesResult: tShape({
newUpdates: t.list(serverUpdateInfoValidator),
}),
});
async function roleDeletionResponder(
viewer: Viewer,
input: mixed,
): Promise<RoleDeletionResult> {
const request = await validateInput(
viewer,
roleDeletionRequestInputValidator,
input,
);
const response = await deleteRole(viewer, request);
return validateOutput(
viewer.platformDetails,
roleDeletionResultValidator,
response,
);
}
export { export {
threadDeletionResponder, threadDeletionResponder,
roleUpdateResponder, roleUpdateResponder,
memberRemovalResponder, memberRemovalResponder,
threadLeaveResponder, threadLeaveResponder,
threadUpdateResponder, threadUpdateResponder,
threadCreationResponder, threadCreationResponder,
threadJoinResponder, threadJoinResponder,
threadFetchMediaResponder, threadFetchMediaResponder,
newThreadRequestInputValidator, newThreadRequestInputValidator,
toggleMessagePinResponder, toggleMessagePinResponder,
roleModificationResponder, roleModificationResponder,
roleDeletionResponder,
}; };