[services][backup] AddAttachments 4/4 - moving to blob
ClosedPublic
Actions

Authored by bartek on Jan 9 2023, 12:29 PM.

Details

Reviewers

varun
tomek
• jon
• max

Commits

rCOMM5ca27556e198: [services][backup] AddAttachments 4/4 - moving to blob

Summary

This implements the last part of the AddAttachments endpoint - moving log item storage to blob service when it's too large.

Depends on D6206

Test Plan

Manually created large log item, ensured that it's slightly less than 400kb.
Called AddAttachments providing a long attachments holder string.
Ensured Blob service got called and the log was uploaded & database entity was updated.

The backup service integration tests cover this case, but this needs remaining endpoints to be implemented.

Diff Detail

Repository

rCOMM Comm

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

bartek created this revision.Jan 9 2023, 12:29 PM

bartek held this revision as a draft.

Herald added subscribers: atul, ashoat. · View Herald TranscriptJan 9 2023, 12:29 PM

bartek published this revision for review.Jan 9 2023, 12:37 PM

Harbormaster completed remote builds in B15214: Diff 20742.Jan 9 2023, 12:43 PM

bartek added a child revision: D6211: [services][backup] SendLog 1/4 - create handler module.Jan 10 2023, 8:33 AM

tomek accepted this revision.Jan 11 2023, 5:55 AM

This revision is now accepted and ready to land.Jan 11 2023, 5:55 AM

Rebase

Harbormaster completed remote builds in B15311: Diff 20886.Jan 12 2023, 9:38 AM

Fixed holder instead of log hash being provided to put client

Harbormaster completed remote builds in B15312: Diff 20887.Jan 12 2023, 9:54 AM

Rebase

Harbormaster completed remote builds in B15467: Diff 21089.Jan 19 2023, 9:56 AM

Closed by commit rCOMM5ca27556e198: [services][backup] AddAttachments 4/4 - moving to blob (authored by bartek). · Explain WhyJan 19 2023, 10:07 AM

This revision was automatically updated to reflect the committed changes.

bartek added a commit: rCOMM5ca27556e198: [services][backup] AddAttachments 4/4 - moving to blob.

Revision Contents
Changeset List

Path

Size

services/

backup/

src/

service/

handlers/

add_attachments.rs

30 lines

Diff 21105

View Options

services/backup/src/service/handlers/add_attachments.rs

Show First 20 Lines • Show All 638 Lines • ▼ Show 20 Lines	return {
cookieInsertedThisRequest: true,		cookieInsertedThisRequest: true,
isScriptViewer: false,		isScriptViewer: false,
};		};
}		}

type UserCookieCreationParams = {		type UserCookieCreationParams = {
platformDetails: PlatformDetails,		platformDetails: PlatformDetails,
deviceToken?: ?string,		deviceToken?: ?string,
		primaryIdentityPublicKey?: ?string,
};		};

// The result of this function should never be passed directly to the Viewer		// The result of this function should never be passed directly to the Viewer
// constructor. Instead, it should be passed to viewer.setNewCookie. There are		// constructor. Instead, it should be passed to viewer.setNewCookie. There are
// several fields on UserViewerData that are not set by this function:		// several fields on UserViewerData that are not set by this function:
// sessionID, sessionIdentifierType, cookieSource, and ipAddress. These		// sessionID, sessionIdentifierType, cookieSource, and ipAddress. These
// parameters all depend on the initial request. If the result of this function		// parameters all depend on the initial request. If the result of this function
// is passed to the Viewer constructor directly, the resultant Viewer object		// is passed to the Viewer constructor directly, the resultant Viewer object
// will throw whenever anybody attempts to access the relevant properties.		// will throw whenever anybody attempts to access the relevant properties.
async function createNewUserCookie(		async function createNewUserCookie(
userID: string,		userID: string,
params: UserCookieCreationParams,		params: UserCookieCreationParams,
): Promise<UserViewerData> {		): Promise<UserViewerData> {
const { platformDetails, deviceToken } = params;		const { platformDetails, deviceToken, primaryIdentityPublicKey } = params;
const { platform, ...versions } = platformDetails \|\| defaultPlatformDetails;		const { platform, ...versions } = platformDetails \|\| defaultPlatformDetails;
const versionsString =		const versionsString =
Object.keys(versions).length > 0 ? JSON.stringify(versions) : null;		Object.keys(versions).length > 0 ? JSON.stringify(versions) : null;

const time = Date.now();		const time = Date.now();
const cookiePassword = crypto.randomBytes(32).toString('hex');		const cookiePassword = crypto.randomBytes(32).toString('hex');
const cookieHash = bcrypt.hashSync(cookiePassword);		const cookieHash = bcrypt.hashSync(cookiePassword);
const [[cookieID]] = await Promise.all([		const [[cookieID]] = await Promise.all([
createIDs('cookies', 1),		createIDs('cookies', 1),
deviceToken ? clearDeviceToken(deviceToken) : undefined,		deviceToken ? clearDeviceToken(deviceToken) : undefined,
]);		]);

const cookieRow = [		const cookieRow = [
cookieID,		cookieID,
cookieHash,		cookieHash,
userID,		userID,
platform,		platform,
time,		time,
time,		time,
deviceToken,		deviceToken,
versionsString,		versionsString,
		primaryIdentityPublicKey,
];		];
const query = SQL`		const query = SQL`
INSERT INTO cookies(id, hash, user, platform, creation_time, last_used,		INSERT INTO cookies(id, hash, user, platform, creation_time, last_used,
device_token, versions)		device_token, versions, public_key)
VALUES ${[cookieRow]}		VALUES ${[cookieRow]}
`;		`;
await dbQuery(query);		await dbQuery(query);
return {		return {
loggedIn: true,		loggedIn: true,
id: userID,		id: userID,
platformDetails,		platformDetails,
deviceToken,		deviceToken,
▲ Show 20 Lines • Show All 142 Lines • Show Last 20 Lines

[services][backup] AddAttachments 4/4 - moving to blobClosedPublicActions