Page MenuHomePhabricator
Differential D9157

[lib] Add blobServiceUpload upload action
ClosedPublic
Actions

Authored by • patryk on Sep 12 2023, 1:53 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sat, Mar 8, 8:01 PM
Unknown Object (File)
Wed, Mar 5, 5:06 PM
Unknown Object (File)
Wed, Mar 5, 5:06 PM
Unknown Object (File)
Wed, Mar 5, 5:06 PM
Unknown Object (File)
Wed, Mar 5, 5:06 PM
Unknown Object (File)
Wed, Mar 5, 5:06 PM
Unknown Object (File)
Wed, Mar 5, 5:05 PM
Unknown Object (File)
Wed, Mar 5, 4:59 PM
View All Files
Subscribers
ashoat
tomek

Details

Reviewers
bartek
atul
Commits
rCOMM0be4d7f5a5af: [lib] Add blobServiceUpload upload action
Summary

This diff introduces new upload action for blob service uploads.

Depends on D9154.

Test Plan

Tested later in the stack.

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

• patryk created this revision.Sep 12 2023, 1:53 PM
• patryk held this revision as a draft.
Herald added subscribers: tomek, ashoat. · View Herald TranscriptSep 12 2023, 1:53 PM
Harbormaster failed remote builds in B22546: Diff 30992!Sep 12 2023, 2:08 PM
• patryk added a child revision: D9159: [web] Use blobServiceUpload upload action.Sep 13 2023, 2:48 AM
• patryk added a child revision: D9163: [native] Use blobServiceUpload upload action.
• patryk edited the summary of this revision. (Show Details)Sep 13 2023, 2:55 AM
• patryk edited the test plan for this revision. (Show Details)
• patryk added a parent revision: D9154: [lib] Add uuid dependency.
• patryk updated this revision to Diff 31027.Sep 13 2023, 3:27 AM

Rebase

Harbormaster failed remote builds in B22576: Diff 31027!Sep 13 2023, 3:42 AM
bartek added a comment.Sep 14 2023, 4:00 AM

I'm not sure what's wrong about the uuid dependency (failed CI). Perhaps the wrong version is hoisted, which is strange

bartek added inline comments.Sep 14 2023, 4:07 AM
lib/actions/upload-actions.js
128 ↗(On Diff #31027)

arguably we could name it blobSource or blobInput, but I don't have strong preference

• patryk published this revision for review.Sep 14 2023, 4:46 AM
bartek requested changes to this revision.Sep 18 2023, 5:11 AM

Requesting changes to fix failing CI - most likely wrong uuid version is being resolved, try the yarn why uuid command to confirm.

lib/utils/blob-service-upload.js
18 ↗(On Diff #31027)

A good thing would be to destructure options to avoid repeatitions

const { timeout, onProgress, abortHandler } = options ?? {};

// ...
if (timeout) { ... }
This revision now requires changes to proceed.Sep 18 2023, 5:11 AM
• patryk updated this revision to Diff 31197.Sep 18 2023, 5:32 AM

Destructure options.

Harbormaster failed remote builds in B22670: Diff 31197!Sep 18 2023, 5:48 AM
• patryk updated this revision to Diff 31200.Sep 18 2023, 7:15 AM

Failing CI fix

Harbormaster completed remote builds in B22673: Diff 31200.Sep 18 2023, 7:41 AM
bartek accepted this revision.Sep 20 2023, 3:15 AM
This revision is now accepted and ready to land.Sep 20 2023, 3:15 AM
Closed by commit rCOMM0be4d7f5a5af: [lib] Add blobServiceUpload upload action (authored by • patryk). · Explain WhySep 28 2023, 12:56 AM
This revision was automatically updated to reflect the committed changes.
• patryk added a commit: rCOMM0be4d7f5a5af: [lib] Add blobServiceUpload upload action.

Revision Contents
Changeset List

PathSize
lib/
actions/
126 lines
utils/
84 lines

Diff 31197

View Options

lib/actions/upload-actions.js

// @flow // @flow
import type { Utility as OlmUtility } from '@commapp/olm'; import type { Utility as OlmUtility } from '@commapp/olm';
import invariant from 'invariant'; import invariant from 'invariant';
import { ErrorTypes, SiweMessage } from 'siwe'; import { ErrorTypes, SiweMessage } from 'siwe';
import t from 'tcomb'; import t, { type TInterface } from 'tcomb';
import bcrypt from 'twin-bcrypt'; import bcrypt from 'twin-bcrypt';
import { baseLegalPolicies, policies } from 'lib/facts/policies.js'; import {
baseLegalPolicies,
policies,
policyTypeValidator,
} from 'lib/facts/policies.js';
import { hasMinCodeVersion } from 'lib/shared/version-utils.js'; import { hasMinCodeVersion } from 'lib/shared/version-utils.js';
import type { import type {
ResetPasswordRequest, ResetPasswordRequest,
LogOutResponse, LogOutResponse,
DeleteAccountRequest, DeleteAccountRequest,
RegisterResponse, RegisterResponse,
RegisterRequest, RegisterRequest,
LogInResponse, LogInResponse,
Show All 10 Lines
import type { import type {
ClientAvatar, ClientAvatar,
UpdateUserAvatarRequest, UpdateUserAvatarRequest,
} from 'lib/types/avatar-types.js'; } from 'lib/types/avatar-types.js';
import type { import type {
IdentityKeysBlob, IdentityKeysBlob,
SignedIdentityKeysBlob, SignedIdentityKeysBlob,
} from 'lib/types/crypto-types.js'; } from 'lib/types/crypto-types.js';
import type { CalendarQuery } from 'lib/types/entry-types.js'; import {
import { defaultNumberPerThread } from 'lib/types/message-types.js'; type CalendarQuery,
rawEntryInfoValidator,
} from 'lib/types/entry-types.js';
import {
defaultNumberPerThread,
rawMessageInfoValidator,
messageTruncationStatusesValidator,
} from 'lib/types/message-types.js';
import type { import type {
SIWEAuthRequest, SIWEAuthRequest,
SIWEMessage, SIWEMessage,
SIWESocialProof, SIWESocialProof,
} from 'lib/types/siwe-types.js'; } from 'lib/types/siwe-types.js';
import type { import type {
SubscriptionUpdateRequest, SubscriptionUpdateRequest,
SubscriptionUpdateResponse, SubscriptionUpdateResponse,
} from 'lib/types/subscription-types.js'; } from 'lib/types/subscription-types.js';
import type { PasswordUpdate } from 'lib/types/user-types.js'; import { rawThreadInfoValidator } from 'lib/types/thread-types.js';
import {
type PasswordUpdate,
loggedOutUserInfoValidator,
loggedInUserInfoValidator,
oldLoggedInUserInfoValidator,
userInfoValidator,
} from 'lib/types/user-types.js';
import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js'; import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js';
import { import {
identityKeysBlobValidator, identityKeysBlobValidator,
signedIdentityKeysBlobValidator, signedIdentityKeysBlobValidator,
} from 'lib/utils/crypto-utils.js'; } from 'lib/utils/crypto-utils.js';
import { ServerError } from 'lib/utils/errors.js'; import { ServerError } from 'lib/utils/errors.js';
import { values } from 'lib/utils/objects.js'; import { values } from 'lib/utils/objects.js';
import { promiseAll } from 'lib/utils/promises.js'; import { promiseAll } from 'lib/utils/promises.js';
import { import {
getPublicKeyFromSIWEStatement, getPublicKeyFromSIWEStatement,
isValidSIWEMessage, isValidSIWEMessage,
isValidSIWEStatementWithPublicKey, isValidSIWEStatementWithPublicKey,
primaryIdentityPublicKeyRegex, primaryIdentityPublicKeyRegex,
} from 'lib/utils/siwe-utils.js'; } from 'lib/utils/siwe-utils.js';
import { import {
tShape, tShape,
tPlatformDetails, tPlatformDetails,
tPassword, tPassword,
tEmail, tEmail,
tOldValidUsername, tOldValidUsername,
tRegex, tRegex,
tID,
} from 'lib/utils/validation-utils.js'; } from 'lib/utils/validation-utils.js';
import { import {
entryQueryInputValidator, entryQueryInputValidator,
newEntryQueryInputValidator, newEntryQueryInputValidator,
normalizeCalendarQuery, normalizeCalendarQuery,
verifyCalendarQueryThreadIDs, verifyCalendarQueryThreadIDs,
} from './entry-responders.js'; } from './entry-responders.js';
▲ Show 20 Lines • Show All 105 Lines • ▼ Show 20 Lines return {
}, },
}; };
} }
const deleteAccountRequestInputValidator = tShape({ const deleteAccountRequestInputValidator = tShape({
password: t.maybe(tPassword), password: t.maybe(tPassword),
}); });
export const logOutResponseValidator: TInterface<LogOutResponse> =
tShape<LogOutResponse>({
currentUserInfo: loggedOutUserInfoValidator,
});
async function accountDeletionResponder( async function accountDeletionResponder(
viewer: Viewer, viewer: Viewer,
input: any, input: any,
): Promise<LogOutResponse> { ): Promise<LogOutResponse> {
const request: DeleteAccountRequest = input; const request: DeleteAccountRequest = input;
await validateInput(viewer, deleteAccountRequestInputValidator, request); await validateInput(viewer, deleteAccountRequestInputValidator, request);
const result = await deleteAccount(viewer, request); const result = await deleteAccount(viewer, request);
invariant(result, 'deleteAccount should return result if handed request'); invariant(result, 'deleteAccount should return result if handed request');
Show All 13 Linesconst registerRequestInputValidator = tShape({
deviceTokenUpdateRequest: t.maybe(deviceTokenUpdateRequestInputValidator), deviceTokenUpdateRequest: t.maybe(deviceTokenUpdateRequestInputValidator),
platformDetails: tPlatformDetails, platformDetails: tPlatformDetails,
// We include `primaryIdentityPublicKey` to avoid breaking // We include `primaryIdentityPublicKey` to avoid breaking
// old clients, but we no longer do anything with it. // old clients, but we no longer do anything with it.
primaryIdentityPublicKey: t.maybe(tRegex(primaryIdentityPublicKeyRegex)), primaryIdentityPublicKey: t.maybe(tRegex(primaryIdentityPublicKeyRegex)),
signedIdentityKeysBlob: t.maybe(signedIdentityKeysBlobValidator), signedIdentityKeysBlob: t.maybe(signedIdentityKeysBlobValidator),
}); });
export const registerResponseValidator: TInterface<RegisterResponse> =
tShape<RegisterResponse>({
id: t.String,
rawMessageInfos: t.list(rawMessageInfoValidator),
currentUserInfo: t.union([
oldLoggedInUserInfoValidator,
loggedInUserInfoValidator,
]),
cookieChange: tShape({
threadInfos: t.dict(t.String, rawThreadInfoValidator),
userInfos: t.list(userInfoValidator),
}),
});
async function accountCreationResponder( async function accountCreationResponder(
viewer: Viewer, viewer: Viewer,
input: any, input: any,
): Promise<RegisterResponse> { ): Promise<RegisterResponse> {
const request: RegisterRequest = input; const request: RegisterRequest = input;
await validateInput(viewer, registerRequestInputValidator, request); await validateInput(viewer, registerRequestInputValidator, request);
const { signedIdentityKeysBlob } = request; const { signedIdentityKeysBlob } = request;
if (signedIdentityKeysBlob) { if (signedIdentityKeysBlob) {
▲ Show 20 Lines • Show All 131 Lines • ▼ Show 20 Linesconst logInRequestInputValidator = tShape({
platformDetails: tPlatformDetails, platformDetails: tPlatformDetails,
source: t.maybe(t.enums.of(values(logInActionSources))), source: t.maybe(t.enums.of(values(logInActionSources))),
// We include `primaryIdentityPublicKey` to avoid breaking // We include `primaryIdentityPublicKey` to avoid breaking
// old clients, but we no longer do anything with it. // old clients, but we no longer do anything with it.
primaryIdentityPublicKey: t.maybe(tRegex(primaryIdentityPublicKeyRegex)), primaryIdentityPublicKey: t.maybe(tRegex(primaryIdentityPublicKeyRegex)),
signedIdentityKeysBlob: t.maybe(signedIdentityKeysBlobValidator), signedIdentityKeysBlob: t.maybe(signedIdentityKeysBlobValidator),
}); });
export const logInResponseValidator: TInterface<LogInResponse> =
tShape<LogInResponse>({
currentUserInfo: t.union([
loggedInUserInfoValidator,
oldLoggedInUserInfoValidator,
]),
rawMessageInfos: t.list(rawMessageInfoValidator),
truncationStatuses: messageTruncationStatusesValidator,
userInfos: t.list(userInfoValidator),
rawEntryInfos: t.maybe(t.list(rawEntryInfoValidator)),
serverTime: t.Number,
cookieChange: tShape({
threadInfos: t.dict(tID, rawThreadInfoValidator),
userInfos: t.list(userInfoValidator),
}),
notAcknowledgedPolicies: t.maybe(t.list(policyTypeValidator)),
});
async function logInResponder( async function logInResponder(
viewer: Viewer, viewer: Viewer,
input: any, input: any,
): Promise<LogInResponse> { ): Promise<LogInResponse> {
await validateInput(viewer, logInRequestInputValidator, input); await validateInput(viewer, logInRequestInputValidator, input);
const request: LogInRequest = input; const request: LogInRequest = input;
let identityKeys: ?IdentityKeysBlob; let identityKeys: ?IdentityKeysBlob;
▲ Show 20 Lines • Show All 281 Lines • Show Last 20 Lines
View Options

lib/utils/blob-service-upload.js

// @flow // @flow
import type { Utility as OlmUtility } from '@commapp/olm'; import type { Utility as OlmUtility } from '@commapp/olm';
import invariant from 'invariant'; import invariant from 'invariant';
import { ErrorTypes, SiweMessage } from 'siwe'; import { ErrorTypes, SiweMessage } from 'siwe';
import t from 'tcomb'; import t, { type TInterface } from 'tcomb';
import bcrypt from 'twin-bcrypt'; import bcrypt from 'twin-bcrypt';
import { baseLegalPolicies, policies } from 'lib/facts/policies.js'; import {
baseLegalPolicies,
policies,
policyTypeValidator,
} from 'lib/facts/policies.js';
import { hasMinCodeVersion } from 'lib/shared/version-utils.js'; import { hasMinCodeVersion } from 'lib/shared/version-utils.js';
import type { import type {
ResetPasswordRequest, ResetPasswordRequest,
LogOutResponse, LogOutResponse,
DeleteAccountRequest, DeleteAccountRequest,
RegisterResponse, RegisterResponse,
RegisterRequest, RegisterRequest,
LogInResponse, LogInResponse,
Show All 10 Lines
import type { import type {
ClientAvatar, ClientAvatar,
UpdateUserAvatarRequest, UpdateUserAvatarRequest,
} from 'lib/types/avatar-types.js'; } from 'lib/types/avatar-types.js';
import type { import type {
IdentityKeysBlob, IdentityKeysBlob,
SignedIdentityKeysBlob, SignedIdentityKeysBlob,
} from 'lib/types/crypto-types.js'; } from 'lib/types/crypto-types.js';
import type { CalendarQuery } from 'lib/types/entry-types.js'; import {
import { defaultNumberPerThread } from 'lib/types/message-types.js'; type CalendarQuery,
rawEntryInfoValidator,
} from 'lib/types/entry-types.js';
import {
defaultNumberPerThread,
rawMessageInfoValidator,
messageTruncationStatusesValidator,
} from 'lib/types/message-types.js';
import type { import type {
SIWEAuthRequest, SIWEAuthRequest,
SIWEMessage, SIWEMessage,
SIWESocialProof, SIWESocialProof,
} from 'lib/types/siwe-types.js'; } from 'lib/types/siwe-types.js';
import type { import type {
SubscriptionUpdateRequest, SubscriptionUpdateRequest,
SubscriptionUpdateResponse, SubscriptionUpdateResponse,
} from 'lib/types/subscription-types.js'; } from 'lib/types/subscription-types.js';
import type { PasswordUpdate } from 'lib/types/user-types.js'; import { rawThreadInfoValidator } from 'lib/types/thread-types.js';
import {
type PasswordUpdate,
loggedOutUserInfoValidator,
loggedInUserInfoValidator,
oldLoggedInUserInfoValidator,
userInfoValidator,
} from 'lib/types/user-types.js';
import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js'; import { updateUserAvatarRequestValidator } from 'lib/utils/avatar-utils.js';
import { import {
identityKeysBlobValidator, identityKeysBlobValidator,
signedIdentityKeysBlobValidator, signedIdentityKeysBlobValidator,
} from 'lib/utils/crypto-utils.js'; } from 'lib/utils/crypto-utils.js';
import { ServerError } from 'lib/utils/errors.js'; import { ServerError } from 'lib/utils/errors.js';
import { values } from 'lib/utils/objects.js'; import { values } from 'lib/utils/objects.js';
import { promiseAll } from 'lib/utils/promises.js'; import { promiseAll } from 'lib/utils/promises.js';
import { import {
getPublicKeyFromSIWEStatement, getPublicKeyFromSIWEStatement,
isValidSIWEMessage, isValidSIWEMessage,
isValidSIWEStatementWithPublicKey, isValidSIWEStatementWithPublicKey,
primaryIdentityPublicKeyRegex, primaryIdentityPublicKeyRegex,
} from 'lib/utils/siwe-utils.js'; } from 'lib/utils/siwe-utils.js';
import { import {
tShape, tShape,
tPlatformDetails, tPlatformDetails,
tPassword, tPassword,
tEmail, tEmail,
tOldValidUsername, tOldValidUsername,
tRegex, tRegex,
tID,
} from 'lib/utils/validation-utils.js'; } from 'lib/utils/validation-utils.js';
import { import {
entryQueryInputValidator, entryQueryInputValidator,
newEntryQueryInputValidator, newEntryQueryInputValidator,
normalizeCalendarQuery, normalizeCalendarQuery,
verifyCalendarQueryThreadIDs, verifyCalendarQueryThreadIDs,
} from './entry-responders.js'; } from './entry-responders.js';
▲ Show 20 Lines • Show All 105 Lines • ▼ Show 20 Lines return {
}, },
}; };
} }
const deleteAccountRequestInputValidator = tShape({ const deleteAccountRequestInputValidator = tShape({
password: t.maybe(tPassword), password: t.maybe(tPassword),
}); });
export const logOutResponseValidator: TInterface<LogOutResponse> =
tShape<LogOutResponse>({
currentUserInfo: loggedOutUserInfoValidator,
});
async function accountDeletionResponder( async function accountDeletionResponder(
viewer: Viewer, viewer: Viewer,
input: any, input: any,
): Promise<LogOutResponse> { ): Promise<LogOutResponse> {
const request: DeleteAccountRequest = input; const request: DeleteAccountRequest = input;
await validateInput(viewer, deleteAccountRequestInputValidator, request); await validateInput(viewer, deleteAccountRequestInputValidator, request);
const result = await deleteAccount(viewer, request); const result = await deleteAccount(viewer, request);
invariant(result, 'deleteAccount should return result if handed request'); invariant(result, 'deleteAccount should return result if handed request');
Show All 13 Linesconst registerRequestInputValidator = tShape({
deviceTokenUpdateRequest: t.maybe(deviceTokenUpdateRequestInputValidator), deviceTokenUpdateRequest: t.maybe(deviceTokenUpdateRequestInputValidator),
platformDetails: tPlatformDetails, platformDetails: tPlatformDetails,
// We include `primaryIdentityPublicKey` to avoid breaking // We include `primaryIdentityPublicKey` to avoid breaking
// old clients, but we no longer do anything with it. // old clients, but we no longer do anything with it.
primaryIdentityPublicKey: t.maybe(tRegex(primaryIdentityPublicKeyRegex)), primaryIdentityPublicKey: t.maybe(tRegex(primaryIdentityPublicKeyRegex)),
signedIdentityKeysBlob: t.maybe(signedIdentityKeysBlobValidator), signedIdentityKeysBlob: t.maybe(signedIdentityKeysBlobValidator),
}); });
export const registerResponseValidator: TInterface<RegisterResponse> =
tShape<RegisterResponse>({
id: t.String,
rawMessageInfos: t.list(rawMessageInfoValidator),
currentUserInfo: t.union([
oldLoggedInUserInfoValidator,
loggedInUserInfoValidator,
]),
cookieChange: tShape({
threadInfos: t.dict(t.String, rawThreadInfoValidator),
userInfos: t.list(userInfoValidator),
}),
});
async function accountCreationResponder( async function accountCreationResponder(
viewer: Viewer, viewer: Viewer,
input: any, input: any,
): Promise<RegisterResponse> { ): Promise<RegisterResponse> {
const request: RegisterRequest = input; const request: RegisterRequest = input;
await validateInput(viewer, registerRequestInputValidator, request); await validateInput(viewer, registerRequestInputValidator, request);
const { signedIdentityKeysBlob } = request; const { signedIdentityKeysBlob } = request;
if (signedIdentityKeysBlob) { if (signedIdentityKeysBlob) {
▲ Show 20 Lines • Show All 131 Lines • ▼ Show 20 Linesconst logInRequestInputValidator = tShape({
platformDetails: tPlatformDetails, platformDetails: tPlatformDetails,
source: t.maybe(t.enums.of(values(logInActionSources))), source: t.maybe(t.enums.of(values(logInActionSources))),
// We include `primaryIdentityPublicKey` to avoid breaking // We include `primaryIdentityPublicKey` to avoid breaking
// old clients, but we no longer do anything with it. // old clients, but we no longer do anything with it.
primaryIdentityPublicKey: t.maybe(tRegex(primaryIdentityPublicKeyRegex)), primaryIdentityPublicKey: t.maybe(tRegex(primaryIdentityPublicKeyRegex)),
signedIdentityKeysBlob: t.maybe(signedIdentityKeysBlobValidator), signedIdentityKeysBlob: t.maybe(signedIdentityKeysBlobValidator),
}); });
export const logInResponseValidator: TInterface<LogInResponse> =
tShape<LogInResponse>({
currentUserInfo: t.union([
loggedInUserInfoValidator,
oldLoggedInUserInfoValidator,
]),
rawMessageInfos: t.list(rawMessageInfoValidator),
truncationStatuses: messageTruncationStatusesValidator,
userInfos: t.list(userInfoValidator),
rawEntryInfos: t.maybe(t.list(rawEntryInfoValidator)),
serverTime: t.Number,
cookieChange: tShape({
threadInfos: t.dict(tID, rawThreadInfoValidator),
userInfos: t.list(userInfoValidator),
}),
notAcknowledgedPolicies: t.maybe(t.list(policyTypeValidator)),
});
async function logInResponder( async function logInResponder(
viewer: Viewer, viewer: Viewer,
input: any, input: any,
): Promise<LogInResponse> { ): Promise<LogInResponse> {
await validateInput(viewer, logInRequestInputValidator, input); await validateInput(viewer, logInRequestInputValidator, input);
const request: LogInRequest = input; const request: LogInRequest = input;
let identityKeys: ?IdentityKeysBlob; let identityKeys: ?IdentityKeysBlob;
▲ Show 20 Lines • Show All 281 Lines • Show Last 20 Lines