Page MenuHomePhabricator
Differential D3415

[services][draft] Blob refactor to async API - Put
AbandonedPublic
Actions

Authored by • karol on Mar 14 2022, 7:19 AM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, Apr 21, 2:21 AM
Unknown Object (File)
Sun, Apr 20, 6:53 PM
Unknown Object (File)
Thu, Apr 17, 10:53 PM
Unknown Object (File)
Thu, Apr 17, 10:10 AM
Unknown Object (File)
Thu, Apr 17, 10:03 AM
Unknown Object (File)
Thu, Apr 17, 1:06 AM
Unknown Object (File)
Wed, Apr 16, 8:30 AM
Unknown Object (File)
Mon, Apr 14, 11:01 PM
View All Files
Subscribers
• adrian
ashoat
atul
• benschac
tomek

Details

Reviewers
tomek
• max
varun
• jim
Summary

Depends on D3248

Put operation in the blob service refactored from sync to async api.

Test Plan
cd services
yarn run-blob-service

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

• karol created this revision.Mar 14 2022, 7:19 AM
Herald added subscribers: • benschac, atul, • adrian and 2 others. · View Herald TranscriptMar 14 2022, 7:19 AM
Harbormaster returned this revision to the author for changes because remote builds failed.Mar 14 2022, 7:20 AM
Harbormaster failed remote builds in B7325: Diff 10326!
• karol retitled this revision from [services] Blob - Put to [services][draft] Blob refactor to async API - Put.Mar 14 2022, 7:24 AM
• karol edited the summary of this revision. (Show Details)
• karol edited the test plan for this revision. (Show Details)
• karol added reviewers: tomek, • max, varun, • jim.
• karol added a parent revision: D3248: [services] Backup - Implementation update.
• karol added a child revision: D3416: [services][draft] Blob refactor to async API - Get.
• karol updated this revision to Diff 10332.Mar 14 2022, 8:23 AM

dummy update

Harbormaster completed remote builds in B7331: Diff 10332.Mar 14 2022, 8:35 AM
• karol requested review of this revision.Mar 14 2022, 8:35 AM
tomek requested changes to this revision.EditedMar 16 2022, 9:53 AM

This code is really hard to read. The logic there is a dense complicated block of code, which would take a lot of time for anyone to read and understand. Could you spend some time refactoring it into e.g. functions (with descriptive names and clearly defined scope)? Or at least, think through this code, check which parts might be confusing and add long comments explaining what's going on.

I can see that there's a draft label here. I guess we've agreed that we should use drafts only for arch review... but I might be wrong. Nevertheless, what's the difference here between draft and non-draft?

This revision now requires changes to proceed.Mar 16 2022, 9:53 AM
• karol abandoned this revision.Mar 21 2022, 2:18 AM

Abandoning, I'm going to put up a new stack for this which will be hopefully easier to review.

Revision Contents
Changeset List

PathSize
services/
blob/
docker-server/
contents/
server/
src/
92 lines
Reactors/
7 lines

Diff 10332

View Options

services/blob/docker-server/contents/server/src/BlobServiceImpl.cpp

// Secrets // Secrets
pub const SECRETS_DIRECTORY: &str = "secrets"; pub const SECRETS_DIRECTORY: &str = "secrets";
pub const SECRETS_FILE_NAME: &str = "secret_key"; pub const SECRETS_FILE_NAME: &str = "secret_key";
pub const SECRETS_FILE_EXTENSION: &str = "txt"; pub const SECRETS_FILE_EXTENSION: &str = "txt";
pub const SECRETS_SETUP_FILE: &str = "server_setup.txt"; pub const SECRETS_SETUP_FILE: &str = "server_setup.txt";
// DynamoDB // DynamoDB
// User table information, supporting opaque_ke 2.0 and X3DH information // User table information, supporting opaque_ke 2.0 and X3DH information
pub mod opaque2 {
// Users can sign in either through username+password or Eth wallet. // Users can sign in either through username+password or Eth wallet.
// //
// This structure should be aligned with the messages defined in // This structure should be aligned with the messages defined in
// shared/protos/identity_client.proto // shared/protos/identity_client.proto
// //
// Structure for a user should be: // Structure for a user should be:
// { // {
// userID: String, // userID: String,
// opaqueRegistrationData: Option<String>, // opaqueRegistrationData: Option<String>,
// username: Option<String>, // username: Option<String>,
// walletAddress: Option<String>, // walletAddress: Option<String>,
// devices: HashMap<String, Device> // devices: HashMap<String, Device>
// } // }
// //
// A device is defined as: // A device is defined as:
// { // {
// deviceType: String, # client or keyserver // deviceType: String, # client or keyserver
// keyPayload: String, // keyPayload: String,
// keyPayloadSignature: String,
// identityPreKey: String, // identityPreKey: String,
// identityPreKeySignature: String, // identityPreKeySignature: String,
// identityOneTimeKeys: Vec<String>, // identityOneTimeKeys: Vec<String>,
// notifPreKey: String, // notifPreKey: String,
// notifPreKeySignature: String, // notifPreKeySignature: String,
// notifOneTimeKeys: Vec<String>, // notifOneTimeKeys: Vec<String>,
// } // }
// } // }
// //
// Additional context: // Additional context:
// "devices" uses the signing public identity key of the device as a key for the devices map // "devices" uses the signing public identity key of the device as a key for the devices map
// "keyPayload" is a JSON encoded string containing identity and notif keys (both signature and verification) // "keyPayload" is a JSON encoded string containing identity and notif keys (both signature and verification)
// if "deviceType" == "keyserver", then the device will not have any notif key information // if "deviceType" == "keyserver", then the device will not have any notif key information
pub const USERS_TABLE: &str = "identity-users-opaque2"; pub const USERS_TABLE: &str = "identity-users";
pub const USERS_TABLE_PARTITION_KEY: &str = "userID"; pub const USERS_TABLE_PARTITION_KEY: &str = "userID";
pub const USERS_TABLE_REGISTRATION_ATTRIBUTE: &str = "opaqueRegistrationData"; pub const USERS_TABLE_REGISTRATION_ATTRIBUTE: &str = "opaqueRegistrationData";
pub const USERS_TABLE_USERNAME_ATTRIBUTE: &str = "username"; pub const USERS_TABLE_USERNAME_ATTRIBUTE: &str = "username";
pub const USERS_TABLE_DEVICES_ATTRIBUTE: &str = "devices"; pub const USERS_TABLE_DEVICES_ATTRIBUTE: &str = "devices";
pub const USERS_TABLE_DEVICES_MAP_DEVICE_TYPE_ATTRIBUTE_NAME: &str =
"deviceType";
pub const USERS_TABLE_DEVICES_MAP_KEY_PAYLOAD_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_KEY_PAYLOAD_ATTRIBUTE_NAME: &str =
"keyPayload"; "keyPayload";
pub const USERS_TABLE_DEVICES_MAP_KEY_PAYLOAD_SIGNATURE_ATTRIBUTE_NAME: &str =
"keyPayloadSignature";
pub const USERS_TABLE_DEVICES_MAP_IDENTITY_PREKEY_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_IDENTITY_PREKEY_ATTRIBUTE_NAME: &str =
"identityPreKey"; "identityPreKey";
pub const USERS_TABLE_DEVICES_MAP_IDENTITY_PREKEY_SIGNATURE_ATTRIBUTE_NAME: pub const USERS_TABLE_DEVICES_MAP_IDENTITY_PREKEY_SIGNATURE_ATTRIBUTE_NAME:
&str = "identityPreKeySignature"; &str = "identityPreKeySignature";
pub const USERS_TABLE_DEVICES_MAP_IDENTITY_ONETIME_KEYS_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_IDENTITY_ONETIME_KEYS_ATTRIBUTE_NAME: &str =
"identityOneTimeKeys"; "identityOneTimeKeys";
pub const USERS_TABLE_DEVICES_MAP_NOTIF_PREKEY_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_NOTIF_PREKEY_ATTRIBUTE_NAME: &str = "preKey";
"preKey"; pub const USERS_TABLE_DEVICES_MAP_NOTIF_PREKEY_SIGNATURE_ATTRIBUTE_NAME: &str =
pub const USERS_TABLE_DEVICES_MAP_NOTIF_PREKEY_SIGNATURE_ATTRIBUTE_NAME: "preKeySignature";
&str = "preKeySignature";
pub const USERS_TABLE_DEVICES_MAP_NOTIF_ONETIME_KEYS_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_NOTIF_ONETIME_KEYS_ATTRIBUTE_NAME: &str =
"notifOneTimeKeys"; "notifOneTimeKeys";
pub const USERS_TABLE_WALLET_ADDRESS_ATTRIBUTE: &str = "walletAddress"; pub const USERS_TABLE_WALLET_ADDRESS_ATTRIBUTE: &str = "walletAddress";
pub const USERS_TABLE_USERNAME_INDEX: &str = "username-index"; pub const USERS_TABLE_USERNAME_INDEX: &str = "username-index";
pub const USERS_TABLE_WALLET_ADDRESS_INDEX: &str = "walletAddress-index"; pub const USERS_TABLE_WALLET_ADDRESS_INDEX: &str = "walletAddress-index";
}
pub const USERS_TABLE: &str = "identity-users";
pub const USERS_TABLE_PARTITION_KEY: &str = "userID";
pub const USERS_TABLE_REGISTRATION_ATTRIBUTE: &str = "pakeRegistrationData";
pub const USERS_TABLE_USERNAME_ATTRIBUTE: &str = "username";
pub const USERS_TABLE_DEVICES_ATTRIBUTE: &str = "devices";
pub const USERS_TABLE_DEVICE_ATTRIBUTE_NAME: &str = "device";
pub const USERS_TABLE_DEVICES_MAP_ATTRIBUTE_NAME: &str = "signingPublicKey";
pub const USERS_TABLE_WALLET_ADDRESS_ATTRIBUTE: &str = "walletAddress";
pub const USERS_TABLE_USERNAME_INDEX: &str = "username-index";
pub const USERS_TABLE_WALLET_ADDRESS_INDEX: &str = "walletAddress-index";
pub const USERS_TABLE_INITIALIZATION_INFO: &str = "initializationInfo";
pub const ACCESS_TOKEN_TABLE: &str = "identity-tokens"; pub const ACCESS_TOKEN_TABLE: &str = "identity-tokens";
pub const ACCESS_TOKEN_TABLE_PARTITION_KEY: &str = "userID"; pub const ACCESS_TOKEN_TABLE_PARTITION_KEY: &str = "userID";
pub const ACCESS_TOKEN_SORT_KEY: &str = "signingPublicKey"; pub const ACCESS_TOKEN_SORT_KEY: &str = "signingPublicKey";
pub const ACCESS_TOKEN_TABLE_CREATED_ATTRIBUTE: &str = "created"; pub const ACCESS_TOKEN_TABLE_CREATED_ATTRIBUTE: &str = "created";
pub const ACCESS_TOKEN_TABLE_AUTH_TYPE_ATTRIBUTE: &str = "authType"; pub const ACCESS_TOKEN_TABLE_AUTH_TYPE_ATTRIBUTE: &str = "authType";
pub const ACCESS_TOKEN_TABLE_VALID_ATTRIBUTE: &str = "valid"; pub const ACCESS_TOKEN_TABLE_VALID_ATTRIBUTE: &str = "valid";
pub const ACCESS_TOKEN_TABLE_TOKEN_ATTRIBUTE: &str = "token"; pub const ACCESS_TOKEN_TABLE_TOKEN_ATTRIBUTE: &str = "token";
Show All 29 Lines
View Options

services/blob/docker-server/contents/server/src/Reactors/BidiReactorBase.h

// Secrets // Secrets
pub const SECRETS_DIRECTORY: &str = "secrets"; pub const SECRETS_DIRECTORY: &str = "secrets";
pub const SECRETS_FILE_NAME: &str = "secret_key"; pub const SECRETS_FILE_NAME: &str = "secret_key";
pub const SECRETS_FILE_EXTENSION: &str = "txt"; pub const SECRETS_FILE_EXTENSION: &str = "txt";
pub const SECRETS_SETUP_FILE: &str = "server_setup.txt"; pub const SECRETS_SETUP_FILE: &str = "server_setup.txt";
// DynamoDB // DynamoDB
// User table information, supporting opaque_ke 2.0 and X3DH information // User table information, supporting opaque_ke 2.0 and X3DH information
pub mod opaque2 {
// Users can sign in either through username+password or Eth wallet. // Users can sign in either through username+password or Eth wallet.
// //
// This structure should be aligned with the messages defined in // This structure should be aligned with the messages defined in
// shared/protos/identity_client.proto // shared/protos/identity_client.proto
// //
// Structure for a user should be: // Structure for a user should be:
// { // {
// userID: String, // userID: String,
// opaqueRegistrationData: Option<String>, // opaqueRegistrationData: Option<String>,
// username: Option<String>, // username: Option<String>,
// walletAddress: Option<String>, // walletAddress: Option<String>,
// devices: HashMap<String, Device> // devices: HashMap<String, Device>
// } // }
// //
// A device is defined as: // A device is defined as:
// { // {
// deviceType: String, # client or keyserver // deviceType: String, # client or keyserver
// keyPayload: String, // keyPayload: String,
// keyPayloadSignature: String,
// identityPreKey: String, // identityPreKey: String,
// identityPreKeySignature: String, // identityPreKeySignature: String,
// identityOneTimeKeys: Vec<String>, // identityOneTimeKeys: Vec<String>,
// notifPreKey: String, // notifPreKey: String,
// notifPreKeySignature: String, // notifPreKeySignature: String,
// notifOneTimeKeys: Vec<String>, // notifOneTimeKeys: Vec<String>,
// } // }
// } // }
// //
// Additional context: // Additional context:
// "devices" uses the signing public identity key of the device as a key for the devices map // "devices" uses the signing public identity key of the device as a key for the devices map
// "keyPayload" is a JSON encoded string containing identity and notif keys (both signature and verification) // "keyPayload" is a JSON encoded string containing identity and notif keys (both signature and verification)
// if "deviceType" == "keyserver", then the device will not have any notif key information // if "deviceType" == "keyserver", then the device will not have any notif key information
pub const USERS_TABLE: &str = "identity-users-opaque2"; pub const USERS_TABLE: &str = "identity-users";
pub const USERS_TABLE_PARTITION_KEY: &str = "userID"; pub const USERS_TABLE_PARTITION_KEY: &str = "userID";
pub const USERS_TABLE_REGISTRATION_ATTRIBUTE: &str = "opaqueRegistrationData"; pub const USERS_TABLE_REGISTRATION_ATTRIBUTE: &str = "opaqueRegistrationData";
pub const USERS_TABLE_USERNAME_ATTRIBUTE: &str = "username"; pub const USERS_TABLE_USERNAME_ATTRIBUTE: &str = "username";
pub const USERS_TABLE_DEVICES_ATTRIBUTE: &str = "devices"; pub const USERS_TABLE_DEVICES_ATTRIBUTE: &str = "devices";
pub const USERS_TABLE_DEVICES_MAP_DEVICE_TYPE_ATTRIBUTE_NAME: &str =
"deviceType";
pub const USERS_TABLE_DEVICES_MAP_KEY_PAYLOAD_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_KEY_PAYLOAD_ATTRIBUTE_NAME: &str =
"keyPayload"; "keyPayload";
pub const USERS_TABLE_DEVICES_MAP_KEY_PAYLOAD_SIGNATURE_ATTRIBUTE_NAME: &str =
"keyPayloadSignature";
pub const USERS_TABLE_DEVICES_MAP_IDENTITY_PREKEY_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_IDENTITY_PREKEY_ATTRIBUTE_NAME: &str =
"identityPreKey"; "identityPreKey";
pub const USERS_TABLE_DEVICES_MAP_IDENTITY_PREKEY_SIGNATURE_ATTRIBUTE_NAME: pub const USERS_TABLE_DEVICES_MAP_IDENTITY_PREKEY_SIGNATURE_ATTRIBUTE_NAME:
&str = "identityPreKeySignature"; &str = "identityPreKeySignature";
pub const USERS_TABLE_DEVICES_MAP_IDENTITY_ONETIME_KEYS_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_IDENTITY_ONETIME_KEYS_ATTRIBUTE_NAME: &str =
"identityOneTimeKeys"; "identityOneTimeKeys";
pub const USERS_TABLE_DEVICES_MAP_NOTIF_PREKEY_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_NOTIF_PREKEY_ATTRIBUTE_NAME: &str = "preKey";
"preKey"; pub const USERS_TABLE_DEVICES_MAP_NOTIF_PREKEY_SIGNATURE_ATTRIBUTE_NAME: &str =
pub const USERS_TABLE_DEVICES_MAP_NOTIF_PREKEY_SIGNATURE_ATTRIBUTE_NAME: "preKeySignature";
&str = "preKeySignature";
pub const USERS_TABLE_DEVICES_MAP_NOTIF_ONETIME_KEYS_ATTRIBUTE_NAME: &str = pub const USERS_TABLE_DEVICES_MAP_NOTIF_ONETIME_KEYS_ATTRIBUTE_NAME: &str =
"notifOneTimeKeys"; "notifOneTimeKeys";
pub const USERS_TABLE_WALLET_ADDRESS_ATTRIBUTE: &str = "walletAddress"; pub const USERS_TABLE_WALLET_ADDRESS_ATTRIBUTE: &str = "walletAddress";
pub const USERS_TABLE_USERNAME_INDEX: &str = "username-index"; pub const USERS_TABLE_USERNAME_INDEX: &str = "username-index";
pub const USERS_TABLE_WALLET_ADDRESS_INDEX: &str = "walletAddress-index"; pub const USERS_TABLE_WALLET_ADDRESS_INDEX: &str = "walletAddress-index";
}
pub const USERS_TABLE: &str = "identity-users";
pub const USERS_TABLE_PARTITION_KEY: &str = "userID";
pub const USERS_TABLE_REGISTRATION_ATTRIBUTE: &str = "pakeRegistrationData";
pub const USERS_TABLE_USERNAME_ATTRIBUTE: &str = "username";
pub const USERS_TABLE_DEVICES_ATTRIBUTE: &str = "devices";
pub const USERS_TABLE_DEVICE_ATTRIBUTE_NAME: &str = "device";
pub const USERS_TABLE_DEVICES_MAP_ATTRIBUTE_NAME: &str = "signingPublicKey";
pub const USERS_TABLE_WALLET_ADDRESS_ATTRIBUTE: &str = "walletAddress";
pub const USERS_TABLE_USERNAME_INDEX: &str = "username-index";
pub const USERS_TABLE_WALLET_ADDRESS_INDEX: &str = "walletAddress-index";
pub const USERS_TABLE_INITIALIZATION_INFO: &str = "initializationInfo";
pub const ACCESS_TOKEN_TABLE: &str = "identity-tokens"; pub const ACCESS_TOKEN_TABLE: &str = "identity-tokens";
pub const ACCESS_TOKEN_TABLE_PARTITION_KEY: &str = "userID"; pub const ACCESS_TOKEN_TABLE_PARTITION_KEY: &str = "userID";
pub const ACCESS_TOKEN_SORT_KEY: &str = "signingPublicKey"; pub const ACCESS_TOKEN_SORT_KEY: &str = "signingPublicKey";
pub const ACCESS_TOKEN_TABLE_CREATED_ATTRIBUTE: &str = "created"; pub const ACCESS_TOKEN_TABLE_CREATED_ATTRIBUTE: &str = "created";
pub const ACCESS_TOKEN_TABLE_AUTH_TYPE_ATTRIBUTE: &str = "authType"; pub const ACCESS_TOKEN_TABLE_AUTH_TYPE_ATTRIBUTE: &str = "authType";
pub const ACCESS_TOKEN_TABLE_VALID_ATTRIBUTE: &str = "valid"; pub const ACCESS_TOKEN_TABLE_VALID_ATTRIBUTE: &str = "valid";
pub const ACCESS_TOKEN_TABLE_TOKEN_ATTRIBUTE: &str = "token"; pub const ACCESS_TOKEN_TABLE_TOKEN_ATTRIBUTE: &str = "token";
Show All 29 Lines