Page MenuHomePhabricator
Differential D10213

[identity] Introduce types for DeviceList DDB table
ClosedPublic
Actions

Authored by bartek on Dec 7 2023, 12:02 AM.
Tags
None
Referenced Files
Unknown Object (File)
Sat, Mar 15, 4:43 PM
Unknown Object (File)
Feb 24 2025, 3:15 AM
Unknown Object (File)
Feb 20 2025, 10:56 PM
Unknown Object (File)
Feb 1 2025, 2:57 PM
Unknown Object (File)
Feb 1 2025, 2:57 PM
Unknown Object (File)
Feb 1 2025, 2:57 PM
Unknown Object (File)
Feb 1 2025, 2:57 PM
Unknown Object (File)
Jan 31 2025, 3:49 PM
View All 81 Files
Subscribers
ashoat
tomek

Details

Reviewers
michal
varun
Commits
rCOMM93021213f36f: [identity] Introduce types for DeviceList DDB table
Summary

Introduced structs representing rows/items in the identity-devices DDB table.

Depends on D10153

Test Plan

None yet

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

bartek created this revision.Dec 7 2023, 12:02 AM
bartek held this revision as a draft.
Herald added subscribers: tomek, ashoat. · View Herald TranscriptDec 7 2023, 12:02 AM
bartek added a child revision: D10214: [identity] Implement attribute conversions for Device row.Dec 7 2023, 12:05 AM
Harbormaster completed remote builds in B24841: Diff 34363.Dec 7 2023, 12:28 AM
bartek published this revision for review.Dec 7 2023, 1:04 AM
bartek added inline comments.
services/identity/src/constants.rs
69–70 ↗(On Diff #34363)

Used in D10216

services/identity/src/database.rs
52–53 ↗(On Diff #34363)

I've decided to create a submodule of database just not to mess this file more (it has 1.5k lines already and my code is 600 more in total)

services/identity/src/database/device_list.rs
25–33 ↗(On Diff #34363)

Wondering if it wouldn't be better to introduce types like IdentityKeyInfo and PreKey and make this a nested structure:

struct PreKey {
  prekey: String,
  prekey_signature: String
}

struct IdentityKeyInfo {
  key_payload: String,
  key_payload_signature: String,
  social_proof: Option<String>,
}

struct DeviceRow {
  pub user_id: String,
  pub device_id: String,
  pub device_type: DeviceType,
  pub identity_keys: IdentityKeyInfo,
  pub content_prekey: PreKey,
  pub notif_prekey: PreKey,
}

We could use proto-generated types directly, but I found this in our code - probably could reuse this and add From traits for AttributeValue (see two child diffs of this).

From the perspective of DDB structure, this doesn't matter (we're not going to index on keys so no downsides).
What do you think?

bartek mentioned this in D10216: identity] Add function to get current device list timestamp.Dec 7 2023, 1:06 AM
varun accepted this revision.Dec 7 2023, 8:25 PM
varun added inline comments.
services/identity/src/database/device_list.rs
25–33 ↗(On Diff #34363)

nested structure makes sense to me

This revision is now accepted and ready to land.Dec 7 2023, 8:25 PM
bartek updated this revision to Diff 34498.Dec 11 2023, 11:27 PM

Use nested types for prekeys / identity keys

Harbormaster completed remote builds in B24960: Diff 34498.Dec 11 2023, 11:45 PM
Closed by commit rCOMM93021213f36f: [identity] Introduce types for DeviceList DDB table (authored by bartek). · Explain WhyJan 5 2024, 1:22 AM
This revision was automatically updated to reflect the committed changes.
bartek added a commit: rCOMM93021213f36f: [identity] Introduce types for DeviceList DDB table.

Revision Contents
Changeset List

PathSize
services/
identity/
src/
36 lines
2 lines
database/
47 lines

Diff 35267

View Options

services/identity/src/constants.rs

  • This file was added.
use std::str::FromStr;
use commtest::identity::device::{
login_user_device, logout_user_device, register_user_device, DEVICE_TYPE,
PLACEHOLDER_CODE_VERSION,
};
use commtest::service_addr;
use grpc_clients::identity::get_auth_client;
use grpc_clients::identity::protos::authenticated::GetDeviceListRequest;
use grpc_clients::identity::DeviceType;
use serde::Deserialize;
// 1. register user with android device
// 2. register a web device
// 3. remove android device
// 4. register ios device
// 5. get device list - should have 4 updates:
// - [android]
// - [android, web]
// - [web]
// - [ios, web] - mobile should be first
#[tokio::test]
async fn test_device_list_updates() {
use commtest::identity::olm_account_infos::{
DEFAULT_CLIENT_KEYS as DEVICE_KEYS_ANDROID,
MOCK_CLIENT_KEYS_1 as DEVICE_KEYS_WEB,
MOCK_CLIENT_KEYS_2 as DEVICE_KEYS_IOS,
};
// Create viewer (user that doesn't change devices)
let viewer = register_user_device(None, None).await;
let mut auth_client = get_auth_client(
&service_addr::IDENTITY_GRPC.to_string(),
viewer.user_id.clone(),
viewer.device_id,
viewer.access_token,
PLACEHOLDER_CODE_VERSION,
DEVICE_TYPE.to_string(),
)
.await
.expect("Couldn't connect to identity service");
let android_device_id =
&DEVICE_KEYS_ANDROID.primary_identity_public_keys.ed25519;
let web_device_id = &DEVICE_KEYS_WEB.primary_identity_public_keys.ed25519;
let ios_device_id = &DEVICE_KEYS_IOS.primary_identity_public_keys.ed25519;
// 1. Register user with primary Android device
let android =
register_user_device(Some(&DEVICE_KEYS_ANDROID), Some(DeviceType::Android))
.await;
let user_id = android.user_id.clone();
let username = android.username.clone();
// 2. Log in a web device
let _web =
login_user_device(&username, Some(&DEVICE_KEYS_WEB), Some(DeviceType::Web))
.await;
// 3. Remove android device
logout_user_device(android).await;
// 4. Log in an iOS device
let _ios =
login_user_device(&username, Some(&DEVICE_KEYS_IOS), Some(DeviceType::Ios))
.await;
// Get device list updates for the user
let request = GetDeviceListRequest {
user_id,
since_timestamp: None,
};
let response = auth_client
.get_device_list_for_user(request)
.await
.expect("Get device list request failed")
.into_inner();
let device_lists_response: Vec<Vec<String>> = response
.device_list_updates
.into_iter()
.map(|update| {
let update: DeviceListUpdate = serde_json::from_str(&update)
.expect("Failed to parse device list update");
let device_list: RawDeviceList = update
.raw_device_list
.parse()
.expect("Failed to parse raw device list");
device_list.devices
})
.collect();
let expected_device_list: Vec<Vec<String>> = vec![
vec![android_device_id.into()],
vec![android_device_id.into(), web_device_id.into()],
vec![web_device_id.into()],
vec![ios_device_id.into(), web_device_id.into()],
];
assert_eq!(device_lists_response, expected_device_list);
}
// See GetDeviceListResponse in identity_authenticated.proto
// for details on the response format.
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
#[allow(unused)]
struct RawDeviceList {
devices: Vec<String>,
timestamp: i64,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
struct DeviceListUpdate {
raw_device_list: String,
}
impl FromStr for RawDeviceList {
type Err = serde_json::Error;
fn from_str(s: &str) -> Result<Self, Self::Err> {
// The device list payload is sent as an escaped JSON payload.
// Escaped double quotes need to be trimmed before attempting to deserialize
serde_json::from_str(&s.replace(r#"\""#, r#"""#))
}
}
View Options

services/identity/src/database.rs

  • This file was added.
use std::str::FromStr;
use commtest::identity::device::{
login_user_device, logout_user_device, register_user_device, DEVICE_TYPE,
PLACEHOLDER_CODE_VERSION,
};
use commtest::service_addr;
use grpc_clients::identity::get_auth_client;
use grpc_clients::identity::protos::authenticated::GetDeviceListRequest;
use grpc_clients::identity::DeviceType;
use serde::Deserialize;
// 1. register user with android device
// 2. register a web device
// 3. remove android device
// 4. register ios device
// 5. get device list - should have 4 updates:
// - [android]
// - [android, web]
// - [web]
// - [ios, web] - mobile should be first
#[tokio::test]
async fn test_device_list_updates() {
use commtest::identity::olm_account_infos::{
DEFAULT_CLIENT_KEYS as DEVICE_KEYS_ANDROID,
MOCK_CLIENT_KEYS_1 as DEVICE_KEYS_WEB,
MOCK_CLIENT_KEYS_2 as DEVICE_KEYS_IOS,
};
// Create viewer (user that doesn't change devices)
let viewer = register_user_device(None, None).await;
let mut auth_client = get_auth_client(
&service_addr::IDENTITY_GRPC.to_string(),
viewer.user_id.clone(),
viewer.device_id,
viewer.access_token,
PLACEHOLDER_CODE_VERSION,
DEVICE_TYPE.to_string(),
)
.await
.expect("Couldn't connect to identity service");
let android_device_id =
&DEVICE_KEYS_ANDROID.primary_identity_public_keys.ed25519;
let web_device_id = &DEVICE_KEYS_WEB.primary_identity_public_keys.ed25519;
let ios_device_id = &DEVICE_KEYS_IOS.primary_identity_public_keys.ed25519;
// 1. Register user with primary Android device
let android =
register_user_device(Some(&DEVICE_KEYS_ANDROID), Some(DeviceType::Android))
.await;
let user_id = android.user_id.clone();
let username = android.username.clone();
// 2. Log in a web device
let _web =
login_user_device(&username, Some(&DEVICE_KEYS_WEB), Some(DeviceType::Web))
.await;
// 3. Remove android device
logout_user_device(android).await;
// 4. Log in an iOS device
let _ios =
login_user_device(&username, Some(&DEVICE_KEYS_IOS), Some(DeviceType::Ios))
.await;
// Get device list updates for the user
let request = GetDeviceListRequest {
user_id,
since_timestamp: None,
};
let response = auth_client
.get_device_list_for_user(request)
.await
.expect("Get device list request failed")
.into_inner();
let device_lists_response: Vec<Vec<String>> = response
.device_list_updates
.into_iter()
.map(|update| {
let update: DeviceListUpdate = serde_json::from_str(&update)
.expect("Failed to parse device list update");
let device_list: RawDeviceList = update
.raw_device_list
.parse()
.expect("Failed to parse raw device list");
device_list.devices
})
.collect();
let expected_device_list: Vec<Vec<String>> = vec![
vec![android_device_id.into()],
vec![android_device_id.into(), web_device_id.into()],
vec![web_device_id.into()],
vec![ios_device_id.into(), web_device_id.into()],
];
assert_eq!(device_lists_response, expected_device_list);
}
// See GetDeviceListResponse in identity_authenticated.proto
// for details on the response format.
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
#[allow(unused)]
struct RawDeviceList {
devices: Vec<String>,
timestamp: i64,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
struct DeviceListUpdate {
raw_device_list: String,
}
impl FromStr for RawDeviceList {
type Err = serde_json::Error;
fn from_str(s: &str) -> Result<Self, Self::Err> {
// The device list payload is sent as an escaped JSON payload.
// Escaped double quotes need to be trimmed before attempting to deserialize
serde_json::from_str(&s.replace(r#"\""#, r#"""#))
}
}
View Options

services/identity/src/database/device_list.rs

  • This file was added.
use std::str::FromStr;
use commtest::identity::device::{
login_user_device, logout_user_device, register_user_device, DEVICE_TYPE,
PLACEHOLDER_CODE_VERSION,
};
use commtest::service_addr;
use grpc_clients::identity::get_auth_client;
use grpc_clients::identity::protos::authenticated::GetDeviceListRequest;
use grpc_clients::identity::DeviceType;
use serde::Deserialize;
// 1. register user with android device
// 2. register a web device
// 3. remove android device
// 4. register ios device
// 5. get device list - should have 4 updates:
// - [android]
// - [android, web]
// - [web]
// - [ios, web] - mobile should be first
#[tokio::test]
async fn test_device_list_updates() {
use commtest::identity::olm_account_infos::{
DEFAULT_CLIENT_KEYS as DEVICE_KEYS_ANDROID,
MOCK_CLIENT_KEYS_1 as DEVICE_KEYS_WEB,
MOCK_CLIENT_KEYS_2 as DEVICE_KEYS_IOS,
};
// Create viewer (user that doesn't change devices)
let viewer = register_user_device(None, None).await;
let mut auth_client = get_auth_client(
&service_addr::IDENTITY_GRPC.to_string(),
viewer.user_id.clone(),
viewer.device_id,
viewer.access_token,
PLACEHOLDER_CODE_VERSION,
DEVICE_TYPE.to_string(),
)
.await
.expect("Couldn't connect to identity service");
let android_device_id =
&DEVICE_KEYS_ANDROID.primary_identity_public_keys.ed25519;
let web_device_id = &DEVICE_KEYS_WEB.primary_identity_public_keys.ed25519;
let ios_device_id = &DEVICE_KEYS_IOS.primary_identity_public_keys.ed25519;
// 1. Register user with primary Android device
let android =
register_user_device(Some(&DEVICE_KEYS_ANDROID), Some(DeviceType::Android))
.await;
let user_id = android.user_id.clone();
let username = android.username.clone();
// 2. Log in a web device
let _web =
login_user_device(&username, Some(&DEVICE_KEYS_WEB), Some(DeviceType::Web))
.await;
// 3. Remove android device
logout_user_device(android).await;
// 4. Log in an iOS device
let _ios =
login_user_device(&username, Some(&DEVICE_KEYS_IOS), Some(DeviceType::Ios))
.await;
// Get device list updates for the user
let request = GetDeviceListRequest {
user_id,
since_timestamp: None,
};
let response = auth_client
.get_device_list_for_user(request)
.await
.expect("Get device list request failed")
.into_inner();
let device_lists_response: Vec<Vec<String>> = response
.device_list_updates
.into_iter()
.map(|update| {
let update: DeviceListUpdate = serde_json::from_str(&update)
.expect("Failed to parse device list update");
let device_list: RawDeviceList = update
.raw_device_list
.parse()
.expect("Failed to parse raw device list");
device_list.devices
})
.collect();
let expected_device_list: Vec<Vec<String>> = vec![
vec![android_device_id.into()],
vec![android_device_id.into(), web_device_id.into()],
vec![web_device_id.into()],
vec![ios_device_id.into(), web_device_id.into()],
];
assert_eq!(device_lists_response, expected_device_list);
}
// See GetDeviceListResponse in identity_authenticated.proto
// for details on the response format.
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
#[allow(unused)]
struct RawDeviceList {
devices: Vec<String>,
timestamp: i64,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
struct DeviceListUpdate {
raw_device_list: String,
}
impl FromStr for RawDeviceList {
type Err = serde_json::Error;
fn from_str(s: &str) -> Result<Self, Self::Err> {
// The device list payload is sent as an escaped JSON payload.
// Escaped double quotes need to be trimmed before attempting to deserialize
serde_json::from_str(&s.replace(r#"\""#, r#"""#))
}
}