Paths

Table of Contentst

-
lib/permissions/
-
permissions/
-
minimally-encoded-raw-thread-info-validators.js
-
minimally-encoded-raw-thread-info-validators.test.js

[lib] Introduce `memberInfoSansPermissionsValidator` and consume in `rawThreadInfoValidator`
ClosedPublic
Actions

Authored by atul on Jul 15 2024, 12:56 PM.

Details

Reviewers

ashoat
tomek

Commits

rCOMM5029a29dfb30: [lib] Introduce `memberInfoSansPermissionsValidator` and consume in…

Summary

Introducing validator, unit tests, and consuming in rawThreadInfoValidator which fixes the validator test case which was failing.

This also expands validator so it'll support memberInfos both with and without permissions.

Depends on D12756

Test Plan

Unit tests

Diff Detail

Repository

rCOMM Comm

Branch

july9 (branched from master)

Lint

No Lint Coverage

Unit

No Test Coverage

Event Timeline

atul created this revision.Jul 15 2024, 12:56 PM

atul published this revision for review.Jul 15 2024, 1:04 PM

Harbormaster completed remote builds in B30339: Diff 42301.Jul 15 2024, 1:12 PM

atul added a child revision: D12763: [lib] Introduce `stripMemberPermissionsFromRawThreadInfo`.Jul 15 2024, 6:32 PM

ashoat mentioned this in D12756: [lib] Move `stripPermissionsFromMemberInfo` out of `...-types.js` file.Jul 17 2024, 12:10 PM

It looks like these validators are used on both the keyserver and the client. It's appropriate for use on the keyserver, which needs to continue supporting older clients.

But it's inappropriate for use on the client, where we don't expect the keyserver to ever respond with the legacy format.

Would love if we could separate these out into two kinds of validators, but it's probably too much work for @atul at this point.

This revision is now accepted and ready to land.Jul 17 2024, 12:17 PM

But it's inappropriate for use on the client, where we don't expect the keyserver to ever respond with the legacy format.

We actually will use this validator on the client. Specifically in convertClientDBThreadInfoToRawThreadInfo where we validate the RawThreadInfo persisted on client. Have a diff that consumes it on client in local stack.

EDIT: To clarify we will consume memberInfoSansPermissionsValidator on the client.

atul mentioned this in D12594: [lib] Change type of `ThinRawThreadInfo.members` from `MemberInfoWithPermissions` to `MemberInfoSansPermissions`.Jul 17 2024, 12:40 PM

Going to go ahead and try cherry-picking this onto master to fix the failing validator unit test and get rest of stack "more green."

cherry-pick off master

Harbormaster completed remote builds in B30429: Diff 42402.Jul 17 2024, 1:06 PM

Closed by commit rCOMM5029a29dfb30: [lib] Introduce `memberInfoSansPermissionsValidator` and consume in… (authored by atul). · Explain WhyJul 17 2024, 1:37 PM

This revision was automatically updated to reflect the committed changes.

atul added a commit: rCOMM5029a29dfb30: [lib] Introduce `memberInfoSansPermissionsValidator` and consume in….

Revision Contents
Changeset List

Path

Size

lib/

permissions/

minimally-encoded-raw-thread-info-validators.js

15 lines

minimally-encoded-raw-thread-info-validators.test.js

30 lines

Diff 42301

View Options

lib/permissions/minimally-encoded-raw-thread-info-validators.js

use std::collections::HashMap;		use std::collections::HashMap;

use chrono::{DateTime, Utc};		use chrono::{DateTime, Utc};
use comm_lib::{		use comm_lib::{
aws::ddb::{		aws::ddb::{
operation::{get_item::GetItemOutput, query::builders::QueryFluentBuilder},		operation::{get_item::GetItemOutput, query::builders::QueryFluentBuilder},
types::{		types::{
error::TransactionCanceledException, AttributeValue, DeleteRequest, Put,		error::TransactionCanceledException, AttributeValue, DeleteRequest, Put,
TransactWriteItem, Update, WriteRequest,		TransactWriteItem, Update, WriteRequest,
},		},
},		},
database::{AttributeMap, DBItemAttributeError, DBItemError, DynamoDBError},		database::{
		AttributeExtractor, AttributeMap, DBItemAttributeError, DBItemError,
		DynamoDBError, TryFromAttribute,
		},
};		};
use tracing::{error, warn};		use tracing::{error, warn};

use crate::{		use crate::{
client_service::FlattenedDeviceKeyUpload,		client_service::FlattenedDeviceKeyUpload,
constants::{		constants::{
devices_table::{self, *},		devices_table::{self, *},
USERS_TABLE, USERS_TABLE_DEVICELIST_TIMESTAMP_ATTRIBUTE_NAME,		USERS_TABLE, USERS_TABLE_DEVICELIST_TIMESTAMP_ATTRIBUTE_NAME,
USERS_TABLE_PARTITION_KEY,		USERS_TABLE_PARTITION_KEY,
},		},
database::parse_string_attribute,
ddb_utils::AttributesOptionExt,		ddb_utils::AttributesOptionExt,
error::{DeviceListError, Error, FromAttributeValue},		error::{DeviceListError, Error, FromAttributeValue},
grpc_services::protos::unauth::DeviceType,		grpc_services::protos::unauth::DeviceType,
};		};

use super::{parse_date_time_attribute, DatabaseClient};		use super::DatabaseClient;

#[derive(Clone, Debug)]		#[derive(Clone, Debug)]
pub struct DeviceRow {		pub struct DeviceRow {
pub user_id: String,		pub user_id: String,
pub device_id: String,		pub device_id: String,
pub device_type: DeviceType,		pub device_type: DeviceType,
pub device_key_info: IdentityKeyInfo,		pub device_key_info: IdentityKeyInfo,
pub content_prekey: PreKey,		pub content_prekey: PreKey,
▲ Show 20 Lines • Show All 76 Lines • ▼ Show 20 Lines	AttributeValue::S(format!(
value.0.to_rfc3339()		value.0.to_rfc3339()
))		))
}		}
}		}

impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute {		impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute {
type Error = DBItemError;		type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {		fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;		let item_id = String::try_from_attr(ATTR_ITEM_ID, value)?;

// remove the device- prefix		// remove the device- prefix
let device_id = item_id		let device_id = item_id
.strip_prefix(DEVICE_ITEM_KEY_PREFIX)		.strip_prefix(DEVICE_ITEM_KEY_PREFIX)
.ok_or_else(\|\| DBItemError {		.ok_or_else(\|\| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(),		attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: item_id.clone().into(),		attribute_value: item_id.clone().into(),
attribute_error: DBItemAttributeError::InvalidValue,		attribute_error: DBItemAttributeError::InvalidValue,
})?		})?
.to_string();		.to_string();

Ok(Self(device_id))		Ok(Self(device_id))
}		}
}		}

impl TryFrom<Option<AttributeValue>> for DeviceListKeyAttribute {		impl TryFrom<Option<AttributeValue>> for DeviceListKeyAttribute {
type Error = DBItemError;		type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {		fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;		let item_id = String::try_from_attr(ATTR_ITEM_ID, value)?;

// remove the device-list- prefix, then parse the timestamp		// remove the device-list- prefix, then parse the timestamp
let timestamp: DateTime<Utc> = item_id		let timestamp: DateTime<Utc> = item_id
.strip_prefix(DEVICE_LIST_KEY_PREFIX)		.strip_prefix(DEVICE_LIST_KEY_PREFIX)
.ok_or_else(\|\| DBItemError {		.ok_or_else(\|\| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(),		attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: item_id.clone().into(),		attribute_value: item_id.clone().into(),
attribute_error: DBItemAttributeError::InvalidValue,		attribute_error: DBItemAttributeError::InvalidValue,
Show All 11 Lines	fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
Ok(Self(timestamp))		Ok(Self(timestamp))
}		}
}		}

impl TryFrom<AttributeMap> for DeviceRow {		impl TryFrom<AttributeMap> for DeviceRow {
type Error = DBItemError;		type Error = DBItemError;

fn try_from(mut attrs: AttributeMap) -> Result<Self, Self::Error> {		fn try_from(mut attrs: AttributeMap) -> Result<Self, Self::Error> {
let user_id =		let user_id = attrs.take_attr(ATTR_USER_ID)?;
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?;		let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?;

let raw_device_type =		let raw_device_type: String = attrs.take_attr(ATTR_DEVICE_TYPE)?;
parse_string_attribute(ATTR_DEVICE_TYPE, attrs.remove(ATTR_DEVICE_TYPE))?;
let device_type =		let device_type =
DeviceType::from_str_name(&raw_device_type).ok_or_else(\|\| {		DeviceType::from_str_name(&raw_device_type).ok_or_else(\|\| {
DBItemError::new(		DBItemError::new(
ATTR_DEVICE_TYPE.to_string(),		ATTR_DEVICE_TYPE.to_string(),
raw_device_type.into(),		raw_device_type.into(),
DBItemAttributeError::InvalidValue,		DBItemAttributeError::InvalidValue,
)		)
})?;		})?;
▲ Show 20 Lines • Show All 72 Lines • ▼ Show 20 Lines	fn from(value: IdentityKeyInfo) -> Self {
}		}
AttributeValue::M(attrs)		AttributeValue::M(attrs)
}		}
}		}

impl TryFrom<AttributeMap> for IdentityKeyInfo {		impl TryFrom<AttributeMap> for IdentityKeyInfo {
type Error = DBItemError;		type Error = DBItemError;
fn try_from(mut attrs: AttributeMap) -> Result<Self, Self::Error> {		fn try_from(mut attrs: AttributeMap) -> Result<Self, Self::Error> {
let key_payload =		let key_payload = attrs.take_attr(ATTR_KEY_PAYLOAD)?;
parse_string_attribute(ATTR_KEY_PAYLOAD, attrs.remove(ATTR_KEY_PAYLOAD))?;		let key_payload_signature = attrs.take_attr(ATTR_KEY_PAYLOAD_SIGNATURE)?;
let key_payload_signature = parse_string_attribute(
ATTR_KEY_PAYLOAD_SIGNATURE,
attrs.remove(ATTR_KEY_PAYLOAD_SIGNATURE),
)?;
// social proof is optional		// social proof is optional
let social_proof = attrs		let social_proof = attrs
.remove(ATTR_SOCIAL_PROOF)		.remove(ATTR_SOCIAL_PROOF)
.map(\|attr\| attr.to_string(ATTR_SOCIAL_PROOF).cloned())		.map(\|attr\| attr.to_string(ATTR_SOCIAL_PROOF).cloned())
.transpose()?;		.transpose()?;

Ok(Self {		Ok(Self {
key_payload,		key_payload,
Show All 14 Lines	fn from(value: PreKey) -> Self {
]);		]);
AttributeValue::M(attrs)		AttributeValue::M(attrs)
}		}
}		}

impl TryFrom<AttributeMap> for PreKey {		impl TryFrom<AttributeMap> for PreKey {
type Error = DBItemError;		type Error = DBItemError;
fn try_from(mut attrs: AttributeMap) -> Result<Self, Self::Error> {		fn try_from(mut attrs: AttributeMap) -> Result<Self, Self::Error> {
let pre_key =		let pre_key = attrs.take_attr(ATTR_PREKEY)?;
parse_string_attribute(ATTR_PREKEY, attrs.remove(ATTR_PREKEY))?;		let pre_key_signature = attrs.take_attr(ATTR_PREKEY_SIGNATURE)?;
let pre_key_signature = parse_string_attribute(
ATTR_PREKEY_SIGNATURE,
attrs.remove(ATTR_PREKEY_SIGNATURE),
)?;
Ok(Self {		Ok(Self {
pre_key,		pre_key,
pre_key_signature,		pre_key_signature,
})		})
}		}
}		}

impl TryFrom<AttributeMap> for DeviceListRow {		impl TryFrom<AttributeMap> for DeviceListRow {
type Error = DBItemError;		type Error = DBItemError;

fn try_from(mut attrs: AttributeMap) -> Result<Self, Self::Error> {		fn try_from(mut attrs: AttributeMap) -> Result<Self, Self::Error> {
let user_id =		let user_id = attrs.take_attr(ATTR_USER_ID)?;
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceListKeyAttribute(timestamp) =		let DeviceListKeyAttribute(timestamp) =
attrs.remove(ATTR_ITEM_ID).try_into()?;		attrs.remove(ATTR_ITEM_ID).try_into()?;

// validate timestamps are in sync		// validate timestamps are in sync
let timestamps_match = attrs		let timestamps_match = attrs
.remove(ATTR_TIMESTAMP)		.remove(ATTR_TIMESTAMP)
.and_then(\|attr\| attr.as_n().ok().cloned())		.and_then(\|attr\| attr.as_n().ok().cloned())
.and_then(\|val\| val.parse::<i64>().ok())		.and_then(\|val\| val.parse::<i64>().ok())
.filter(\|val\| *val == timestamp.timestamp_millis())		.filter(\|val\| *val == timestamp.timestamp_millis())
.is_some();		.is_some();
if !timestamps_match {		if !timestamps_match {
warn!(		warn!(
"DeviceList timestamp mismatch for (userID={}, itemID={})",		"DeviceList timestamp mismatch for (userID={}, itemID={})",
&user_id,		&user_id,
timestamp.to_rfc3339()		timestamp.to_rfc3339()
);		);
}		}

// this should be a list of strings		let device_ids: Vec<String> = attrs.take_attr(ATTR_DEVICE_IDS)?;
let device_ids = attrs
.remove(ATTR_DEVICE_IDS)
.ok_or_else(\|\| {
DBItemError::new(
ATTR_DEVICE_IDS.to_string(),
None.into(),
DBItemAttributeError::Missing,
)
})?
.to_vec(ATTR_DEVICE_IDS)?
.iter()
.map(\|v\| v.to_string("device_ids[?]").cloned())
.collect::<Result<Vec<String>, DBItemError>>()?;

Ok(Self {		Ok(Self {
user_id,		user_id,
timestamp,		timestamp,
device_ids,		device_ids,
})		})
}		}
}		}
▲ Show 20 Lines • Show All 362 Lines • ▼ Show 20 Lines	let raw_datetime =
user_item.remove(USERS_TABLE_DEVICELIST_TIMESTAMP_ATTRIBUTE_NAME);		user_item.remove(USERS_TABLE_DEVICELIST_TIMESTAMP_ATTRIBUTE_NAME);

// existing records will not have this field when		// existing records will not have this field when
// updating device list for the first time		// updating device list for the first time
if raw_datetime.is_none() {		if raw_datetime.is_none() {
return Ok(None);		return Ok(None);
}		}

let timestamp = parse_date_time_attribute(		let timestamp = DateTime::<Utc>::try_from_attr(
USERS_TABLE_DEVICELIST_TIMESTAMP_ATTRIBUTE_NAME,		USERS_TABLE_DEVICELIST_TIMESTAMP_ATTRIBUTE_NAME,
raw_datetime,		raw_datetime,
)?;		)?;
Ok(Some(timestamp))		Ok(Some(timestamp))
}		}

/// Generates update expression for current device list timestamp in users table.		/// Generates update expression for current device list timestamp in users table.
/// The previous timestamp is used as a condition to ensure that the value hasn't changed		/// The previous timestamp is used as a condition to ensure that the value hasn't changed
▲ Show 20 Lines • Show All 56 Lines • Show Last 20 Lines