Page MenuHomePhabricator
Differential D10635

[lib][web][native] Make it possible to remove a subset of drafts
ClosedPublic
Actions

Authored by inka on Jan 15 2024, 7:27 AM.
Tags
None
Referenced Files
Unknown Object (File)
Sat, Apr 19, 11:04 AM
Unknown Object (File)
Sat, Apr 19, 6:30 AM
Unknown Object (File)
Thu, Apr 17, 8:53 PM
Unknown Object (File)
Wed, Apr 9, 9:11 PM
Unknown Object (File)
Wed, Apr 9, 12:34 AM
Unknown Object (File)
Mar 18 2025, 2:50 PM
Unknown Object (File)
Mar 7 2025, 6:17 PM
Unknown Object (File)
Mar 7 2025, 1:13 PM
View All 59 Files
Subscribers
ashoat
tomek

Details

Reviewers
kamil
marcin
Commits
rCOMM80179ace3906: [lib][web][native] Make it possible to remove a subset of drafts
Summary

issue: ENG-5812

Test Plan

Ran the provided test

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

inka created this revision.Jan 15 2024, 7:27 AM
Herald added subscribers: tomek, ashoat. · View Herald TranscriptJan 15 2024, 7:27 AM
inka edited the test plan for this revision. (Show Details)Jan 15 2024, 7:27 AM
inka added 1 blocking reviewer(s): kamil.
Harbormaster returned this revision to the author for changes because remote builds failed.Jan 15 2024, 7:46 AM
Harbormaster failed remote builds in B25773: Diff 35635!
inka updated this revision to Diff 35637.Jan 15 2024, 9:00 AM

Fix error

inka added a parent revision: D10628: [lib] Refactor user reducer.Jan 15 2024, 9:00 AM
inka added a child revision: D10636: [lib] Refactor draft reducer.Jan 15 2024, 9:05 AM
Harbormaster completed remote builds in B25774: Diff 35637.Jan 15 2024, 9:20 AM
inka requested review of this revision.Jan 15 2024, 9:20 AM
kamil accepted this revision.Jan 17 2024, 1:38 AM
kamil added inline comments.
native/cpp/CommonCpp/NativeModules/DraftStoreOperations.h
69 ↗(On Diff #35637)

the convention is mixed across different files but let's keep it consistent at least in one file

web/database/queries/draft-queries.test.js
108–112 ↗(On Diff #35637)

I would also check if the remaining draft has key thread_b

This revision is now accepted and ready to land.Jan 17 2024, 1:38 AM
inka updated this revision to Diff 35730.Jan 17 2024, 8:49 AM

Rebase & address review

Harbormaster completed remote builds in B25841: Diff 35730.Jan 17 2024, 9:20 AM
inka updated this revision to Diff 36267.Jan 29 2024, 7:07 AM

Rebase

tomek added inline comments.Jan 29 2024, 7:14 AM
lib/types/draft-types.js
28

It might be more performant and safer to allow deleting all the entries associated with a given keyserver, e.g.
`DELETE FROM drafts WHERE id LIKE 'threadID|%'. It's probably not worth fixing it now, but it might be a good idea to introduce this improvement later.

Harbormaster completed remote builds in B26256: Diff 36267.Jan 29 2024, 7:36 AM
inka added inline comments.Jan 29 2024, 8:35 AM
lib/types/draft-types.js
28

I created a task to discuss this, because that could be done in all stores actually ENG-6603

Closed by commit rCOMM80179ace3906: [lib][web][native] Make it possible to remove a subset of drafts (authored by inka). · Explain WhyJan 29 2024, 8:41 AM
This revision was automatically updated to reflect the committed changes.
inka added a commit: rCOMM80179ace3906: [lib][web][native] Make it possible to remove a subset of drafts.

Revision Contents
Changeset List

PathSize
lib/
types/
8 lines
native/
cpp/
CommonCpp/
DatabaseManagers/
1 line
1 line
6 lines
NativeModules/
19 lines
PersistentStorageUtilities/
DataStores/
1 line
4 lines
web/
cpp/
1 line
database/
_generated/
queries/
6 lines
types/
1 line
worker/
3 lines

Diff 36267

View Options

lib/types/draft-types.js

// TODO: get rid of this // TODO: get rid of this
#![allow(dead_code)] #![allow(dead_code)]
use std::collections::HashMap; use std::collections::HashMap;
use aws_sdk_dynamodb::model::AttributeValue; use aws_sdk_dynamodb::model::AttributeValue;
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use tracing::warn;
use crate::{ use crate::{
constants::devices_table::*, constants::devices_table::*,
database::parse_string_attribute, database::parse_string_attribute,
error::{DBItemAttributeError, DBItemError, FromAttributeValue}, error::{DBItemAttributeError, DBItemError, FromAttributeValue},
grpc_services::protos::unauth::DeviceType, grpc_services::protos::unauth::DeviceType,
}; };
Show All 34 Lines
struct DeviceListKeyAttribute(DateTime<Utc>); struct DeviceListKeyAttribute(DateTime<Utc>);
impl From<DeviceIDAttribute> for AttributeValue { impl From<DeviceIDAttribute> for AttributeValue {
fn from(value: DeviceIDAttribute) -> Self { fn from(value: DeviceIDAttribute) -> Self {
AttributeValue::S(format!("{DEVICE_ITEM_KEY_PREFIX}{}", value.0)) AttributeValue::S(format!("{DEVICE_ITEM_KEY_PREFIX}{}", value.0))
} }
} }
impl From<DeviceListKeyAttribute> for AttributeValue {
fn from(value: DeviceListKeyAttribute) -> Self {
AttributeValue::S(format!(
"{DEVICE_LIST_KEY_PREFIX}{}",
value.0.to_rfc3339()
))
}
}
impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute { impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute {
type Error = DBItemError; type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> { fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?; let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;
// remove the device- prefix // remove the device- prefix
let device_id = item_id let device_id = item_id
.strip_prefix(DEVICE_ITEM_KEY_PREFIX) .strip_prefix(DEVICE_ITEM_KEY_PREFIX)
.ok_or_else(|| DBItemError { .ok_or_else(|| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(), attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: Some(AttributeValue::S(item_id.clone())), attribute_value: Some(AttributeValue::S(item_id.clone())),
attribute_error: DBItemAttributeError::InvalidValue, attribute_error: DBItemAttributeError::InvalidValue,
})? })?
.to_string(); .to_string();
Ok(Self(device_id)) Ok(Self(device_id))
} }
} }
impl TryFrom<Option<AttributeValue>> for DeviceListKeyAttribute {
type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;
// remove the device-list- prefix, then parse the timestamp
let timestamp: DateTime<Utc> = item_id
.strip_prefix(DEVICE_LIST_KEY_PREFIX)
.ok_or_else(|| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: Some(AttributeValue::S(item_id.clone())),
attribute_error: DBItemAttributeError::InvalidValue,
})
.and_then(|s| {
s.parse().map_err(|e| {
DBItemError::new(
ATTR_ITEM_ID.to_string(),
Some(AttributeValue::S(item_id.clone())),
DBItemAttributeError::InvalidTimestamp(e),
)
})
})?;
Ok(Self(timestamp))
}
}
impl TryFrom<RawAttributes> for DeviceRow { impl TryFrom<RawAttributes> for DeviceRow {
type Error = DBItemError; type Error = DBItemError;
fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> { fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> {
let user_id = let user_id =
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?; parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?; let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?;
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Lines if let Some(social_proof) = value.social_proof {
ATTR_SOCIAL_PROOF.to_string(), ATTR_SOCIAL_PROOF.to_string(),
AttributeValue::S(social_proof), AttributeValue::S(social_proof),
); );
} }
attrs attrs
} }
} }
impl TryFrom<RawAttributes> for DeviceListRow {
type Error = DBItemError;
fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> {
let user_id =
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceListKeyAttribute(timestamp) =
attrs.remove(ATTR_ITEM_ID).try_into()?;
// validate timestamps are in sync
let timestamps_match = attrs
.remove(ATTR_TIMESTAMP)
.and_then(|attr| attr.as_n().ok().cloned())
.and_then(|val| val.parse::<i64>().ok())
.filter(|val| *val == timestamp.timestamp_millis())
.is_some();
if !timestamps_match {
warn!(
"DeviceList timestamp mismatch for (userID={}, itemID={})",
&user_id,
timestamp.to_rfc3339()
);
}
// this should be a list of strings
let device_ids = attrs
.remove(ATTR_DEVICE_IDS)
.ok_or_else(|| {
DBItemError::new(
ATTR_DEVICE_IDS.to_string(),
None,
DBItemAttributeError::Missing,
)
})?
.to_vec(ATTR_DEVICE_IDS)?
.iter()
.map(|v| v.to_string("device_ids[?]").cloned())
.collect::<Result<Vec<String>, DBItemError>>()?;
Ok(Self {
user_id,
timestamp,
device_ids,
})
}
}
impl From<DeviceListRow> for RawAttributes {
fn from(device_list: DeviceListRow) -> Self {
let mut attrs = HashMap::new();
attrs.insert(
ATTR_USER_ID.to_string(),
AttributeValue::S(device_list.user_id.clone()),
);
attrs.insert(
ATTR_ITEM_ID.to_string(),
DeviceListKeyAttribute(device_list.timestamp).into(),
);
attrs.insert(
ATTR_TIMESTAMP.to_string(),
AttributeValue::N(device_list.timestamp.timestamp_millis().to_string()),
);
bartekAuthorUnsubmitted
Done
Inline Actions

Note that we use RFC 3339 string for ItemID while the ATTR_TIMESTAMP uses milliseconds number. Generally I use the former everywhere (simpler parsing without NaiveDateTime conversions) but I thought the latter is easier for DDB to sort by (it's used only as index sorting key and not read anywhere else)

bartek: Note that we use RFC 3339 string for `ItemID` while the `ATTR_TIMESTAMP` uses milliseconds…
attrs.insert(
ATTR_DEVICE_IDS.to_string(),
AttributeValue::L(
device_list
.device_ids
.into_iter()
.map(AttributeValue::S)
.collect(),
),
);
attrs
}
}
View Options

native/cpp/CommonCpp/DatabaseManagers/DatabaseQueryExecutor.h

// TODO: get rid of this // TODO: get rid of this
#![allow(dead_code)] #![allow(dead_code)]
use std::collections::HashMap; use std::collections::HashMap;
use aws_sdk_dynamodb::model::AttributeValue; use aws_sdk_dynamodb::model::AttributeValue;
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use tracing::warn;
use crate::{ use crate::{
constants::devices_table::*, constants::devices_table::*,
database::parse_string_attribute, database::parse_string_attribute,
error::{DBItemAttributeError, DBItemError, FromAttributeValue}, error::{DBItemAttributeError, DBItemError, FromAttributeValue},
grpc_services::protos::unauth::DeviceType, grpc_services::protos::unauth::DeviceType,
}; };
Show All 34 Lines
struct DeviceListKeyAttribute(DateTime<Utc>); struct DeviceListKeyAttribute(DateTime<Utc>);
impl From<DeviceIDAttribute> for AttributeValue { impl From<DeviceIDAttribute> for AttributeValue {
fn from(value: DeviceIDAttribute) -> Self { fn from(value: DeviceIDAttribute) -> Self {
AttributeValue::S(format!("{DEVICE_ITEM_KEY_PREFIX}{}", value.0)) AttributeValue::S(format!("{DEVICE_ITEM_KEY_PREFIX}{}", value.0))
} }
} }
impl From<DeviceListKeyAttribute> for AttributeValue {
fn from(value: DeviceListKeyAttribute) -> Self {
AttributeValue::S(format!(
"{DEVICE_LIST_KEY_PREFIX}{}",
value.0.to_rfc3339()
))
}
}
impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute { impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute {
type Error = DBItemError; type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> { fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?; let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;
// remove the device- prefix // remove the device- prefix
let device_id = item_id let device_id = item_id
.strip_prefix(DEVICE_ITEM_KEY_PREFIX) .strip_prefix(DEVICE_ITEM_KEY_PREFIX)
.ok_or_else(|| DBItemError { .ok_or_else(|| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(), attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: Some(AttributeValue::S(item_id.clone())), attribute_value: Some(AttributeValue::S(item_id.clone())),
attribute_error: DBItemAttributeError::InvalidValue, attribute_error: DBItemAttributeError::InvalidValue,
})? })?
.to_string(); .to_string();
Ok(Self(device_id)) Ok(Self(device_id))
} }
} }
impl TryFrom<Option<AttributeValue>> for DeviceListKeyAttribute {
type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;
// remove the device-list- prefix, then parse the timestamp
let timestamp: DateTime<Utc> = item_id
.strip_prefix(DEVICE_LIST_KEY_PREFIX)
.ok_or_else(|| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: Some(AttributeValue::S(item_id.clone())),
attribute_error: DBItemAttributeError::InvalidValue,
})
.and_then(|s| {
s.parse().map_err(|e| {
DBItemError::new(
ATTR_ITEM_ID.to_string(),
Some(AttributeValue::S(item_id.clone())),
DBItemAttributeError::InvalidTimestamp(e),
)
})
})?;
Ok(Self(timestamp))
}
}
impl TryFrom<RawAttributes> for DeviceRow { impl TryFrom<RawAttributes> for DeviceRow {
type Error = DBItemError; type Error = DBItemError;
fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> { fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> {
let user_id = let user_id =
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?; parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?; let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?;
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Lines if let Some(social_proof) = value.social_proof {
ATTR_SOCIAL_PROOF.to_string(), ATTR_SOCIAL_PROOF.to_string(),
AttributeValue::S(social_proof), AttributeValue::S(social_proof),
); );
} }
attrs attrs
} }
} }
impl TryFrom<RawAttributes> for DeviceListRow {
type Error = DBItemError;
fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> {
let user_id =
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceListKeyAttribute(timestamp) =
attrs.remove(ATTR_ITEM_ID).try_into()?;
// validate timestamps are in sync
let timestamps_match = attrs
.remove(ATTR_TIMESTAMP)
.and_then(|attr| attr.as_n().ok().cloned())
.and_then(|val| val.parse::<i64>().ok())
.filter(|val| *val == timestamp.timestamp_millis())
.is_some();
if !timestamps_match {
warn!(
"DeviceList timestamp mismatch for (userID={}, itemID={})",
&user_id,
timestamp.to_rfc3339()
);
}
// this should be a list of strings
let device_ids = attrs
.remove(ATTR_DEVICE_IDS)
.ok_or_else(|| {
DBItemError::new(
ATTR_DEVICE_IDS.to_string(),
None,
DBItemAttributeError::Missing,
)
})?
.to_vec(ATTR_DEVICE_IDS)?
.iter()
.map(|v| v.to_string("device_ids[?]").cloned())
.collect::<Result<Vec<String>, DBItemError>>()?;
Ok(Self {
user_id,
timestamp,
device_ids,
})
}
}
impl From<DeviceListRow> for RawAttributes {
fn from(device_list: DeviceListRow) -> Self {
let mut attrs = HashMap::new();
attrs.insert(
ATTR_USER_ID.to_string(),
AttributeValue::S(device_list.user_id.clone()),
);
attrs.insert(
ATTR_ITEM_ID.to_string(),
DeviceListKeyAttribute(device_list.timestamp).into(),
);
attrs.insert(
ATTR_TIMESTAMP.to_string(),
AttributeValue::N(device_list.timestamp.timestamp_millis().to_string()),
);
bartekAuthorUnsubmitted
Done
Inline Actions

Note that we use RFC 3339 string for ItemID while the ATTR_TIMESTAMP uses milliseconds number. Generally I use the former everywhere (simpler parsing without NaiveDateTime conversions) but I thought the latter is easier for DDB to sort by (it's used only as index sorting key and not read anywhere else)

bartek: Note that we use RFC 3339 string for `ItemID` while the `ATTR_TIMESTAMP` uses milliseconds…
attrs.insert(
ATTR_DEVICE_IDS.to_string(),
AttributeValue::L(
device_list
.device_ids
.into_iter()
.map(AttributeValue::S)
.collect(),
),
);
attrs
}
}
View Options

native/cpp/CommonCpp/DatabaseManagers/SQLiteQueryExecutor.h

// TODO: get rid of this // TODO: get rid of this
#![allow(dead_code)] #![allow(dead_code)]
use std::collections::HashMap; use std::collections::HashMap;
use aws_sdk_dynamodb::model::AttributeValue; use aws_sdk_dynamodb::model::AttributeValue;
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use tracing::warn;
use crate::{ use crate::{
constants::devices_table::*, constants::devices_table::*,
database::parse_string_attribute, database::parse_string_attribute,
error::{DBItemAttributeError, DBItemError, FromAttributeValue}, error::{DBItemAttributeError, DBItemError, FromAttributeValue},
grpc_services::protos::unauth::DeviceType, grpc_services::protos::unauth::DeviceType,
}; };
Show All 34 Lines
struct DeviceListKeyAttribute(DateTime<Utc>); struct DeviceListKeyAttribute(DateTime<Utc>);
impl From<DeviceIDAttribute> for AttributeValue { impl From<DeviceIDAttribute> for AttributeValue {
fn from(value: DeviceIDAttribute) -> Self { fn from(value: DeviceIDAttribute) -> Self {
AttributeValue::S(format!("{DEVICE_ITEM_KEY_PREFIX}{}", value.0)) AttributeValue::S(format!("{DEVICE_ITEM_KEY_PREFIX}{}", value.0))
} }
} }
impl From<DeviceListKeyAttribute> for AttributeValue {
fn from(value: DeviceListKeyAttribute) -> Self {
AttributeValue::S(format!(
"{DEVICE_LIST_KEY_PREFIX}{}",
value.0.to_rfc3339()
))
}
}
impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute { impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute {
type Error = DBItemError; type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> { fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?; let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;
// remove the device- prefix // remove the device- prefix
let device_id = item_id let device_id = item_id
.strip_prefix(DEVICE_ITEM_KEY_PREFIX) .strip_prefix(DEVICE_ITEM_KEY_PREFIX)
.ok_or_else(|| DBItemError { .ok_or_else(|| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(), attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: Some(AttributeValue::S(item_id.clone())), attribute_value: Some(AttributeValue::S(item_id.clone())),
attribute_error: DBItemAttributeError::InvalidValue, attribute_error: DBItemAttributeError::InvalidValue,
})? })?
.to_string(); .to_string();
Ok(Self(device_id)) Ok(Self(device_id))
} }
} }
impl TryFrom<Option<AttributeValue>> for DeviceListKeyAttribute {
type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;
// remove the device-list- prefix, then parse the timestamp
let timestamp: DateTime<Utc> = item_id
.strip_prefix(DEVICE_LIST_KEY_PREFIX)
.ok_or_else(|| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: Some(AttributeValue::S(item_id.clone())),
attribute_error: DBItemAttributeError::InvalidValue,
})
.and_then(|s| {
s.parse().map_err(|e| {
DBItemError::new(
ATTR_ITEM_ID.to_string(),
Some(AttributeValue::S(item_id.clone())),
DBItemAttributeError::InvalidTimestamp(e),
)
})
})?;
Ok(Self(timestamp))
}
}
impl TryFrom<RawAttributes> for DeviceRow { impl TryFrom<RawAttributes> for DeviceRow {
type Error = DBItemError; type Error = DBItemError;
fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> { fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> {
let user_id = let user_id =
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?; parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?; let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?;
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Lines if let Some(social_proof) = value.social_proof {
ATTR_SOCIAL_PROOF.to_string(), ATTR_SOCIAL_PROOF.to_string(),
AttributeValue::S(social_proof), AttributeValue::S(social_proof),
); );
} }
attrs attrs
} }
} }
impl TryFrom<RawAttributes> for DeviceListRow {
type Error = DBItemError;
fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> {
let user_id =
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceListKeyAttribute(timestamp) =
attrs.remove(ATTR_ITEM_ID).try_into()?;
// validate timestamps are in sync
let timestamps_match = attrs
.remove(ATTR_TIMESTAMP)
.and_then(|attr| attr.as_n().ok().cloned())
.and_then(|val| val.parse::<i64>().ok())
.filter(|val| *val == timestamp.timestamp_millis())
.is_some();
if !timestamps_match {
warn!(
"DeviceList timestamp mismatch for (userID={}, itemID={})",
&user_id,
timestamp.to_rfc3339()
);
}
// this should be a list of strings
let device_ids = attrs
.remove(ATTR_DEVICE_IDS)
.ok_or_else(|| {
DBItemError::new(
ATTR_DEVICE_IDS.to_string(),
None,
DBItemAttributeError::Missing,
)
})?
.to_vec(ATTR_DEVICE_IDS)?
.iter()
.map(|v| v.to_string("device_ids[?]").cloned())
.collect::<Result<Vec<String>, DBItemError>>()?;
Ok(Self {
user_id,
timestamp,
device_ids,
})
}
}
impl From<DeviceListRow> for RawAttributes {
fn from(device_list: DeviceListRow) -> Self {
let mut attrs = HashMap::new();
attrs.insert(
ATTR_USER_ID.to_string(),
AttributeValue::S(device_list.user_id.clone()),
);
attrs.insert(
ATTR_ITEM_ID.to_string(),
DeviceListKeyAttribute(device_list.timestamp).into(),
);
attrs.insert(
ATTR_TIMESTAMP.to_string(),
AttributeValue::N(device_list.timestamp.timestamp_millis().to_string()),
);
bartekAuthorUnsubmitted
Done
Inline Actions

Note that we use RFC 3339 string for ItemID while the ATTR_TIMESTAMP uses milliseconds number. Generally I use the former everywhere (simpler parsing without NaiveDateTime conversions) but I thought the latter is easier for DDB to sort by (it's used only as index sorting key and not read anywhere else)

bartek: Note that we use RFC 3339 string for `ItemID` while the `ATTR_TIMESTAMP` uses milliseconds…
attrs.insert(
ATTR_DEVICE_IDS.to_string(),
AttributeValue::L(
device_list
.device_ids
.into_iter()
.map(AttributeValue::S)
.collect(),
),
);
attrs
}
}
View Options

native/cpp/CommonCpp/DatabaseManagers/SQLiteQueryExecutor.cpp

// TODO: get rid of this // TODO: get rid of this
#![allow(dead_code)] #![allow(dead_code)]
use std::collections::HashMap; use std::collections::HashMap;
use aws_sdk_dynamodb::model::AttributeValue; use aws_sdk_dynamodb::model::AttributeValue;
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use tracing::warn;
use crate::{ use crate::{
constants::devices_table::*, constants::devices_table::*,
database::parse_string_attribute, database::parse_string_attribute,
error::{DBItemAttributeError, DBItemError, FromAttributeValue}, error::{DBItemAttributeError, DBItemError, FromAttributeValue},
grpc_services::protos::unauth::DeviceType, grpc_services::protos::unauth::DeviceType,
}; };
Show All 34 Lines
struct DeviceListKeyAttribute(DateTime<Utc>); struct DeviceListKeyAttribute(DateTime<Utc>);
impl From<DeviceIDAttribute> for AttributeValue { impl From<DeviceIDAttribute> for AttributeValue {
fn from(value: DeviceIDAttribute) -> Self { fn from(value: DeviceIDAttribute) -> Self {
AttributeValue::S(format!("{DEVICE_ITEM_KEY_PREFIX}{}", value.0)) AttributeValue::S(format!("{DEVICE_ITEM_KEY_PREFIX}{}", value.0))
} }
} }
impl From<DeviceListKeyAttribute> for AttributeValue {
fn from(value: DeviceListKeyAttribute) -> Self {
AttributeValue::S(format!(
"{DEVICE_LIST_KEY_PREFIX}{}",
value.0.to_rfc3339()
))
}
}
impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute { impl TryFrom<Option<AttributeValue>> for DeviceIDAttribute {
type Error = DBItemError; type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> { fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?; let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;
// remove the device- prefix // remove the device- prefix
let device_id = item_id let device_id = item_id
.strip_prefix(DEVICE_ITEM_KEY_PREFIX) .strip_prefix(DEVICE_ITEM_KEY_PREFIX)
.ok_or_else(|| DBItemError { .ok_or_else(|| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(), attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: Some(AttributeValue::S(item_id.clone())), attribute_value: Some(AttributeValue::S(item_id.clone())),
attribute_error: DBItemAttributeError::InvalidValue, attribute_error: DBItemAttributeError::InvalidValue,
})? })?
.to_string(); .to_string();
Ok(Self(device_id)) Ok(Self(device_id))
} }
} }
impl TryFrom<Option<AttributeValue>> for DeviceListKeyAttribute {
type Error = DBItemError;
fn try_from(value: Option<AttributeValue>) -> Result<Self, Self::Error> {
let item_id = parse_string_attribute(ATTR_ITEM_ID, value)?;
// remove the device-list- prefix, then parse the timestamp
let timestamp: DateTime<Utc> = item_id
.strip_prefix(DEVICE_LIST_KEY_PREFIX)
.ok_or_else(|| DBItemError {
attribute_name: ATTR_ITEM_ID.to_string(),
attribute_value: Some(AttributeValue::S(item_id.clone())),
attribute_error: DBItemAttributeError::InvalidValue,
})
.and_then(|s| {
s.parse().map_err(|e| {
DBItemError::new(
ATTR_ITEM_ID.to_string(),
Some(AttributeValue::S(item_id.clone())),
DBItemAttributeError::InvalidTimestamp(e),
)
})
})?;
Ok(Self(timestamp))
}
}
impl TryFrom<RawAttributes> for DeviceRow { impl TryFrom<RawAttributes> for DeviceRow {
type Error = DBItemError; type Error = DBItemError;
fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> { fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> {
let user_id = let user_id =
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?; parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?; let DeviceIDAttribute(device_id) = attrs.remove(ATTR_ITEM_ID).try_into()?;
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Lines if let Some(social_proof) = value.social_proof {
ATTR_SOCIAL_PROOF.to_string(), ATTR_SOCIAL_PROOF.to_string(),
AttributeValue::S(social_proof), AttributeValue::S(social_proof),
); );
} }
attrs attrs
} }
} }
impl TryFrom<RawAttributes> for DeviceListRow {
type Error = DBItemError;
fn try_from(mut attrs: RawAttributes) -> Result<Self, Self::Error> {
let user_id =
parse_string_attribute(ATTR_USER_ID, attrs.remove(ATTR_USER_ID))?;
let DeviceListKeyAttribute(timestamp) =
attrs.remove(ATTR_ITEM_ID).try_into()?;
// validate timestamps are in sync
let timestamps_match = attrs
.remove(ATTR_TIMESTAMP)
.and_then(|attr| attr.as_n().ok().cloned())
.and_then(|val| val.parse::<i64>().ok())
.filter(|val| *val == timestamp.timestamp_millis())
.is_some();
if !timestamps_match {
warn!(
"DeviceList timestamp mismatch for (userID={}, itemID={})",
&user_id,
timestamp.to_rfc3339()
);
}
// this should be a list of strings
let device_ids = attrs
.remove(ATTR_DEVICE_IDS)
.ok_or_else(|| {
DBItemError::new(
ATTR_DEVICE_IDS.to_string(),
None,
DBItemAttributeError::Missing,
)
})?
.to_vec(ATTR_DEVICE_IDS)?
.iter()
.map(|v| v.to_string("device_ids[?]").cloned())
.collect::<Result<Vec<String>, DBItemError>>()?;
Ok(Self {
user_id,
timestamp,
device_ids,
})
}
}
impl From<DeviceListRow> for RawAttributes {
fn from(device_list: DeviceListRow) -> Self {
let mut attrs = HashMap::new();
attrs.insert(
ATTR_USER_ID.to_string(),
AttributeValue::S(device_list.user_id.clone()),
);
attrs.insert(
ATTR_ITEM_ID.to_string(),
DeviceListKeyAttribute(device_list.timestamp).into(),
);
attrs.insert(
ATTR_TIMESTAMP.to_string(),
AttributeValue::N(device_list.timestamp.timestamp_millis().to_string()),
);
bartekAuthorUnsubmitted
Done
Inline Actions

Note that we use RFC 3339 string for ItemID while the ATTR_TIMESTAMP uses milliseconds number. Generally I use the former everywhere (simpler parsing without NaiveDateTime conversions) but I thought the latter is easier for DDB to sort by (it's used only as index sorting key and not read anywhere else)

bartek: Note that we use RFC 3339 string for `ItemID` while the `ATTR_TIMESTAMP` uses milliseconds…
attrs.insert(
ATTR_DEVICE_IDS.to_string(),
AttributeValue::L(
device_list
.device_ids
.into_iter()
.map(AttributeValue::S)
.collect(),
),
);
attrs
}
}
View Options

native/cpp/CommonCpp/NativeModules/DraftStoreOperations.h

Loading...
View Options

native/cpp/CommonCpp/NativeModules/PersistentStorageUtilities/DataStores/DraftStore.h

Loading...
View Options

native/cpp/CommonCpp/NativeModules/PersistentStorageUtilities/DataStores/DraftStore.cpp

Loading...
View Options

web/cpp/SQLiteQueryExecutorBindings.cpp

Loading...
View Options

web/database/_generated/comm_query_executor.wasm

Loading...
View Options

web/database/queries/draft-queries.test.js

Loading...
View Options

web/database/types/sqlite-query-executor.js

Loading...
View Options

web/database/worker/process-operations.js

Loading...