We can do it, sorry I didn't do it earlier.

I think we can handle this.

Let's break this down.

For now, we have something like this:

* backup
*  userID[PK]            string
*  created[SK]           timestamp
*  backupID              string
*  recoveryData          bytes
*  compactionHolder      string
*  attachmentHolders     list<string>

so the primary key is (userID, created).

For efficient queries:

• we definitely need the userID as a partition key - we're always going to query the database by the user id because we want to operate on the specific user's backups.
• we need created as well since we want to be able to fetch the newest backups

If we need also to query the database by the backupID effectively, I do not see why we don't create an index for it. I thought about this for a second and it seems like for the simple auth we only need to get that backupID. The way indexes work in dynamo DB is, they copy certain data to "a new place" and we can manually specify what data are going to be copied. So this index could be really small and consist of just the primary key(which is always projected) and the backupID.

If you are sure that we cannot perform backups based on the backup ID field, please bring this up in the architecture review.

I think we can handle this in the DB settings here without affecting the architecture of the whole service.

Regarding the original issue raised by Tomek here: I think every time we want to look up a backup in the database, we should identify it by the userID and created. backupID is more like a secret(https://phabricator.ashoat.com/D2950#83151). I realize this is misleading as usually a name like xxxID indicates an identifier by which we can query the database. I think it might be good then to rename this field.

Conclusion:

• we want to create an index for the backupID that would project backupID and the primary key(userID,created)
• we may rename backupID so it's less misleading, maybe something like backupSecret

What do you think @ashoat @palys-swm ?

We probably also don't want to replace the sort key with the backupID, because the queries that we need are going to need all the data of a backup when querying by the created value and we only need(I think) the minimum of the data when querying by the backupID. So making the backupID a sort key and indexing created would cause more data duplication than if we go the way I proposed above, which is: created stays as a sort key and we add an index for the backupID. Just clarifying.

Yes

Whenever the data in the table is modified, the index is automatically changed to reflect the changes in the table.

Yes

Whenever the data in the table is modified, the index is automatically changed to reflect the changes in the table.