HomePhabricator
Diffusion Comm df0b40f1a30c

[backup] Improve attachment handling

Description

[backup] Improve attachment handling

Summary:
ENG

Added better attachment handling and cleanup:

  • attachments actually need to be complete BlobInfo and not just holders so changes their type to Vec<BlobInfo> and their dynamodb type to list of maps
    • also changes the column/constants name to better match this
    • added a new helper conversion method for vector of any TryFromAttr type
  • added attachment cleanup on upload error, and on standard backup cleanup

Test Plan:
Send upload endpoint with two attachments: "non-existant-hash" and "keys_hash" (the same hash as for the user_keys, so it will be uploaded during backup upload before handling attachments)
Checked that:

  • there two holders for "keys_hash" (one from user keys and one from an attachment)
  • there is a holder for "non-existant-hash" but there was a warning from backup

Uploaded similar backup, with different hashes, to trigger backup cleanup:

  • check that all hashes and holders from the previous backup are gone
  • check that there are new hashes and holders analogous to the previous backup

Restart backup service with dummy localstack endpoint so the upload fails:

  • observer no changes in the blob dynamodb

Reviewers: bartek, patryk, kamil, jon

Reviewed By: bartek

Subscribers: ashoat, tomek

Differential Revision: https://phab.comm.dev/D9016

Event Timeline

Michal Gniadek <michal.gniadek@swmansion.com> committed rCOMMdf0b40f1a30c: [backup] Improve attachment handling (authored by Michal Gniadek <michal.gniadek@swmansion.com>).Sep 5 2023, 7:32 AM