HomePhabricator
Diffusion Comm c0cd77ac4a13

[native] Backup compaction upload

Description

[native] Backup compaction upload

Summary:
ENG-6217 : Procedure for handling compaction/log files
ENG-6218 : Upload logic for compactions

General logic for file (compaction and log) upload and compaction specific upload. After enabling the backup handler and each time TRIGGER_BACKUP_FILE_UPLOAD is triggered we want to iterate over all files in backup directory and upload data. A compaction should consist of 3 files: backup-XXX which contains the compaction data (created in C++ database code), backup-123-userkeys which contains the UserKeys in JSON (this will be created from Rust in a later diff) and an optional backup-123-attachments file which contains newline-separated list of attachment hashes (files containing sensitive data will are encrypted). The files names are parsed with regex to get e.g. backupID. The data is uploaded and after a succesfull upload, files are deleted.

Depends on D10630

Test Plan: Create backup-123, backup-123-attachments, backup-123-userkeys files. Enable backup upload. Make sure all data was successfuly uploaded to the backup service and the files were removed.

Reviewers: marcin, bartek, kamil

Reviewed By: bartek

Subscribers: ashoat, tomek

Differential Revision: https://phab.comm.dev/D10632

Details

Provenance
Michal Gniadek <michal.gniadek@swmansion.com>Authored on Jan 12 2024, 7:00 AM
Reviewer
bartek
Differential Revision
D10632: [native] Backup compaction upload
Parents
rCOMM5db0c6fc62c7: [native] Function for triggering file data upload
Branches
Unknown
Tags
Unknown