HomePhabricator
Diffusion Comm 6d18590b8b05

Capture backup logs at the end of each data store transaction.

Description

Capture backup logs at the end of each data store transaction.

Summary:
This differential captures backup log at the end of each data store transaction to ensure all changes made to data stores are persisted as backup logs.
How does it ensure that backup logs are created for each change:

  1. The transaction starts
  2. Operations on db are executed.
  3. We read current logID from the db.
  4. We use this logID to create backup logs file.
  5. We write to the database logID incremented by 1.
  6. Transaction is committed.

Now if the app crashes before transaction is committed but after log file is created we are left with too small logID in the db since incrementing logID was rolled back.. So next run of captureBackupLogs will use the same logID thus overwriting previous file (which must bo considered invalid since it contains changes that weren't actually persisted in the db).

The advantage of this approach is that we have consistency between logs and database. Unfortunately it removes the possibility to schedule writing log bytes to a file on separate thread. We can prioritize safety for now and if we see that app is too slow we can consider going back to my initial approach that assumed backup log creation outside of transaction.

Test Plan: This has already been tested in D10835 since the test plan of this diff inlcuded introducing this change.

Reviewers: kamil, michal

Reviewed By: kamil

Subscribers: ashoat, tomek

Differential Revision: https://phab.comm.dev/D10911

Details

Provenance
marcinAuthored on Feb 6 2024, 6:55 AM
Reviewer
kamil
Differential Revision
D10911: Capture backup logs at the end of each data store transaction.
Parents
rCOMMd0fb20607662: Implement restore from backup logs
Branches
Unknown
Tags
Unknown