HomePhabricator
Diffusion Comm 45efacf6e151

[blob] Add function to perform cleanup operation

Description

[blob] Add function to perform cleanup operation

Summary:
This diff adds a function that performs the core cleanup logic. Its logic should be fail-proof: failure of any steps should prevent blob deletion.

The algorithm (also documented in code comments):

  1. Get both blobs and holders rows that are marked as "unchecked".
  2. Organize them into structures that have information about blob-holder assignments and their existence.
  3. Filter out entries that contain both blob and holders - they're valid.
  4. Get more information about entries that contain only blob or only holders:
    • For holders, get the blob hash and check if it exists in DB
    • For blobs, check if at least 1 holder exists in DB
  5. Update the struct with query results and repeat step 3.
  6. Delete rows that still contain only blob or only holders. Mark others as "checked".

Depends on D9351

Test Plan:
Ran this function on a dataset that contained the following combinations:

  • Blobs having blob hashes and holders, none marked as unchecked
  • Blobs having blob hashes and holders, blob item marked as unchecked, holders not marked
  • Blobs having blob hashes and holders, some holders marked as unchecked, blob hash not marked
  • Blobs having blob hashes and holders, some holders marked as unchecked, blob hash marked too
  • Blobs having blob hashes and holders, all holders marked as unchecked, blob hash not marked
  • Blobs having blob hashes and holders, blob hash and all holders marked as unchecked
  • Blobs having only blob hash row (marked as unchecked), no holders.
  • Blobs having only holder rows (marked as unchecked), no blob hash.

Reviewers: tomek, michal, varun, jon

Reviewed By: tomek, michal

Subscribers: ashoat, wyilio

Differential Revision: https://phab.comm.dev/D9352

Details

Provenance
bartekAuthored on Oct 4 2023, 2:47 AM
Reviewer
tomek
Differential Revision
D9352: [blob] Add function to perform cleanup operation
Parents
rCOMM9936af0fbb13: [blob] Implement cleanup helper functions
Branches
Unknown
Tags
Unknown