[services] Backup - Add chunk counter to proto
AbandonedPublic
Actions

Authored by • karol on Jun 7 2022, 6:51 AM.

Details

Reviewers

tomek
varun
• max
ashoat

Summary

Depends on D4228

We need to be able to separate logs from each other when we pull the backup from the cloud.
For now, we just read the following log chunks without knowing how many logs we really get since multiple chunks can fit into one log.

With the chunk counter, we can easily recognize when we get a new log - whenever the counter is 0.

Test Plan

this itself doesn't do anything - none.

Diff Detail

Repository

rCOMM Comm

Lint

No Lint Coverage

Unit

No Test Coverage

Event Timeline

• karol created this revision.Jun 7 2022, 6:51 AM

Herald added subscribers: • abosh, atul, • adrian and 2 others. · View Herald TranscriptJun 7 2022, 6:51 AM

Harbormaster completed remote builds in B9658: Diff 13385.Jun 7 2022, 7:24 AM

• karol requested review of this revision.Jun 7 2022, 7:24 AM

• karol edited the summary of this revision. (Show Details)Jun 7 2022, 8:22 AM

• karol edited the test plan for this revision. (Show Details)

• karol added reviewers: tomek, varun, • max.

• karol added a parent revision: D4228: [services] Tests - Add pull backup.

• karol added a child revision: D4230: [services] Backup - Add chunk counter to the pull backup reactor.

Adding myself since it's a .proto diff! Please remember to include me on all of these.

So the idea here is that chunkCounter is 0 when there is a new chunk? What's the value when there is an old chunk?
Does a "log chunk" contain multiple chunks? Or does a log get split into multiple chunks?

It looks like there are some answers to my questions in D4230:

A log or a compaction can contain multiple chunks. A single chunk only pertains to a single log or a single compaction
The chunkCounter gets incremented for each subsequent chunk that pertains to the same log or compaction