[services][backup] PullBackup 5/5 - blob download stream
Summary:
This diff leverages both ResponseBuffer and BlobStoredItem to create a generic stream that downloads the item from blob service, buffers the data and sends in chunks not exceeding the gRPC message size. A few notes:
- First message includes "additional extra info" - fields like attachment holders etc. Rest receive only ID and data chunk
- The data is buffered before sending, this is the solution 2 from https://phab.comm.dev/D4439#126984
- More data is not downloaded when the buffer is already filled enough
- Stream ends when there is no more data
The order of instructions is important here:
- Push new data if not full
- Finish if empty
- Pop if not empty
Depends on D6243
Test Plan:
At this point, the whole service can be tested with integration tests
cd services yarn reset-local-cloud nix run .#comm-blob RUST_LOG=backup=trace cargo run -- --port 50052 --sandbox --blob-service-url "http://localhost:50051" yarn run-integration-tests backup
Reviewers: tomek, varun, jon, michal
Reviewed By: tomek
Subscribers: ashoat, atul
Differential Revision: https://phab.comm.dev/D6246