[backup] Log download endpoint
ClosedPublic
Actions

Authored by michal on Dec 22 2023, 8:08 AM.

Details

Reviewers

bartek
kamil

Commits

rCOMMdf7964047d75: [backup] Log download endpoint

Summary

ENG-5330 : Add endpoint for downloading logs

Log download logic. Each request specifies (optional) index from which the fetch should start which allows us to paginate the results. The final message sent by the backup service contains the id from which should the next fetch be started or None of we have already fetched all logs for a given backup.

Depends on D10456

Test Plan

Created a 100 logs for 3 backups, half of which were bigger than the dynamodb maximum storage size (so they were stored in blob service)
Run this code (simplified):

tx.send(DownloadLogsRequest { from_id: None });

'outer: loop {
  loop {
    match rx.next().await {
      Some(Ok(LogWSResponse::LogDownload {
        log_id,
        content,
        attachments,
      })) => {
        // Make sure that content and attachments match the expected values
        // for this specific logs
        // ...
      }
      Some(Ok(LogWSResponse::LogDownloadFinished { last_log_id })) => {
        if let Some(last_log_id) = last_log_id {
          tx.send(DownloadLogsRequest {
            from_id: Some(last_log_id),
          });
        } else {
          break 'outer;
        }
      }
    };
  }
}