Page MenuHomePhabricator
Differential D5008

[services] Backup - Blob Put Client - Add address constant
ClosedPublic
Actions

Authored by • karol on Sep 1 2022, 4:07 AM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Apr 8, 11:20 PM
Unknown Object (File)
Sun, Apr 6, 12:50 AM
Unknown Object (File)
Wed, Apr 2, 11:01 AM
Unknown Object (File)
Wed, Apr 2, 6:17 AM
Unknown Object (File)
Mon, Mar 31, 5:22 PM
Unknown Object (File)
Sun, Mar 30, 7:32 AM
Unknown Object (File)
Sun, Mar 16, 12:23 AM
Unknown Object (File)
Wed, Mar 12, 5:04 AM
View All Files
Subscribers
• abosh
ashoat
atul
tomek

Details

Reviewers
• jon
tomek
• max
varun
Commits
rCOMMa9fbedbf1692: [services] Backup - Blob Put Client - Add address constant
Summary

Depends on D5007

Adding the blob address that's going to be used by the blob client to connect to the blob server

Test Plan

backup builds

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

• karol created this revision.Sep 1 2022, 4:07 AM
Herald added a reviewer: • jon. · View Herald TranscriptSep 1 2022, 4:07 AM
Herald added subscribers: • abosh, atul, tomek, ashoat. · View Herald Transcript
Harbormaster completed remote builds in B11765: Diff 16168.Sep 1 2022, 4:19 AM
• karol requested review of this revision.Sep 1 2022, 4:19 AM
• karol edited the summary of this revision. (Show Details)Sep 1 2022, 4:21 AM
• karol edited the test plan for this revision. (Show Details)
• karol added reviewers: tomek, • max, varun.
• karol added a parent revision: D5007: [services] Backup - Blob Put Client - Install async stream.
• karol added a child revision: D5009: [services] Backup - Blob Put Client - Add initialize.
• jon requested changes to this revision.Sep 1 2022, 8:44 AM
• jon added inline comments.
services/backup/blob_client/src/constants.rs
2 ↗(On Diff #16168)

We should have some way to configure this

The following should be a valid way to use blob client, or something similar:

BLOB_ADDRESS=localhost:3829 ./blob_client
This revision now requires changes to proceed.Sep 1 2022, 8:44 AM
• karol requested review of this revision.Sep 2 2022, 5:53 AM
• karol added inline comments.
services/backup/blob_client/src/constants.rs
2 ↗(On Diff #16168)

I created a task for this, I think it can be addressed later https://linear.app/comm/issue/ENG-1731/enable-configuring-the-blob-address-from-the-outside

tomek accepted this revision.Sep 5 2022, 2:38 AM
• jon accepted this revision.Sep 7 2022, 10:38 AM
This revision is now accepted and ready to land.Sep 7 2022, 10:38 AM
• karol updated this revision to Diff 16440.Sep 8 2022, 3:58 AM

rebase

Harbormaster completed remote builds in B11985: Diff 16440.Sep 8 2022, 4:09 AM
Closed by commit rCOMMa9fbedbf1692: [services] Backup - Blob Put Client - Add address constant (authored by • karol). · Explain WhySep 8 2022, 4:10 AM
This revision was automatically updated to reflect the committed changes.
• karol added a commit: rCOMMa9fbedbf1692: [services] Backup - Blob Put Client - Add address constant.

Revision Contents
Changeset List

PathSize
services/
backup/
blob_client/
src/
1 line

Diff 16440

View Options

services/backup/blob_client/src/constants.rs

mod proto { mod proto {
tonic::include_proto!("blob"); tonic::include_proto!("blob");
} }
use proto::blob_service_client::BlobServiceClient; use proto::blob_service_client::BlobServiceClient;
use proto::put_request; use proto::put_request;
use proto::put_request::Data::*; use proto::put_request::Data::*;
use proto::PutRequest; use proto::PutRequest;
use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY}; use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY};
use crate::tools::{ use crate::tools::{
c_char_pointer_to_string, c_char_pointer_to_string_new, check_error, c_char_pointer_to_string, c_char_pointer_to_string_new, report_error, string_to_c_char_pointer_new,
report_error, string_to_c_char_pointer, string_to_c_char_pointer_new,
}; };
use anyhow::bail; use anyhow::bail;
use lazy_static::lazy_static; use lazy_static::lazy_static;
use libc; use libc;
use libc::c_char; use libc::c_char;
use std::collections::HashMap; use std::collections::HashMap;
use std::ffi::CStr; use std::ffi::CStr;
use std::sync::Mutex; use std::sync::Mutex;
▲ Show 20 Lines • Show All 172 Lines • ▼ Show 20 Lines if let Ok(mut grpc_client) =
} }
return Err(format!("could not access client for holder {}", holder)); return Err(format!("could not access client for holder {}", holder));
} }
Err("could not successfully connect to the blob server".to_string()) Err("could not successfully connect to the blob server".to_string())
} }
pub fn put_client_blocking_read_cxx( pub fn put_client_blocking_read_cxx(
holder_char: *const c_char, holder_char: *const c_char,
) -> Result<String, String> { ) -> anyhow::Result<String, anyhow::Error> {
tomekUnsubmitted
Not Done
Inline Actions
holder_char: *const c_char,
- ) -> anyhow::Result<String, anyhow::Error> {
+ ) -> anyhow::Result<String> {
let holder = c_char_pointer_to_string_new(holder_char)?;
tomek:
let holder = c_char_pointer_to_string(holder_char)?; let holder = c_char_pointer_to_string_new(holder_char)?;
check_error(&ERROR_MESSAGES)?; Ok(RUNTIME.block_on(async {
let response: Option<String> = RUNTIME.block_on(async {
if let Ok(mut clients) = CLIENTS.lock() { if let Ok(mut clients) = CLIENTS.lock() {
let maybe_client = clients.get_mut(&holder); let maybe_client = clients.get_mut(&holder);
if let Some(client) = maybe_client { if let Some(client) = maybe_client {
if let Some(data) = client.rx.recv().await { if let Some(data) = client.rx.recv().await {
return Some(data); return Ok(data);
} else { } else {
report_error( bail!("couldn't receive data via client's receiver");
&ERROR_MESSAGES,
"couldn't receive data via client's receiver",
Some("put"),
);
} }
} else { } else {
report_error( bail!(format!("no client detected for {} in blocking read", holder));
&ERROR_MESSAGES,
"no client detected in blocking read",
Some("put"),
);
} }
} else { } else {
report_error(&ERROR_MESSAGES, "couldn't access client", Some("put")); bail!("couldn't access clients");
} }
None })?)
});
check_error(&ERROR_MESSAGES)?;
response.ok_or("response not received properly".to_string())
} }
/** /**
* field index: * field index:
* 1 - holder (utf8 string) * 1 - holder (utf8 string)
* 2 - blob hash (utf8 string) * 2 - blob hash (utf8 string)
* 3 - data chunk (bytes) * 3 - data chunk (bytes)
*/ */
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines