Page MenuHomePhabricator
Differential D4992

[Shared] Copy protos to shared/protos, adjust CMake
ClosedPublic
Actions

Authored by • jon on Aug 31 2022, 10:29 AM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Mar 25, 2:42 AM
Unknown Object (File)
Fri, Mar 21, 5:45 PM
Unknown Object (File)
Tue, Mar 18, 2:15 PM
Unknown Object (File)
Sat, Mar 15, 10:18 PM
Unknown Object (File)
Sat, Mar 15, 12:48 PM
Unknown Object (File)
Wed, Mar 12, 10:07 AM
Unknown Object (File)
Wed, Mar 12, 5:51 AM
Unknown Object (File)
Tue, Mar 11, 10:05 PM
View All Files
Subscribers
ashoat
tomek

Details

Reviewers
atul
• abosh
• max
• karol
tomek
Commits
rCOMMaa8c3fa0bca1: [Shared] Copy protos to shared/protos, adjust CMake
Summary

Initial move of protobuf files

The files are copied to the desired end state so that I can update the references in later revisions.

The CMakeLists.txt was pruned of the grpc client project.

https://linear.app/comm/issue/ENG-1231

Test Plan
nix develop
cd shared/protos
rm -rf build && mkdir build && cd build && cmake .. && make -j
# builds 4 grpc projects

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

• jon created this revision.Aug 31 2022, 10:29 AM
Herald added subscribers: tomek, ashoat. · View Herald TranscriptAug 31 2022, 10:29 AM
• jon added a child revision: D4993: [Shared/docs] Update `yarn codegen-grpc`.Aug 31 2022, 10:39 AM
Harbormaster completed remote builds in B11735: Diff 16126.Aug 31 2022, 10:39 AM
• jon requested review of this revision.Aug 31 2022, 10:39 AM
• jon updated this revision to Diff 16134.Aug 31 2022, 11:02 AM

Update target_include_directory for downstream consumers

Harbormaster completed remote builds in B11741: Diff 16134.Aug 31 2022, 11:12 AM
• max accepted this revision.Aug 31 2022, 4:59 PM
This revision is now accepted and ready to land.Aug 31 2022, 4:59 PM
• karol requested changes to this revision.Sep 1 2022, 1:52 AM
• karol added a reviewer: tomek.

You didn't move a thing, you doubled everything. Please, fix this revision.

shared/protos/_generated/.clang-format
4 ↗(On Diff #16134)

new line (I know it wasn't there but we can add it)

This revision now requires changes to proceed.Sep 1 2022, 1:52 AM
• jon added a comment.EditedSep 1 2022, 10:54 AM

You didn't move a thing, you doubled everything. Please, fix this revision.

@karol please look at the stack, iOS, android, and 3 services (+ related docker files) all reference these files. I'm not going to move and FIX everything in one go. The final piece (not yet present, because I got blocked) in the stack will be to remove the old files.

• jon requested review of this revision.Sep 1 2022, 10:54 AM
• jon edited the summary of this revision. (Show Details)Sep 1 2022, 12:39 PM
• karol added a comment.EditedSep 1 2022, 11:49 PM
  1. Ok, I understand, but then the diff title isn't valid. The diff title says explicitly "move" and yet you're not moving things, so maybe it should say "add"? If this is about to be addressed in the future, we should either have a diff for it or a task on linear and add a reference here.
  2. What about my inline comment? Has not been addressed.
• karol requested changes to this revision.Sep 2 2022, 12:07 AM
This revision now requires changes to proceed.Sep 2 2022, 12:07 AM
• jon retitled this revision from [Shared] Move protos to shared/protos to [Shared] Copy protos to shared/protos, adjust CMake.Sep 2 2022, 10:43 AM
• jon updated this revision to Diff 16254.Sep 2 2022, 11:26 AM

Add new line, export cmake targets

Harbormaster completed remote builds in B11849: Diff 16254.Sep 2 2022, 11:36 AM
• jon updated this revision to Diff 16265.Sep 2 2022, 3:23 PM

Rebase on top of native/cpp changes

Harbormaster completed remote builds in B11852: Diff 16265.Sep 2 2022, 3:33 PM
• jon added a child revision: D5042: [Services] Update tunnelbroker reference to protos.Sep 2 2022, 5:14 PM
• jon added a child revision: D5046: [Native] Remove old protos and related files.Sep 2 2022, 5:41 PM
tomek accepted this revision.Sep 5 2022, 1:13 AM
tomek added a comment.Sep 5 2022, 4:50 AM

Should we update get_clang_paths.js to include shared?

• karol accepted this revision.Sep 5 2022, 5:42 AM

What about this @jon

If this is about to be addressed in the future, we should either have a diff for it or a task on linear and add a reference here.

This revision is now accepted and ready to land.Sep 5 2022, 5:42 AM
• jon marked an inline comment as done.Sep 6 2022, 7:27 AM
• jon edited the summary of this revision. (Show Details)Sep 6 2022, 7:31 AM
• jon added a comment.Sep 6 2022, 7:44 AM

or a task on linear and add a reference here.

Yea, that was my bad.

• jon added a child revision: D5044: [Services] Update docker references to protobuf.Sep 6 2022, 9:12 AM
• jon added a child revision: D5043: [Services] Update backup to reference new proto location.
• jon added a child revision: D5045: [Native] Prune protobuf logic from CMakeLists.txt.
• jon updated this revision to Diff 16416.Sep 7 2022, 9:41 AM

Rebase on master

Harbormaster completed remote builds in B11962: Diff 16416.Sep 7 2022, 9:51 AM
Closed by commit rCOMMaa8c3fa0bca1: [Shared] Copy protos to shared/protos, adjust CMake (authored by • jon). · Explain WhySep 8 2022, 10:29 AM
This revision was automatically updated to reflect the committed changes.
• jon added a commit: rCOMMaa8c3fa0bca1: [Shared] Copy protos to shared/protos, adjust CMake.
• jon mentioned this in D5105: [Chore] Add new protos build dir to gitignore.Sep 9 2022, 4:46 PM
• jon mentioned this in rCOMM0721bc662099: [Chore] Add new protos build dir to gitignore.Sep 12 2022, 10:50 AM

Revision Contents
Changeset List

Diff 16254

View Options

shared/protos/CMakeLists.txt

mod proto { mod proto {
tonic::include_proto!("blob"); tonic::include_proto!("blob");
} }
use proto::blob_service_client::BlobServiceClient; use proto::blob_service_client::BlobServiceClient;
use proto::put_request; use proto::put_request;
use proto::put_request::Data::*; use proto::put_request::Data::*;
use proto::PutRequest; use proto::PutRequest;
use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY}; use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY};
use crate::tools::{ use crate::tools::{
c_char_pointer_to_string, c_char_pointer_to_string_new, check_error, c_char_pointer_to_string, c_char_pointer_to_string_new, report_error, string_to_c_char_pointer_new,
report_error, string_to_c_char_pointer, string_to_c_char_pointer_new,
}; };
use anyhow::bail; use anyhow::bail;
use crate::RUNTIME; use crate::RUNTIME;
use lazy_static::lazy_static; use lazy_static::lazy_static;
use libc; use libc;
use libc::c_char; use libc::c_char;
use std::collections::HashMap; use std::collections::HashMap;
use std::ffi::CStr; use std::ffi::CStr;
▲ Show 20 Lines • Show All 168 Lines • ▼ Show 20 Lines if let Ok(mut grpc_client) =
} }
return Err(format!("could not access client for holder {}", holder)); return Err(format!("could not access client for holder {}", holder));
} }
Err("could not successfully connect to the blob server".to_string()) Err("could not successfully connect to the blob server".to_string())
} }
pub fn put_client_blocking_read_cxx( pub fn put_client_blocking_read_cxx(
holder_char: *const c_char, holder_char: *const c_char,
) -> Result<String, String> { ) -> anyhow::Result<String, anyhow::Error> {
let holder = c_char_pointer_to_string(holder_char)?; let holder = c_char_pointer_to_string_new(holder_char)?;
check_error(&ERROR_MESSAGES)?; Ok(RUNTIME.block_on(async {
let response: Option<String> = RUNTIME.block_on(async {
if let Ok(mut clients) = CLIENTS.lock() { if let Ok(mut clients) = CLIENTS.lock() {
let maybe_client = clients.get_mut(&holder); let maybe_client = clients.get_mut(&holder);
if let Some(client) = maybe_client { if let Some(client) = maybe_client {
if let Some(data) = client.rx.recv().await { if let Some(data) = client.rx.recv().await {
return Some(data); return Ok(data);
} else { } else {
report_error( bail!("couldn't receive data via client's receiver");
&ERROR_MESSAGES,
"couldn't receive data via client's receiver",
Some("put"),
);
} }
} else { } else {
report_error( bail!(format!("no client detected for {} in blocking read", holder));
&ERROR_MESSAGES,
"no client detected in blocking read",
Some("put"),
);
} }
} else { } else {
report_error(&ERROR_MESSAGES, "couldn't access client", Some("put")); bail!("couldn't access clients");
} }
None })?)
});
check_error(&ERROR_MESSAGES)?;
response.ok_or("response not received properly".to_string())
} }
/** /**
* field index: * field index:
* 1 - holder (utf8 string) * 1 - holder (utf8 string)
* 2 - blob hash (utf8 string) * 2 - blob hash (utf8 string)
* 3 - data chunk (bytes) * 3 - data chunk (bytes)
*/ */
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines
View Options

shared/protos/_generated/.clang-format

mod proto { mod proto {
tonic::include_proto!("blob"); tonic::include_proto!("blob");
} }
use proto::blob_service_client::BlobServiceClient; use proto::blob_service_client::BlobServiceClient;
use proto::put_request; use proto::put_request;
use proto::put_request::Data::*; use proto::put_request::Data::*;
use proto::PutRequest; use proto::PutRequest;
use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY}; use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY};
use crate::tools::{ use crate::tools::{
c_char_pointer_to_string, c_char_pointer_to_string_new, check_error, c_char_pointer_to_string, c_char_pointer_to_string_new, report_error, string_to_c_char_pointer_new,
report_error, string_to_c_char_pointer, string_to_c_char_pointer_new,
}; };
use anyhow::bail; use anyhow::bail;
use crate::RUNTIME; use crate::RUNTIME;
use lazy_static::lazy_static; use lazy_static::lazy_static;
use libc; use libc;
use libc::c_char; use libc::c_char;
use std::collections::HashMap; use std::collections::HashMap;
use std::ffi::CStr; use std::ffi::CStr;
▲ Show 20 Lines • Show All 168 Lines • ▼ Show 20 Lines if let Ok(mut grpc_client) =
} }
return Err(format!("could not access client for holder {}", holder)); return Err(format!("could not access client for holder {}", holder));
} }
Err("could not successfully connect to the blob server".to_string()) Err("could not successfully connect to the blob server".to_string())
} }
pub fn put_client_blocking_read_cxx( pub fn put_client_blocking_read_cxx(
holder_char: *const c_char, holder_char: *const c_char,
) -> Result<String, String> { ) -> anyhow::Result<String, anyhow::Error> {
let holder = c_char_pointer_to_string(holder_char)?; let holder = c_char_pointer_to_string_new(holder_char)?;
check_error(&ERROR_MESSAGES)?; Ok(RUNTIME.block_on(async {
let response: Option<String> = RUNTIME.block_on(async {
if let Ok(mut clients) = CLIENTS.lock() { if let Ok(mut clients) = CLIENTS.lock() {
let maybe_client = clients.get_mut(&holder); let maybe_client = clients.get_mut(&holder);
if let Some(client) = maybe_client { if let Some(client) = maybe_client {
if let Some(data) = client.rx.recv().await { if let Some(data) = client.rx.recv().await {
return Some(data); return Ok(data);
} else { } else {
report_error( bail!("couldn't receive data via client's receiver");
&ERROR_MESSAGES,
"couldn't receive data via client's receiver",
Some("put"),
);
} }
} else { } else {
report_error( bail!(format!("no client detected for {} in blocking read", holder));
&ERROR_MESSAGES,
"no client detected in blocking read",
Some("put"),
);
} }
} else { } else {
report_error(&ERROR_MESSAGES, "couldn't access client", Some("put")); bail!("couldn't access clients");
} }
None })?)
});
check_error(&ERROR_MESSAGES)?;
response.ok_or("response not received properly".to_string())
} }
/** /**
* field index: * field index:
* 1 - holder (utf8 string) * 1 - holder (utf8 string)
* 2 - blob hash (utf8 string) * 2 - blob hash (utf8 string)
* 3 - data chunk (bytes) * 3 - data chunk (bytes)
*/ */
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines
View Options

shared/protos/_generated/backup.grpc.pb.h

mod proto { mod proto {
tonic::include_proto!("blob"); tonic::include_proto!("blob");
} }
use proto::blob_service_client::BlobServiceClient; use proto::blob_service_client::BlobServiceClient;
use proto::put_request; use proto::put_request;
use proto::put_request::Data::*; use proto::put_request::Data::*;
use proto::PutRequest; use proto::PutRequest;
use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY}; use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY};
use crate::tools::{ use crate::tools::{
c_char_pointer_to_string, c_char_pointer_to_string_new, check_error, c_char_pointer_to_string, c_char_pointer_to_string_new, report_error, string_to_c_char_pointer_new,
report_error, string_to_c_char_pointer, string_to_c_char_pointer_new,
}; };
use anyhow::bail; use anyhow::bail;
use crate::RUNTIME; use crate::RUNTIME;
use lazy_static::lazy_static; use lazy_static::lazy_static;
use libc; use libc;
use libc::c_char; use libc::c_char;
use std::collections::HashMap; use std::collections::HashMap;
use std::ffi::CStr; use std::ffi::CStr;
▲ Show 20 Lines • Show All 168 Lines • ▼ Show 20 Lines if let Ok(mut grpc_client) =
} }
return Err(format!("could not access client for holder {}", holder)); return Err(format!("could not access client for holder {}", holder));
} }
Err("could not successfully connect to the blob server".to_string()) Err("could not successfully connect to the blob server".to_string())
} }
pub fn put_client_blocking_read_cxx( pub fn put_client_blocking_read_cxx(
holder_char: *const c_char, holder_char: *const c_char,
) -> Result<String, String> { ) -> anyhow::Result<String, anyhow::Error> {
let holder = c_char_pointer_to_string(holder_char)?; let holder = c_char_pointer_to_string_new(holder_char)?;
check_error(&ERROR_MESSAGES)?; Ok(RUNTIME.block_on(async {
let response: Option<String> = RUNTIME.block_on(async {
if let Ok(mut clients) = CLIENTS.lock() { if let Ok(mut clients) = CLIENTS.lock() {
let maybe_client = clients.get_mut(&holder); let maybe_client = clients.get_mut(&holder);
if let Some(client) = maybe_client { if let Some(client) = maybe_client {
if let Some(data) = client.rx.recv().await { if let Some(data) = client.rx.recv().await {
return Some(data); return Ok(data);
} else { } else {
report_error( bail!("couldn't receive data via client's receiver");
&ERROR_MESSAGES,
"couldn't receive data via client's receiver",
Some("put"),
);
} }
} else { } else {
report_error( bail!(format!("no client detected for {} in blocking read", holder));
&ERROR_MESSAGES,
"no client detected in blocking read",
Some("put"),
);
} }
} else { } else {
report_error(&ERROR_MESSAGES, "couldn't access client", Some("put")); bail!("couldn't access clients");
} }
None })?)
});
check_error(&ERROR_MESSAGES)?;
response.ok_or("response not received properly".to_string())
} }
/** /**
* field index: * field index:
* 1 - holder (utf8 string) * 1 - holder (utf8 string)
* 2 - blob hash (utf8 string) * 2 - blob hash (utf8 string)
* 3 - data chunk (bytes) * 3 - data chunk (bytes)
*/ */
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines
View Options

shared/protos/_generated/backup.grpc.pb.cc

mod proto { mod proto {
tonic::include_proto!("blob"); tonic::include_proto!("blob");
} }
use proto::blob_service_client::BlobServiceClient; use proto::blob_service_client::BlobServiceClient;
use proto::put_request; use proto::put_request;
use proto::put_request::Data::*; use proto::put_request::Data::*;
use proto::PutRequest; use proto::PutRequest;
use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY}; use crate::constants::{BLOB_ADDRESS, MPSC_CHANNEL_BUFFER_CAPACITY};
use crate::tools::{ use crate::tools::{
c_char_pointer_to_string, c_char_pointer_to_string_new, check_error, c_char_pointer_to_string, c_char_pointer_to_string_new, report_error, string_to_c_char_pointer_new,
report_error, string_to_c_char_pointer, string_to_c_char_pointer_new,
}; };
use anyhow::bail; use anyhow::bail;
use crate::RUNTIME; use crate::RUNTIME;
use lazy_static::lazy_static; use lazy_static::lazy_static;
use libc; use libc;
use libc::c_char; use libc::c_char;
use std::collections::HashMap; use std::collections::HashMap;
use std::ffi::CStr; use std::ffi::CStr;
▲ Show 20 Lines • Show All 168 Lines • ▼ Show 20 Lines if let Ok(mut grpc_client) =
} }
return Err(format!("could not access client for holder {}", holder)); return Err(format!("could not access client for holder {}", holder));
} }
Err("could not successfully connect to the blob server".to_string()) Err("could not successfully connect to the blob server".to_string())
} }
pub fn put_client_blocking_read_cxx( pub fn put_client_blocking_read_cxx(
holder_char: *const c_char, holder_char: *const c_char,
) -> Result<String, String> { ) -> anyhow::Result<String, anyhow::Error> {
let holder = c_char_pointer_to_string(holder_char)?; let holder = c_char_pointer_to_string_new(holder_char)?;
check_error(&ERROR_MESSAGES)?; Ok(RUNTIME.block_on(async {
let response: Option<String> = RUNTIME.block_on(async {
if let Ok(mut clients) = CLIENTS.lock() { if let Ok(mut clients) = CLIENTS.lock() {
let maybe_client = clients.get_mut(&holder); let maybe_client = clients.get_mut(&holder);
if let Some(client) = maybe_client { if let Some(client) = maybe_client {
if let Some(data) = client.rx.recv().await { if let Some(data) = client.rx.recv().await {
return Some(data); return Ok(data);
} else { } else {
report_error( bail!("couldn't receive data via client's receiver");
&ERROR_MESSAGES,
"couldn't receive data via client's receiver",
Some("put"),
);
} }
} else { } else {
report_error( bail!(format!("no client detected for {} in blocking read", holder));
&ERROR_MESSAGES,
"no client detected in blocking read",
Some("put"),
);
} }
} else { } else {
report_error(&ERROR_MESSAGES, "couldn't access client", Some("put")); bail!("couldn't access clients");
} }
None })?)
});
check_error(&ERROR_MESSAGES)?;
response.ok_or("response not received properly".to_string())
} }
/** /**
* field index: * field index:
* 1 - holder (utf8 string) * 1 - holder (utf8 string)
* 2 - blob hash (utf8 string) * 2 - blob hash (utf8 string)
* 3 - data chunk (bytes) * 3 - data chunk (bytes)
*/ */
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines
View Options

shared/protos/_generated/backup.pb.h

Loading...
View Options

shared/protos/_generated/backup.pb.cc

Loading...
View Options

shared/protos/_generated/blob.grpc.pb.h

Loading...
View Options

shared/protos/_generated/blob.grpc.pb.cc

Loading...
View Options

shared/protos/_generated/blob.pb.h

Loading...
View Options

shared/protos/_generated/blob.pb.cc

Loading...
View Options

shared/protos/_generated/identity.grpc.pb.h

Loading...
View Options

shared/protos/_generated/identity.grpc.pb.cc

Loading...
View Options

shared/protos/_generated/identity.pb.h

Loading...
View Options

shared/protos/_generated/identity.pb.cc

Loading...
View Options

shared/protos/_generated/tunnelbroker.grpc.pb.h

Loading...
View Options

shared/protos/_generated/tunnelbroker.grpc.pb.cc

Loading...
View Options

shared/protos/_generated/tunnelbroker.pb.h

Loading...
View Options

shared/protos/_generated/tunnelbroker.pb.cc

Loading...
View Options

shared/protos/backup.proto

Loading...
View Options

shared/protos/blob.proto

Loading...
View Options

shared/protos/identity.proto

Loading...
View Options

shared/protos/tunnelbroker.proto

Loading...