Paths

Table of Contentst

1/2
.gitignore
-
native/
-
cpp/CommonCpp/grpc/grpc_client/
-
CommonCpp/
-
grpc/
-
grpc_client/
1/2
Cargo.lock
1/2
Cargo.toml
1/2
build.rs
-
cxx.h
-
lib.rs.h
-
src/
-
identity_client.rs
-
lib.rs
-
opaque.rs
-
ios/Comm.xcodeproj/
-
Comm.xcodeproj/
-
project.pbxproj
-
native_rust_library/
-
.gitignore
-
Cargo.lock
-
Cargo.toml
-
build.rs
-
src/
-
identity_client.rs
-
lib.rs
-
opaque.rs
-
scripts/
-
build-rust-native-library.sh
-
clean-up-native-rust-library.sh
-
clean-up-rust-native-library.sh

[native] more refactoring of the native rust library
ClosedPublic
Actions

Authored by varun on Sep 16 2022, 2:56 PM.

Details

Reviewers

ashoat
• jon

Commits

rCOMM3b94e3aedfd1: [native] more refactoring of the native rust library

Summary

remove the header files after build so they don't appear in the devs' git status. also rename grpc_client -> native_rust_library and move it out of the cpp folder. we're renaming the cargo project because it will be more than a gRPC client library going forward.

Depends on D5104

Test Plan

cleaned the Xcode build folder, removed all build artifacts from git repo with cargo clean, deleted the header files, and then ran Xcode build

Diff Detail

Repository

rCOMM Comm

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

varun created this revision.Sep 16 2022, 2:56 PM

Herald added subscribers: • abosh, ashoat. · View Herald TranscriptSep 16 2022, 2:56 PM

Harbormaster completed remote builds in B12233: Diff 16782.Sep 16 2022, 3:06 PM

varun requested review of this revision.Sep 16 2022, 3:06 PM

varun edited the summary of this revision. (Show Details)Sep 16 2022, 3:13 PM

varun added subscribers: inka, marcin.Sep 16 2022, 3:34 PM

varun added inline comments.Sep 16 2022, 3:52 PM

.gitignore
11 ↗	(On Diff #16782)	created a separate .gitignore file for the cargo project
native/cpp/CommonCpp/grpc/grpc_client/cxx.h
1 ↗	(On Diff #16782)	we don't need to check this file in anymore, since we're creating it in the Xcode build and deleting it in the last build phase. same with the other header file
native/native_rust_library/.gitignore
1 ↗	(On Diff #16782)	it's easier to just have a separate .gitignore for the cargo project in case we move the folder to a new location in the future
scripts/clean-up-native-rust-library.sh
1–3 ↗	(On Diff #16782)	addresses @jon 's feedback on previous diff in stack

ashoat accepted this revision.Sep 16 2022, 4:32 PM

ashoat added inline comments.

native/cpp/CommonCpp/grpc/grpc_client/cxx.h
1 ↗	(On Diff #16782)	we're creating it in the Xcode build and deleting it in the last build phase Where exactly does this change occur? Can't tell if there's something in the `.pbxproj` or what
scripts/clean-up-native-rust-library.sh
1 ↗	(On Diff #16782)	Newline after shebang

varun added inline comments.Sep 16 2022, 4:39 PM

native/cpp/CommonCpp/grpc/grpc_client/cxx.h
1 ↗	(On Diff #16782)	the files are generated in `scripts/build-rust-native-library.sh` (previous diff in stack) and they're deleted in `scripts/clean-up-native-rust-library.sh`. these scripts are being called in Xcode Build Phases (you can see it in `project.pbxproj` lines 895 and 914)
scripts/clean-up-native-rust-library.sh
1 ↗	(On Diff #16782)	ack

varun removed reviewers: • jon, atul, tomek.Sep 16 2022, 4:45 PM

This revision is now accepted and ready to land.Sep 16 2022, 4:45 PM

Herald added a reviewer: • jon. · View Herald TranscriptSep 16 2022, 4:45 PM

Closed by commit rCOMM3b94e3aedfd1: [native] more refactoring of the native rust library (authored by varun). · Explain WhySep 16 2022, 5:23 PM

This revision was automatically updated to reflect the committed changes.

varun added a commit: rCOMM3b94e3aedfd1: [native] more refactoring of the native rust library.

tomek mentioned this in D5350: Remove deleting cxx files after build.Oct 14 2022, 2:45 AM

Revision Contents
Changeset List

Path

Size

.gitignore

1 line

native/

cpp/

CommonCpp/

grpc/

grpc_client/

src/

ios/

Comm.xcodeproj/

project.pbxproj

45 lines

native_rust_library/

.gitignore

1 line

	native_rust_library/
	cpp/	CommonCpp/	grpc/	grpc_client/

Cargo.lock

Cargo.toml

2 lines

build.rs

2 lines

src/

identity_client.rs

lib.rs

opaque.rs

scripts/

build-rust-native-library.sh

6 lines

clean-up-native-rust-library.sh

4 lines

clean-up-rust-native-library.sh

Diff 16785

View Options

.gitignore

use derive_more::{Display, Error, From};		use derive_more::{Display, Error, From};
use futures_core::Stream;		use futures_core::Stream;
use futures_util::StreamExt;		use futures_util::{StreamExt, TryStreamExt};
use reqwest::Url;		use reqwest::{
		multipart::{Form, Part},
		Body, Url,
		};
use tracing::{debug, trace, warn};		use tracing::{debug, trace, warn};

// publicly re-export some reqwest types		// publicly re-export some reqwest types
pub use reqwest::Error as ReqwestError;		pub use reqwest::Error as ReqwestError;
pub use reqwest::StatusCode;		pub use reqwest::StatusCode;

#[derive(From, Error, Debug, Display)]		#[derive(From, Error, Debug, Display)]
pub enum BlobServiceError {		pub enum BlobServiceError {
▲ Show 20 Lines • Show All 91 Lines • ▼ Show 20 Lines	) -> BlobResult<impl Stream<Item = BlobResult<Vec<u8>>>> {
}		}

let error = handle_http_error(response.status());		let error = handle_http_error(response.status());
if let Ok(message) = response.text().await {		if let Ok(message) = response.text().await {
trace!("Error response message: {}", message);		trace!("Error response message: {}", message);
}		}
Err(error)		Err(error)
}		}

		/// Uploads a blob. Returns `BlobServiceError::AlreadyExists` if blob with given hash
		/// already exists.
		///
		/// # Example
		/// ```rust
		/// use std::io::{Error, ErrorKind};
		///
		/// let client =
		/// BlobServiceClient::new("http://localhost:50053".parse()?);
		///
		/// let stream = async_stream::stream! {
		/// yield Ok(vec![1, 2, 3]);
		/// yield Ok(vec![4, 5, 6]);
		/// yield Err(Error::new(ErrorKind::Other, "Oops"));
		/// };
		/// client.upload_blob(&blob_hash, stream).await?;
		/// ```
		pub async fn upload_blob<H, S>(
		&self,
		blob_hash: H,
		data_stream: S,
		) -> BlobResult<()>
		where
		H: Into<String>,
		S: futures_core::stream::TryStream + Send + Sync + 'static,
		S::Error: Into<Box<dyn std::error::Error + Send + Sync>>,
		Vec<u8>: From<S::Ok>,
		bartekAuthorUnsubmitted Done Inline Actions Requirements of the reqwest body stream: A stream of `Result<impl Into<Vec<u8>>, impl Into<Box<dyn Error>>>` bartek: Requirements of the reqwest body stream: A stream of `Result<impl Into<Vec<u8>>, impl…
		{
		debug!("Revoke holder request");
		michalUnsubmitted Not Done Inline Actions Wrong message? michal: Wrong message?
		let url = self.get_blob_url(None)?;

		let stream = data_stream.map_ok(Vec::from);
		let streaming_body = Body::wrap_stream(stream);
		let form = Form::new()
		.text("blob_hash", blob_hash.into())
		.part("blob_data", Part::stream(streaming_body));

		let response = self.http_client.put(url).multipart(form).send().await?;
		debug!("Response status: {}", response.status());

		if response.status().is_success() {
		trace!("Blob upload successful");
		return Ok(());
		}

		let error = handle_http_error(response.status());
		if let Ok(message) = response.text().await {
		trace!("Error response message: {}", message);
		}
		Err(error)
		}
}		}

// private helper methods		// private helper methods
impl BlobServiceClient {		impl BlobServiceClient {
fn get_blob_url(		fn get_blob_url(
&self,		&self,
blob_hash: Option<&str>,		blob_hash: Option<&str>,
) -> Result<Url, BlobServiceError> {		) -> Result<Url, BlobServiceError> {
Show All 24 Lines

View Options

native/cpp/CommonCpp/grpc/grpc_client/Cargo.lock

use derive_more::{Display, Error, From};		use derive_more::{Display, Error, From};
use futures_core::Stream;		use futures_core::Stream;
use futures_util::StreamExt;		use futures_util::{StreamExt, TryStreamExt};
use reqwest::Url;		use reqwest::{
		multipart::{Form, Part},
		Body, Url,
		};
use tracing::{debug, trace, warn};		use tracing::{debug, trace, warn};

// publicly re-export some reqwest types		// publicly re-export some reqwest types
pub use reqwest::Error as ReqwestError;		pub use reqwest::Error as ReqwestError;
pub use reqwest::StatusCode;		pub use reqwest::StatusCode;

#[derive(From, Error, Debug, Display)]		#[derive(From, Error, Debug, Display)]
pub enum BlobServiceError {		pub enum BlobServiceError {
▲ Show 20 Lines • Show All 91 Lines • ▼ Show 20 Lines	) -> BlobResult<impl Stream<Item = BlobResult<Vec<u8>>>> {
}		}

let error = handle_http_error(response.status());		let error = handle_http_error(response.status());
if let Ok(message) = response.text().await {		if let Ok(message) = response.text().await {
trace!("Error response message: {}", message);		trace!("Error response message: {}", message);
}		}
Err(error)		Err(error)
}		}

		/// Uploads a blob. Returns `BlobServiceError::AlreadyExists` if blob with given hash
		/// already exists.
		///
		/// # Example
		/// ```rust
		/// use std::io::{Error, ErrorKind};
		///
		/// let client =
		/// BlobServiceClient::new("http://localhost:50053".parse()?);
		///
		/// let stream = async_stream::stream! {
		/// yield Ok(vec![1, 2, 3]);
		/// yield Ok(vec![4, 5, 6]);
		/// yield Err(Error::new(ErrorKind::Other, "Oops"));
		/// };
		/// client.upload_blob(&blob_hash, stream).await?;
		/// ```
		pub async fn upload_blob<H, S>(
		&self,
		blob_hash: H,
		data_stream: S,
		) -> BlobResult<()>
		where
		H: Into<String>,
		S: futures_core::stream::TryStream + Send + Sync + 'static,
		S::Error: Into<Box<dyn std::error::Error + Send + Sync>>,
		Vec<u8>: From<S::Ok>,
		bartekAuthorUnsubmitted Done Inline Actions Requirements of the reqwest body stream: A stream of `Result<impl Into<Vec<u8>>, impl Into<Box<dyn Error>>>` bartek: Requirements of the reqwest body stream: A stream of `Result<impl Into<Vec<u8>>, impl…
		{
		debug!("Revoke holder request");
		michalUnsubmitted Not Done Inline Actions Wrong message? michal: Wrong message?
		let url = self.get_blob_url(None)?;

		let stream = data_stream.map_ok(Vec::from);
		let streaming_body = Body::wrap_stream(stream);
		let form = Form::new()
		.text("blob_hash", blob_hash.into())
		.part("blob_data", Part::stream(streaming_body));

		let response = self.http_client.put(url).multipart(form).send().await?;
		debug!("Response status: {}", response.status());

		if response.status().is_success() {
		trace!("Blob upload successful");
		return Ok(());
		}

		let error = handle_http_error(response.status());
		if let Ok(message) = response.text().await {
		trace!("Error response message: {}", message);
		}
		Err(error)
		}
}		}

// private helper methods		// private helper methods
impl BlobServiceClient {		impl BlobServiceClient {
fn get_blob_url(		fn get_blob_url(
&self,		&self,
blob_hash: Option<&str>,		blob_hash: Option<&str>,
) -> Result<Url, BlobServiceError> {		) -> Result<Url, BlobServiceError> {
Show All 24 Lines

View Options

native/cpp/CommonCpp/grpc/grpc_client/Cargo.toml

use derive_more::{Display, Error, From};		use derive_more::{Display, Error, From};
use futures_core::Stream;		use futures_core::Stream;
use futures_util::StreamExt;		use futures_util::{StreamExt, TryStreamExt};
use reqwest::Url;		use reqwest::{
		multipart::{Form, Part},
		Body, Url,
		};
use tracing::{debug, trace, warn};		use tracing::{debug, trace, warn};

// publicly re-export some reqwest types		// publicly re-export some reqwest types
pub use reqwest::Error as ReqwestError;		pub use reqwest::Error as ReqwestError;
pub use reqwest::StatusCode;		pub use reqwest::StatusCode;

#[derive(From, Error, Debug, Display)]		#[derive(From, Error, Debug, Display)]
pub enum BlobServiceError {		pub enum BlobServiceError {
▲ Show 20 Lines • Show All 91 Lines • ▼ Show 20 Lines	) -> BlobResult<impl Stream<Item = BlobResult<Vec<u8>>>> {
}		}

let error = handle_http_error(response.status());		let error = handle_http_error(response.status());
if let Ok(message) = response.text().await {		if let Ok(message) = response.text().await {
trace!("Error response message: {}", message);		trace!("Error response message: {}", message);
}		}
Err(error)		Err(error)
}		}

		/// Uploads a blob. Returns `BlobServiceError::AlreadyExists` if blob with given hash
		/// already exists.
		///
		/// # Example
		/// ```rust
		/// use std::io::{Error, ErrorKind};
		///
		/// let client =
		/// BlobServiceClient::new("http://localhost:50053".parse()?);
		///
		/// let stream = async_stream::stream! {
		/// yield Ok(vec![1, 2, 3]);
		/// yield Ok(vec![4, 5, 6]);
		/// yield Err(Error::new(ErrorKind::Other, "Oops"));
		/// };
		/// client.upload_blob(&blob_hash, stream).await?;
		/// ```
		pub async fn upload_blob<H, S>(
		&self,
		blob_hash: H,
		data_stream: S,
		) -> BlobResult<()>
		where
		H: Into<String>,
		S: futures_core::stream::TryStream + Send + Sync + 'static,
		S::Error: Into<Box<dyn std::error::Error + Send + Sync>>,
		Vec<u8>: From<S::Ok>,
		bartekAuthorUnsubmitted Done Inline Actions Requirements of the reqwest body stream: A stream of `Result<impl Into<Vec<u8>>, impl Into<Box<dyn Error>>>` bartek: Requirements of the reqwest body stream: A stream of `Result<impl Into<Vec<u8>>, impl…
		{
		debug!("Revoke holder request");
		michalUnsubmitted Not Done Inline Actions Wrong message? michal: Wrong message?
		let url = self.get_blob_url(None)?;

		let stream = data_stream.map_ok(Vec::from);
		let streaming_body = Body::wrap_stream(stream);
		let form = Form::new()
		.text("blob_hash", blob_hash.into())
		.part("blob_data", Part::stream(streaming_body));

		let response = self.http_client.put(url).multipart(form).send().await?;
		debug!("Response status: {}", response.status());

		if response.status().is_success() {
		trace!("Blob upload successful");
		return Ok(());
		}

		let error = handle_http_error(response.status());
		if let Ok(message) = response.text().await {
		trace!("Error response message: {}", message);
		}
		Err(error)
		}
}		}

// private helper methods		// private helper methods
impl BlobServiceClient {		impl BlobServiceClient {
fn get_blob_url(		fn get_blob_url(
&self,		&self,
blob_hash: Option<&str>,		blob_hash: Option<&str>,
) -> Result<Url, BlobServiceError> {		) -> Result<Url, BlobServiceError> {
Show All 24 Lines

View Options

native/cpp/CommonCpp/grpc/grpc_client/build.rs

use derive_more::{Display, Error, From};		use derive_more::{Display, Error, From};
use futures_core::Stream;		use futures_core::Stream;
use futures_util::StreamExt;		use futures_util::{StreamExt, TryStreamExt};
use reqwest::Url;		use reqwest::{
		multipart::{Form, Part},
		Body, Url,
		};
use tracing::{debug, trace, warn};		use tracing::{debug, trace, warn};

// publicly re-export some reqwest types		// publicly re-export some reqwest types
pub use reqwest::Error as ReqwestError;		pub use reqwest::Error as ReqwestError;
pub use reqwest::StatusCode;		pub use reqwest::StatusCode;

#[derive(From, Error, Debug, Display)]		#[derive(From, Error, Debug, Display)]
pub enum BlobServiceError {		pub enum BlobServiceError {
▲ Show 20 Lines • Show All 91 Lines • ▼ Show 20 Lines	) -> BlobResult<impl Stream<Item = BlobResult<Vec<u8>>>> {
}		}

let error = handle_http_error(response.status());		let error = handle_http_error(response.status());
if let Ok(message) = response.text().await {		if let Ok(message) = response.text().await {
trace!("Error response message: {}", message);		trace!("Error response message: {}", message);
}		}
Err(error)		Err(error)
}		}

		/// Uploads a blob. Returns `BlobServiceError::AlreadyExists` if blob with given hash
		/// already exists.
		///
		/// # Example
		/// ```rust
		/// use std::io::{Error, ErrorKind};
		///
		/// let client =
		/// BlobServiceClient::new("http://localhost:50053".parse()?);
		///
		/// let stream = async_stream::stream! {
		/// yield Ok(vec![1, 2, 3]);
		/// yield Ok(vec![4, 5, 6]);
		/// yield Err(Error::new(ErrorKind::Other, "Oops"));
		/// };
		/// client.upload_blob(&blob_hash, stream).await?;
		/// ```
		pub async fn upload_blob<H, S>(
		&self,
		blob_hash: H,
		data_stream: S,
		) -> BlobResult<()>
		where
		H: Into<String>,
		S: futures_core::stream::TryStream + Send + Sync + 'static,
		S::Error: Into<Box<dyn std::error::Error + Send + Sync>>,
		Vec<u8>: From<S::Ok>,
		bartekAuthorUnsubmitted Done Inline Actions Requirements of the reqwest body stream: A stream of `Result<impl Into<Vec<u8>>, impl Into<Box<dyn Error>>>` bartek: Requirements of the reqwest body stream: A stream of `Result<impl Into<Vec<u8>>, impl…
		{
		debug!("Revoke holder request");
		michalUnsubmitted Not Done Inline Actions Wrong message? michal: Wrong message?
		let url = self.get_blob_url(None)?;

		let stream = data_stream.map_ok(Vec::from);
		let streaming_body = Body::wrap_stream(stream);
		let form = Form::new()
		.text("blob_hash", blob_hash.into())
		.part("blob_data", Part::stream(streaming_body));

		let response = self.http_client.put(url).multipart(form).send().await?;
		debug!("Response status: {}", response.status());

		if response.status().is_success() {
		trace!("Blob upload successful");
		return Ok(());
		}

		let error = handle_http_error(response.status());
		if let Ok(message) = response.text().await {
		trace!("Error response message: {}", message);
		}
		Err(error)
		}
}		}

// private helper methods		// private helper methods
impl BlobServiceClient {		impl BlobServiceClient {
fn get_blob_url(		fn get_blob_url(
&self,		&self,
blob_hash: Option<&str>,		blob_hash: Option<&str>,
) -> Result<Url, BlobServiceError> {		) -> Result<Url, BlobServiceError> {
Show All 24 Lines