Page MenuHomePhabricator
Differential D8658

[native] install CommRustModule
ClosedPublic
Actions

Authored by kamil on Jul 28 2023, 9:55 AM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Mar 26, 7:09 AM
Unknown Object (File)
Sat, Mar 22, 3:40 PM
Unknown Object (File)
Tue, Mar 18, 9:24 AM
Unknown Object (File)
Sun, Mar 2, 2:42 AM
Unknown Object (File)
Sat, Mar 1, 9:53 AM
Unknown Object (File)
Feb 22 2025, 1:53 PM
Unknown Object (File)
Feb 22 2025, 1:53 PM
Unknown Object (File)
Feb 22 2025, 1:53 PM
View All Files
Subscribers
ashoat

Details

Reviewers
bartek
michal
tomek
varun
Commits
rCOMM9ef783dfd079: [native] install CommRustModule
Summary

ENG-4232

Creating a separate module for Rust API. This is a part of CommCoreModule refactor.

Code to install new module for both iOS and Android.

Depends on D8657

Test Plan

Build Android nad iOS
On iOS added test generateNonce to make sure Rust methods are callable. Inspired by test plan from D7215.

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Revision Contents
Changeset List

PathSize
native/
android/
app/
src/
cpp/
7 lines
ios/
Comm/
7 lines
2 lines

Diff 29290

View Options

native/android/app/src/cpp/jsiInstaller.cpp

#include "CreateNewBackupReactor.h" #include "CreateNewBackupReactor.h"
#include "DatabaseManager.h" #include "DatabaseManager.h"
#include "GlobalTools.h" #include "GlobalTools.h"
#include "Tools.h" #include "Tools.h"
#include "blob_client/src/lib.rs.h"
namespace comm { namespace comm {
namespace network { namespace network {
namespace reactor { namespace reactor {
std::string CreateNewBackupReactor::generateBackupID() { std::string CreateNewBackupReactor::generateBackupID() {
if (this->deviceID.empty()) { if (this->deviceID.empty()) {
throw std::runtime_error( throw std::runtime_error(
"trying to generate a backup ID with an empty device ID"); "trying to generate a backup ID with an empty device ID");
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines case State::DATA_HASH: {
if (database::DatabaseManager::getInstance().findBackupItem( if (database::DatabaseManager::getInstance().findBackupItem(
this->userID, this->backupID) != nullptr) { this->userID, this->backupID) != nullptr) {
throw std::runtime_error( throw std::runtime_error(
"Backup with id [" + this->backupID + "] for user [" + "Backup with id [" + this->backupID + "] for user [" +
this->userID + "] already exists, creation aborted"); this->userID + "] already exists, creation aborted");
} }
response->set_backupid(this->backupID); response->set_backupid(this->backupID);
this->holder = tools::generateHolder(this->dataHash, this->backupID); this->holder = tools::generateHolder(this->dataHash, this->backupID);
// todo:blob perform put:initialize bool dataExists = false;
tomekUnsubmitted
Not Done
Inline Actions

This isn't used anywhere

tomek: This isn't used anywhere
karolAuthorUnsubmitted
Done
Inline Actions

right

karol: right
try {
put_client_initialize_cxx();
put_client_write_cxx(
tools::getBlobPutField(blob::PutRequest::DataCase::kHolder),
this->holder.c_str());
put_client_blocking_read_cxx(); // todo this should be avoided
// (blocking); we should be able to
// ignore responses; we probably want to
// delegate performing ops to separate
// threads in the base reactors
tomekUnsubmitted
Not Done
Inline Actions

We should create some tasks that cover things mentioned in the comment

tomek: We should create some tasks that cover things mentioned in the comment
karolAuthorUnsubmitted
Done
Inline Actions

https://linear.app/comm/issue/ENG-1724/solve-the-problem-with-blocked-reactors-in-grpc

karol: https://linear.app/comm/issue/ENG-1724/solve-the-problem-with-blocked-reactors-in-grpc
put_client_write_cxx(
tools::getBlobPutField(blob::PutRequest::DataCase::kBlobHash),
this->dataHash.c_str());
rust::String responseStr =
put_client_blocking_read_cxx(); // todo this should be avoided
tomekUnsubmitted
Not Done
Inline Actions

This is confusing. We call put_client_blocking_read_cxx twice, and we ignore the result once. Could you explain (preferably in code comment) what's going on here?

tomek: This is confusing. We call `put_client_blocking_read_cxx` twice, and we ignore the result once.
karolAuthorUnsubmitted
Done
Inline Actions

We read the response to unblock the reading queue. grpc in the bidi stream always yields a response even if it's empty. And it is in this case, that's why we ignore it, but we read so the queue's not blocked (this queue is a rust's channel with MPSC_CHANNEL_BUFFER_CAPACITY limit).

karol: We read the response to unblock the reading queue. grpc in the bidi stream always yields a…
// (blocking); we should be able to
// ignore responses; we probably
// want to delegate performing ops
// to separate threads in the base
// reactors
// data exists?
if ((bool)tools::charPtrToInt(responseStr.c_str())) {
return std::make_unique<ServerBidiReactorStatus>(
grpc::Status::OK, true);
}
} catch (std::exception &e) {
throw std::runtime_error(
e.what()); // todo in base reactors we can just handle std exception
// instead of keep rethrowing here
}
return nullptr; return nullptr;
} }
case State::DATA_CHUNKS: { case State::DATA_CHUNKS: {
// todo:blob perform put:add chunk if (request.mutable_newcompactionchunk()->empty()) {
// (std::move(*request.mutable_newcompactionchunk()) return std::make_unique<ServerBidiReactorStatus>(grpc::Status::OK);
}
try {
put_client_write_cxx(
tools::getBlobPutField(blob::PutRequest::DataCase::kDataChunk),
std::string(std::move(*request.mutable_newcompactionchunk()))
.c_str());
put_client_blocking_read_cxx(); // todo this should be avoided
tomekUnsubmitted
Not Done
Inline Actions

Why do we need to read after write?

tomek: Why do we need to read after write?
karolAuthorUnsubmitted
Done
Inline Actions

Explained in the other comment

karol: Explained in the other comment
// (blocking); we should be able to
// ignore responses; we probably want to
// delegate performing ops to separate
// threads in the base reactors
} catch (std::exception &e) {
throw std::runtime_error(
e.what()); // todo in base reactors we can just handle std exception
// instead of keep rethrowing here
}
return nullptr; return nullptr;
} }
} }
throw std::runtime_error("new backup - invalid state"); throw std::runtime_error("new backup - invalid state");
} }
void CreateNewBackupReactor::terminateCallback() { void CreateNewBackupReactor::terminateCallback() {
const std::lock_guard<std::mutex> lock(this->reactorStateMutex); const std::lock_guard<std::mutex> lock(this->reactorStateMutex);
// todo:blob perform put:add chunk ("") try {
// todo:blob perform put:wait for completion put_client_terminate_cxx();
} catch (std::exception &e) {
throw std::runtime_error(
e.what()); // todo in base reactors we can just handle std exception
// instead of keep rethrowing here
}
// TODO add recovery data // TODO add recovery data
// TODO handle attachments holders // TODO handle attachments holders
database::BackupItem backupItem( database::BackupItem backupItem(
this->userID, this->userID,
this->backupID, this->backupID,
tools::getCurrentTimestamp(), tools::getCurrentTimestamp(),
tools::generateRandomString(), tools::generateRandomString(),
this->holder, this->holder,
{}); {});
database::DatabaseManager::getInstance().putBackupItem(backupItem); database::DatabaseManager::getInstance().putBackupItem(backupItem);
} }
} // namespace reactor } // namespace reactor
} // namespace network } // namespace network
} // namespace comm } // namespace comm
View Options

native/ios/Comm/AppDelegate.mm

#include "CreateNewBackupReactor.h" #include "CreateNewBackupReactor.h"
#include "DatabaseManager.h" #include "DatabaseManager.h"
#include "GlobalTools.h" #include "GlobalTools.h"
#include "Tools.h" #include "Tools.h"
#include "blob_client/src/lib.rs.h"
namespace comm { namespace comm {
namespace network { namespace network {
namespace reactor { namespace reactor {
std::string CreateNewBackupReactor::generateBackupID() { std::string CreateNewBackupReactor::generateBackupID() {
if (this->deviceID.empty()) { if (this->deviceID.empty()) {
throw std::runtime_error( throw std::runtime_error(
"trying to generate a backup ID with an empty device ID"); "trying to generate a backup ID with an empty device ID");
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines case State::DATA_HASH: {
if (database::DatabaseManager::getInstance().findBackupItem( if (database::DatabaseManager::getInstance().findBackupItem(
this->userID, this->backupID) != nullptr) { this->userID, this->backupID) != nullptr) {
throw std::runtime_error( throw std::runtime_error(
"Backup with id [" + this->backupID + "] for user [" + "Backup with id [" + this->backupID + "] for user [" +
this->userID + "] already exists, creation aborted"); this->userID + "] already exists, creation aborted");
} }
response->set_backupid(this->backupID); response->set_backupid(this->backupID);
this->holder = tools::generateHolder(this->dataHash, this->backupID); this->holder = tools::generateHolder(this->dataHash, this->backupID);
// todo:blob perform put:initialize bool dataExists = false;
tomekUnsubmitted
Not Done
Inline Actions

This isn't used anywhere

tomek: This isn't used anywhere
karolAuthorUnsubmitted
Done
Inline Actions

right

karol: right
try {
put_client_initialize_cxx();
put_client_write_cxx(
tools::getBlobPutField(blob::PutRequest::DataCase::kHolder),
this->holder.c_str());
put_client_blocking_read_cxx(); // todo this should be avoided
// (blocking); we should be able to
// ignore responses; we probably want to
// delegate performing ops to separate
// threads in the base reactors
tomekUnsubmitted
Not Done
Inline Actions

We should create some tasks that cover things mentioned in the comment

tomek: We should create some tasks that cover things mentioned in the comment
karolAuthorUnsubmitted
Done
Inline Actions

https://linear.app/comm/issue/ENG-1724/solve-the-problem-with-blocked-reactors-in-grpc

karol: https://linear.app/comm/issue/ENG-1724/solve-the-problem-with-blocked-reactors-in-grpc
put_client_write_cxx(
tools::getBlobPutField(blob::PutRequest::DataCase::kBlobHash),
this->dataHash.c_str());
rust::String responseStr =
put_client_blocking_read_cxx(); // todo this should be avoided
tomekUnsubmitted
Not Done
Inline Actions

This is confusing. We call put_client_blocking_read_cxx twice, and we ignore the result once. Could you explain (preferably in code comment) what's going on here?

tomek: This is confusing. We call `put_client_blocking_read_cxx` twice, and we ignore the result once.
karolAuthorUnsubmitted
Done
Inline Actions

We read the response to unblock the reading queue. grpc in the bidi stream always yields a response even if it's empty. And it is in this case, that's why we ignore it, but we read so the queue's not blocked (this queue is a rust's channel with MPSC_CHANNEL_BUFFER_CAPACITY limit).

karol: We read the response to unblock the reading queue. grpc in the bidi stream always yields a…
// (blocking); we should be able to
// ignore responses; we probably
// want to delegate performing ops
// to separate threads in the base
// reactors
// data exists?
if ((bool)tools::charPtrToInt(responseStr.c_str())) {
return std::make_unique<ServerBidiReactorStatus>(
grpc::Status::OK, true);
}
} catch (std::exception &e) {
throw std::runtime_error(
e.what()); // todo in base reactors we can just handle std exception
// instead of keep rethrowing here
}
return nullptr; return nullptr;
} }
case State::DATA_CHUNKS: { case State::DATA_CHUNKS: {
// todo:blob perform put:add chunk if (request.mutable_newcompactionchunk()->empty()) {
// (std::move(*request.mutable_newcompactionchunk()) return std::make_unique<ServerBidiReactorStatus>(grpc::Status::OK);
}
try {
put_client_write_cxx(
tools::getBlobPutField(blob::PutRequest::DataCase::kDataChunk),
std::string(std::move(*request.mutable_newcompactionchunk()))
.c_str());
put_client_blocking_read_cxx(); // todo this should be avoided
tomekUnsubmitted
Not Done
Inline Actions

Why do we need to read after write?

tomek: Why do we need to read after write?
karolAuthorUnsubmitted
Done
Inline Actions

Explained in the other comment

karol: Explained in the other comment
// (blocking); we should be able to
// ignore responses; we probably want to
// delegate performing ops to separate
// threads in the base reactors
} catch (std::exception &e) {
throw std::runtime_error(
e.what()); // todo in base reactors we can just handle std exception
// instead of keep rethrowing here
}
return nullptr; return nullptr;
} }
} }
throw std::runtime_error("new backup - invalid state"); throw std::runtime_error("new backup - invalid state");
} }
void CreateNewBackupReactor::terminateCallback() { void CreateNewBackupReactor::terminateCallback() {
const std::lock_guard<std::mutex> lock(this->reactorStateMutex); const std::lock_guard<std::mutex> lock(this->reactorStateMutex);
// todo:blob perform put:add chunk ("") try {
// todo:blob perform put:wait for completion put_client_terminate_cxx();
} catch (std::exception &e) {
throw std::runtime_error(
e.what()); // todo in base reactors we can just handle std exception
// instead of keep rethrowing here
}
// TODO add recovery data // TODO add recovery data
// TODO handle attachments holders // TODO handle attachments holders
database::BackupItem backupItem( database::BackupItem backupItem(
this->userID, this->userID,
this->backupID, this->backupID,
tools::getCurrentTimestamp(), tools::getCurrentTimestamp(),
tools::generateRandomString(), tools::generateRandomString(),
this->holder, this->holder,
{}); {});
database::DatabaseManager::getInstance().putBackupItem(backupItem); database::DatabaseManager::getInstance().putBackupItem(backupItem);
} }
} // namespace reactor } // namespace reactor
} // namespace network } // namespace network
} // namespace comm } // namespace comm
View Options

native/native-modules.js

#include "CreateNewBackupReactor.h" #include "CreateNewBackupReactor.h"
#include "DatabaseManager.h" #include "DatabaseManager.h"
#include "GlobalTools.h" #include "GlobalTools.h"
#include "Tools.h" #include "Tools.h"
#include "blob_client/src/lib.rs.h"
namespace comm { namespace comm {
namespace network { namespace network {
namespace reactor { namespace reactor {
std::string CreateNewBackupReactor::generateBackupID() { std::string CreateNewBackupReactor::generateBackupID() {
if (this->deviceID.empty()) { if (this->deviceID.empty()) {
throw std::runtime_error( throw std::runtime_error(
"trying to generate a backup ID with an empty device ID"); "trying to generate a backup ID with an empty device ID");
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines case State::DATA_HASH: {
if (database::DatabaseManager::getInstance().findBackupItem( if (database::DatabaseManager::getInstance().findBackupItem(
this->userID, this->backupID) != nullptr) { this->userID, this->backupID) != nullptr) {
throw std::runtime_error( throw std::runtime_error(
"Backup with id [" + this->backupID + "] for user [" + "Backup with id [" + this->backupID + "] for user [" +
this->userID + "] already exists, creation aborted"); this->userID + "] already exists, creation aborted");
} }
response->set_backupid(this->backupID); response->set_backupid(this->backupID);
this->holder = tools::generateHolder(this->dataHash, this->backupID); this->holder = tools::generateHolder(this->dataHash, this->backupID);
// todo:blob perform put:initialize bool dataExists = false;
tomekUnsubmitted
Not Done
Inline Actions

This isn't used anywhere

tomek: This isn't used anywhere
karolAuthorUnsubmitted
Done
Inline Actions

right

karol: right
try {
put_client_initialize_cxx();
put_client_write_cxx(
tools::getBlobPutField(blob::PutRequest::DataCase::kHolder),
this->holder.c_str());
put_client_blocking_read_cxx(); // todo this should be avoided
// (blocking); we should be able to
// ignore responses; we probably want to
// delegate performing ops to separate
// threads in the base reactors
tomekUnsubmitted
Not Done
Inline Actions

We should create some tasks that cover things mentioned in the comment

tomek: We should create some tasks that cover things mentioned in the comment
karolAuthorUnsubmitted
Done
Inline Actions

https://linear.app/comm/issue/ENG-1724/solve-the-problem-with-blocked-reactors-in-grpc

karol: https://linear.app/comm/issue/ENG-1724/solve-the-problem-with-blocked-reactors-in-grpc
put_client_write_cxx(
tools::getBlobPutField(blob::PutRequest::DataCase::kBlobHash),
this->dataHash.c_str());
rust::String responseStr =
put_client_blocking_read_cxx(); // todo this should be avoided
tomekUnsubmitted
Not Done
Inline Actions

This is confusing. We call put_client_blocking_read_cxx twice, and we ignore the result once. Could you explain (preferably in code comment) what's going on here?

tomek: This is confusing. We call `put_client_blocking_read_cxx` twice, and we ignore the result once.
karolAuthorUnsubmitted
Done
Inline Actions

We read the response to unblock the reading queue. grpc in the bidi stream always yields a response even if it's empty. And it is in this case, that's why we ignore it, but we read so the queue's not blocked (this queue is a rust's channel with MPSC_CHANNEL_BUFFER_CAPACITY limit).

karol: We read the response to unblock the reading queue. grpc in the bidi stream always yields a…
// (blocking); we should be able to
// ignore responses; we probably
// want to delegate performing ops
// to separate threads in the base
// reactors
// data exists?
if ((bool)tools::charPtrToInt(responseStr.c_str())) {
return std::make_unique<ServerBidiReactorStatus>(
grpc::Status::OK, true);
}
} catch (std::exception &e) {
throw std::runtime_error(
e.what()); // todo in base reactors we can just handle std exception
// instead of keep rethrowing here
}
return nullptr; return nullptr;
} }
case State::DATA_CHUNKS: { case State::DATA_CHUNKS: {
// todo:blob perform put:add chunk if (request.mutable_newcompactionchunk()->empty()) {
// (std::move(*request.mutable_newcompactionchunk()) return std::make_unique<ServerBidiReactorStatus>(grpc::Status::OK);
}
try {
put_client_write_cxx(
tools::getBlobPutField(blob::PutRequest::DataCase::kDataChunk),
std::string(std::move(*request.mutable_newcompactionchunk()))
.c_str());
put_client_blocking_read_cxx(); // todo this should be avoided
tomekUnsubmitted
Not Done
Inline Actions

Why do we need to read after write?

tomek: Why do we need to read after write?
karolAuthorUnsubmitted
Done
Inline Actions

Explained in the other comment

karol: Explained in the other comment
// (blocking); we should be able to
// ignore responses; we probably want to
// delegate performing ops to separate
// threads in the base reactors
} catch (std::exception &e) {
throw std::runtime_error(
e.what()); // todo in base reactors we can just handle std exception
// instead of keep rethrowing here
}
return nullptr; return nullptr;
} }
} }
throw std::runtime_error("new backup - invalid state"); throw std::runtime_error("new backup - invalid state");
} }
void CreateNewBackupReactor::terminateCallback() { void CreateNewBackupReactor::terminateCallback() {
const std::lock_guard<std::mutex> lock(this->reactorStateMutex); const std::lock_guard<std::mutex> lock(this->reactorStateMutex);
// todo:blob perform put:add chunk ("") try {
// todo:blob perform put:wait for completion put_client_terminate_cxx();
} catch (std::exception &e) {
throw std::runtime_error(
e.what()); // todo in base reactors we can just handle std exception
// instead of keep rethrowing here
}
// TODO add recovery data // TODO add recovery data
// TODO handle attachments holders // TODO handle attachments holders
database::BackupItem backupItem( database::BackupItem backupItem(
this->userID, this->userID,
this->backupID, this->backupID,
tools::getCurrentTimestamp(), tools::getCurrentTimestamp(),
tools::generateRandomString(), tools::generateRandomString(),
this->holder, this->holder,
{}); {});
database::DatabaseManager::getInstance().putBackupItem(backupItem); database::DatabaseManager::getInstance().putBackupItem(backupItem);
} }
} // namespace reactor } // namespace reactor
} // namespace network } // namespace network
} // namespace comm } // namespace comm