Paths

Table of Contentst

-
nix/
-
dev-shell.nix
-
scripts/
-
install_homebrew_deps.sh

[nix] Replaces brew install of watchman with nix
ClosedPublic
Actions

Authored by will on Jul 16 2024, 7:25 AM.

Details

Reviewers

ashoat
bartek
varun

Group Reviewers

Restricted Owners Package

(Owns No Changed Paths)

Commits

rCOMM30b605adaa15: [nix] Replaces brew install of watchman with nix

Summary

Because we updated nix packages, we can use nix-based watchman instead of the brew install

Context: https://linear.app/comm/issue/ENG-2784/add-updated-nix-version-of-watchman#comment-79fd27f8

Test Plan

 ➜ whereis watchman
watchman: /nix/store/pmvk9zd5z98b5vvc83w4dqjq7d4w8ri1-watchman-2023.08.14.00/bin/watchman

To verify the new Nix watchman is working. I uninstalled brew-based watchman and ran nix develop. I then ran the iOS simulator and made changes to text on the login screen, verifying that changes were immediate

Diff Detail

Repository

rCOMM Comm

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

will created this revision.Jul 16 2024, 7:25 AM

Owners added a reviewer: Restricted Owners Package.Jul 16 2024, 7:25 AM

Herald added a subscriber: tomek. · View Herald TranscriptJul 16 2024, 7:25 AM

We can have devs uninstall brew-based watchman but the nix shell watchman takes precedence

Can you modify the Test Plan to actually test that Watchman works with React Native?

Harbormaster completed remote builds in B30388: Diff 42351.Jul 16 2024, 7:48 AM

will requested review of this revision.Jul 16 2024, 7:48 AM

will edited the test plan for this revision. (Show Details)Jul 16 2024, 7:54 AM

ashoat accepted this revision.Jul 16 2024, 11:59 AM

ashoat added inline comments.

nix/dev-shell.nix
143 ↗	(On Diff #42351)	Do we need this step anymore? I noticed `prompt_direnv_macos.sh` calls Brew, so guessing yes
145 ↗	(On Diff #42351)	This is the only place that script is used. Should we delete the script?

This revision is now accepted and ready to land.Jul 16 2024, 11:59 AM

will added inline comments.Jul 16 2024, 12:12 PM

nix/dev-shell.nix
143 ↗	(On Diff #42351)	On looking at the diff where this is introduced (https://phab.comm.dev/D6313), Jon also mentions that we might need it for direnv. Seems to check out with the `prompt_direnv_macos.sh` script so I think it's best we keep it
145 ↗	(On Diff #42351)	Was debating on this. On second thought, if we have no plans to reintroduce any brew-based installations, we can delete for now. We always have the prior diffs if we want to recreate the script. I'll delete before landing

feedback. remove unnecessary homebrew install deps script

Harbormaster completed remote builds in B30402: Diff 42373.Jul 16 2024, 8:18 PM

Closed by commit rCOMM30b605adaa15: [nix] Replaces brew install of watchman with nix (authored by will). · Explain WhyJul 17 2024, 6:13 AM

This revision was automatically updated to reflect the committed changes.

will added a commit: rCOMM30b605adaa15: [nix] Replaces brew install of watchman with nix.

will mentioned this in D12848: [nix] revert nix watchman to use brew.Jul 22 2024, 12:30 PM

Revision Contents
Changeset List

Path

Size

nix/

dev-shell.nix

4 lines

scripts/

install_homebrew_deps.sh

Diff 42383

View Options

nix/dev-shell.nix

#include "NotificationsCryptoModule.h"		#include "NotificationsCryptoModule.h"
#include "../../CryptoTools/Persist.h"		#include "../../CryptoTools/Persist.h"
#include "../../CryptoTools/Tools.h"		#include "../../CryptoTools/Tools.h"
#include "../../Tools/CommMMKV.h"		#include "../../Tools/CommMMKV.h"
#include "../../Tools/CommSecureStore.h"		#include "../../Tools/CommSecureStore.h"
#include "../../Tools/Logger.h"		#include "../../Tools/Logger.h"
#include "../../Tools/PlatformSpecificTools.h"		#include "../../Tools/PlatformSpecificTools.h"

		#include <fcntl.h>
#include <folly/String.h>		#include <folly/String.h>
#include <folly/dynamic.h>		#include <folly/dynamic.h>
#include <folly/json.h>		#include <folly/json.h>
		#include <unistd.h>
#include <fstream>		#include <fstream>
#include <memory>		#include <memory>
#include <sstream>		#include <sstream>

namespace comm {		namespace comm {
		const std::string
		NotificationsCryptoModule::secureStoreNotificationsAccountDataKey =
		"notificationsCryptoAccountDataKey";
const std::string NotificationsCryptoModule::notificationsCryptoAccountID =		const std::string NotificationsCryptoModule::notificationsCryptoAccountID =
"notificationsCryptoAccountDataID";		"notificationsCryptoAccountDataID";
		const std::string NotificationsCryptoModule::keyserverHostedNotificationsID =
		"keyserverHostedNotificationsID";
const std::string NotificationsCryptoModule::initialEncryptedMessageContent =		const std::string NotificationsCryptoModule::initialEncryptedMessageContent =
"{\"type\": \"init\"}";		"{\"type\": \"init\"}";
const int NotificationsCryptoModule::olmEncryptedTypeMessage = 1;		const int NotificationsCryptoModule::olmEncryptedTypeMessage = 1;

// This constant is only used to migrate the existing notifications		// This constant is only used to migrate the existing notifications
// session with production keyserver from flat file to MMKV. This		// session with production keyserver from flat file to MMKV. This
// migration will fire when user updates the app. It will also fire		// migration will fire when user updates the app. It will also fire
// on dev env provided old keyserver set up is used. Developers willing		// on dev env provided old keyserver set up is used. Developers willing
// to use new keyserver set up must log out before installing updated		// to use new keyserver set up must log out before installing updated
// app version. Do not introduce new usages of this constant in the code!!!		// app version. Do not introduce new usages of this constant in the code!!!
const std::string ashoatKeyserverIDUsedOnlyForMigrationFromLegacyNotifStorage =		const std::string ashoatKeyserverIDUsedOnlyForMigrationFromLegacyNotifStorage =
"256";		"256";
		const int temporaryFilePathRandomSuffixLength = 32;

std::unique_ptr<crypto::CryptoModule>		std::unique_ptr<crypto::CryptoModule>
NotificationsCryptoModule::deserializeCryptoModule(		NotificationsCryptoModule::deserializeCryptoModule(
const std::string &path,		const std::string &path,
const std::string &picklingKey) {		const std::string &picklingKey) {
std::ifstream pickledPersistStream(path, std::ifstream::in);		std::ifstream pickledPersistStream(path, std::ifstream::in);
if (!pickledPersistStream.good()) {		if (!pickledPersistStream.good()) {
throw std::runtime_error(		throw std::runtime_error(
Show All 32 Lines	sessions[targetUserID] =
std::vector<uint8_t>(sessionData.begin(), sessionData.end());		std::vector<uint8_t>(sessionData.begin(), sessionData.end());
}		}
return std::make_unique<crypto::CryptoModule>(		return std::make_unique<crypto::CryptoModule>(
notificationsCryptoAccountID,		notificationsCryptoAccountID,
picklingKey,		picklingKey,
crypto::Persist({account, sessions}));		crypto::Persist({account, sessions}));
}		}

		void NotificationsCryptoModule::serializeAndFlushCryptoModule(
		std::unique_ptr<crypto::CryptoModule> cryptoModule,
		const std::string &path,
		const std::string &picklingKey) {
		crypto::Persist persist = cryptoModule->storeAsB64(picklingKey);

		folly::dynamic sessions = folly::dynamic::object;
		for (auto &sessionKeyValuePair : persist.sessions) {
		std::string targetUserID = sessionKeyValuePair.first;
		crypto::OlmBuffer sessionData = sessionKeyValuePair.second;
		sessions[targetUserID] =
		std::string(sessionData.begin(), sessionData.end());
		}

		std::string account =
		std::string(persist.account.begin(), persist.account.end());
		folly::dynamic persistJSON =
		folly::dynamic::object("account", account)("sessions", sessions);
		std::string pickledPersist = folly::toJson(persistJSON);

		std::string temporaryFilePathRandomSuffix =
		crypto::Tools::generateRandomHexString(
		temporaryFilePathRandomSuffixLength);
		std::string temporaryPath = path + temporaryFilePathRandomSuffix;

		mode_t readWritePermissionsMode = 0666;
		int temporaryFD =
		open(temporaryPath.c_str(), O_CREAT \| O_WRONLY, readWritePermissionsMode);
		if (temporaryFD == -1) {
		throw std::runtime_error(
		"Failed to create temporary file. Unable to atomically update "
		"notifications crypto account. Details: " +
		std::string(strerror(errno)));
		}
		ssize_t bytesWritten =
		write(temporaryFD, pickledPersist.c_str(), pickledPersist.length());
		if (bytesWritten == -1 \|\| bytesWritten != pickledPersist.length()) {
		remove(temporaryPath.c_str());
		throw std::runtime_error(
		"Failed to write all data to temporary file. Unable to atomically "
		"update notifications crypto account. Details: " +
		std::string(strerror(errno)));
		}
		if (fsync(temporaryFD) == -1) {
		remove(temporaryPath.c_str());
		throw std::runtime_error(
		"Failed to synchronize temporary file data with hardware storage. "
		"Unable to atomically update notifications crypto account. Details: " +
		std::string(strerror(errno)));
		};
		close(temporaryFD);
		if (rename(temporaryPath.c_str(), path.c_str()) == -1) {
		remove(temporaryPath.c_str());
		throw std::runtime_error(
		"Failed to replace temporary file content with notifications crypto "
		"account. Unable to atomically update notifications crypto account. "
		"Details: " +
		std::string(strerror(errno)));
		}
		remove(temporaryPath.c_str());
		}

std::string NotificationsCryptoModule::getKeyserverNotificationsSessionKey(		std::string NotificationsCryptoModule::getKeyserverNotificationsSessionKey(
const std::string &keyserverID) {		const std::string &keyserverID) {
return "KEYSERVER." + keyserverID + ".NOTIFS_SESSION";		return "KEYSERVER." + keyserverID + ".NOTIFS_SESSION";
}		}

std::string NotificationsCryptoModule::serializeNotificationsSession(		std::string NotificationsCryptoModule::serializeNotificationsSession(
std::shared_ptr<crypto::Session> session,		std::shared_ptr<crypto::Session> session,
std::string picklingKey) {		std::string picklingKey) {
▲ Show 20 Lines • Show All 88 Lines • ▼ Show 20 Lines	void NotificationsCryptoModule::persistNotificationsSessionInternal(

if (!sessionStored) {		if (!sessionStored) {
throw std::runtime_error(		throw std::runtime_error(
"Failed to persist to MMKV notifications session for keyserver: " +		"Failed to persist to MMKV notifications session for keyserver: " +
keyserverID);		keyserverID);
}		}
}		}

std::pair<std::unique_ptr<crypto::Session>, std::string>		std::optional<std::pair<std::unique_ptr<crypto::Session>, std::string>>
NotificationsCryptoModule::fetchNotificationsSession(		NotificationsCryptoModule::fetchNotificationsSession(
const std::string &keyserverID) {		const std::string &keyserverID) {
std::string keyserverNotificationsSessionKey =		std::string keyserverNotificationsSessionKey =
NotificationsCryptoModule::getKeyserverNotificationsSessionKey(		NotificationsCryptoModule::getKeyserverNotificationsSessionKey(
keyserverID);		keyserverID);
std::optional<std::string> serializedSession =
CommMMKV::getString(keyserverNotificationsSessionKey);

if (!serializedSession.has_value()) {		std::optional<std::string> serializedSession;
		try {
		serializedSession = CommMMKV::getString(keyserverNotificationsSessionKey);
		} catch (const CommMMKV::InitFromNSEForbiddenError &e) {
		serializedSession = std::nullopt;
		}

		if (!serializedSession.has_value() &&
		keyserverID !=
		ashoatKeyserverIDUsedOnlyForMigrationFromLegacyNotifStorage) {
throw std::runtime_error(		throw std::runtime_error(
"Missing notifications session for keyserver: " + keyserverID);		"Missing notifications session for keyserver: " + keyserverID);
		} else if (!serializedSession.has_value()) {
		return std::nullopt;
}		}

return NotificationsCryptoModule::deserializeNotificationsSession(		return NotificationsCryptoModule::deserializeNotificationsSession(
serializedSession.value());		serializedSession.value());
}		}

void NotificationsCryptoModule::persistNotificationsSession(		void NotificationsCryptoModule::persistNotificationsSession(
const std::string &keyserverID,		const std::string &keyserverID,
std::shared_ptr<crypto::Session> keyserverNotificationsSession) {		std::shared_ptr<crypto::Session> keyserverNotificationsSession) {
std::string picklingKey = crypto::Tools::generateRandomString(64);		std::string picklingKey = crypto::Tools::generateRandomString(64);
NotificationsCryptoModule::persistNotificationsSessionInternal(		NotificationsCryptoModule::persistNotificationsSessionInternal(
keyserverID, picklingKey, keyserverNotificationsSession);		keyserverID, picklingKey, keyserverNotificationsSession);
}		}

bool NotificationsCryptoModule::isNotificationsSessionInitialized(		bool NotificationsCryptoModule::isNotificationsSessionInitialized(
const std::string &keyserverID) {		const std::string &keyserverID) {
std::string keyserverNotificationsSessionKey =		std::string keyserverNotificationsSessionKey =
"KEYSERVER." + keyserverID + ".NOTIFS_SESSION";		"KEYSERVER." + keyserverID + ".NOTIFS_SESSION";
return CommMMKV::getString(keyserverNotificationsSessionKey).has_value();		return CommMMKV::getString(keyserverNotificationsSessionKey).has_value();
}		}

std::string NotificationsCryptoModule::decrypt(		NotificationsCryptoModule::BaseStatefulDecryptResult::BaseStatefulDecryptResult(
const std::string &keyserverID,		std::string picklingKey,
const std::string &data,		std::string decryptedData)
const size_t messageType) {		: picklingKey(picklingKey), decryptedData(decryptedData) {
std::unique_ptr<crypto::Session> session;		}
std::string picklingKey;
std::tie(session, picklingKey) =
NotificationsCryptoModule::fetchNotificationsSession(keyserverID);

crypto::EncryptedData encryptedData{		std::string
std::vector<uint8_t>(data.begin(), data.end()), messageType};		NotificationsCryptoModule::BaseStatefulDecryptResult::getDecryptedData() {
std::string decryptedData = session->decrypt(encryptedData);		return this->decryptedData;
NotificationsCryptoModule::persistNotificationsSessionInternal(
keyserverID, picklingKey, std::move(session));
return decryptedData;
}		}

NotificationsCryptoModule::StatefulDecryptResult::StatefulDecryptResult(		NotificationsCryptoModule::StatefulDecryptResult::StatefulDecryptResult(
std::unique_ptr<crypto::Session> session,		std::unique_ptr<crypto::Session> session,
std::string keyserverID,		std::string keyserverID,
std::string picklingKey,		std::string picklingKey,
std::string decryptedData)		std::string decryptedData)
: sessionState(std::move(session)),		: NotificationsCryptoModule::BaseStatefulDecryptResult::
keyserverID(keyserverID),		BaseStatefulDecryptResult(picklingKey, decryptedData),
picklingKey(picklingKey),		sessionState(std::move(session)),
decryptedData(decryptedData) {		keyserverID(keyserverID) {
}		}

std::string		void NotificationsCryptoModule::StatefulDecryptResult::flushState() {
NotificationsCryptoModule::StatefulDecryptResult::getDecryptedData() {		NotificationsCryptoModule::persistNotificationsSessionInternal(
return this->decryptedData;		this->keyserverID, this->picklingKey, std::move(this->sessionState));
}		}

std::string NotificationsCryptoModule::StatefulDecryptResult::getKeyserverID() {		NotificationsCryptoModule::LegacyStatefulDecryptResult::
return this->keyserverID;		LegacyStatefulDecryptResult(
		std::unique_ptr<crypto::CryptoModule> cryptoModule,
		std::string path,
		std::string picklingKey,
		std::string decryptedData)
		: NotificationsCryptoModule::BaseStatefulDecryptResult::
		BaseStatefulDecryptResult(picklingKey, decryptedData),
		path(path),
		cryptoModule(std::move(cryptoModule)) {
}		}

std::string NotificationsCryptoModule::StatefulDecryptResult::getPicklingKey() {		void NotificationsCryptoModule::LegacyStatefulDecryptResult::flushState() {
return this->picklingKey;		NotificationsCryptoModule::serializeAndFlushCryptoModule(
		std::move(this->cryptoModule), this->path, this->picklingKey);
}		}

std::unique_ptr<NotificationsCryptoModule::StatefulDecryptResult>		std::unique_ptr<NotificationsCryptoModule::BaseStatefulDecryptResult>
NotificationsCryptoModule::statefulDecrypt(		NotificationsCryptoModule::prepareLegacyDecryptedState(
		const std::string &data,
		const size_t messageType) {
		folly::Optional<std::string> picklingKey = comm::CommSecureStore::get(
		NotificationsCryptoModule::secureStoreNotificationsAccountDataKey);

		if (!picklingKey.hasValue()) {
		throw std::runtime_error(
		"Legacy notifications session pickling key missing.");
		}

		std::string legacyNotificationsAccountPath =
		comm::PlatformSpecificTools::getNotificationsCryptoAccountPath();

		crypto::EncryptedData encryptedData{
		std::vector<uint8_t>(data.begin(), data.end()), messageType};

		auto cryptoModule = NotificationsCryptoModule::deserializeCryptoModule(
		legacyNotificationsAccountPath, picklingKey.value());

		std::string decryptedData = cryptoModule->decrypt(
		NotificationsCryptoModule::keyserverHostedNotificationsID, encryptedData);

		LegacyStatefulDecryptResult statefulDecryptResult(
		std::move(cryptoModule),
		legacyNotificationsAccountPath,
		picklingKey.value(),
		decryptedData);

		return std::make_unique<LegacyStatefulDecryptResult>(
		std::move(statefulDecryptResult));
		}

		std::string NotificationsCryptoModule::decrypt(
const std::string &keyserverID,		const std::string &keyserverID,
const std::string &data,		const std::string &data,
const size_t messageType) {		const size_t messageType) {
std::unique_ptr<crypto::Session> session;
std::string picklingKey;		auto sessionWithPicklingKey =
std::tie(session, picklingKey) =
NotificationsCryptoModule::fetchNotificationsSession(keyserverID);		NotificationsCryptoModule::fetchNotificationsSession(keyserverID);
		if (!sessionWithPicklingKey.has_value()) {
		auto statefulDecryptResult =
		NotificationsCryptoModule::prepareLegacyDecryptedState(
		data, messageType);
		statefulDecryptResult->flushState();
		return statefulDecryptResult->getDecryptedData();
		}

		std::unique_ptr<crypto::Session> session =
		std::move(sessionWithPicklingKey.value().first);
		std::string picklingKey = sessionWithPicklingKey.value().second;
crypto::EncryptedData encryptedData{		crypto::EncryptedData encryptedData{
std::vector<uint8_t>(data.begin(), data.end()), messageType};		std::vector<uint8_t>(data.begin(), data.end()), messageType};

std::string decryptedData = session->decrypt(encryptedData);		std::string decryptedData = session->decrypt(encryptedData);
		NotificationsCryptoModule::persistNotificationsSessionInternal(
		keyserverID, picklingKey, std::move(session));
		return decryptedData;
		}

		std::unique_ptr<NotificationsCryptoModule::BaseStatefulDecryptResult>
		NotificationsCryptoModule::statefulDecrypt(
		const std::string &keyserverID,
		const std::string &data,
		const size_t messageType) {

		auto sessionWithPicklingKey =
		NotificationsCryptoModule::fetchNotificationsSession(keyserverID);
		if (!sessionWithPicklingKey.has_value()) {
		return NotificationsCryptoModule::prepareLegacyDecryptedState(
		data, messageType);
		}

		std::unique_ptr<crypto::Session> session =
		std::move(sessionWithPicklingKey.value().first);
		std::string picklingKey = sessionWithPicklingKey.value().second;
		crypto::EncryptedData encryptedData{
		std::vector<uint8_t>(data.begin(), data.end()), messageType};
		std::string decryptedData = session->decrypt(encryptedData);
StatefulDecryptResult statefulDecryptResult(		StatefulDecryptResult statefulDecryptResult(
std::move(session), keyserverID, picklingKey, decryptedData);		std::move(session), keyserverID, picklingKey, decryptedData);

return std::make_unique<StatefulDecryptResult>(		return std::make_unique<StatefulDecryptResult>(
std::move(statefulDecryptResult));		std::move(statefulDecryptResult));
}		}

void NotificationsCryptoModule::flushState(		void NotificationsCryptoModule::flushState(
std::unique_ptr<StatefulDecryptResult> statefulDecryptResult) {		std::unique_ptr<BaseStatefulDecryptResult> baseStatefulDecryptResult) {
NotificationsCryptoModule::persistNotificationsSessionInternal(		baseStatefulDecryptResult->flushState();
statefulDecryptResult->getKeyserverID(),
statefulDecryptResult->getPicklingKey(),
std::move(statefulDecryptResult->sessionState));
}		}
} // namespace comm		} // namespace comm