Paths

Table of Contentst

-
services/identity/src/
-
identity/
-
src/
-
client_service.rs
-
constants.rs
-
database.rs
-
database/
-
device_list.rs
-
farcaster.rs
-
one_time_keys.rs
-
token.rs
-
grpc_services/
-
authenticated.rs
-
siwe.rs
-
sync_identity_search.rs
-
tunnelbroker.rs
-
websockets/
-
auth.rs
-
errors.rs
-
mod.rs
-
send.rs

Differential D11852

[identity] Add prefixes to error logs for filtering
ClosedPublic
Actions

Authored by will on May 1 2024, 8:03 PM.

Details

Reviewers

bartek

Commits

rCOMM6190b986ef59: [identity] Add prefixes to error logs for filtering

Summary

This adds prefixes to our error! tracing logs on the identity service

Depends on D11831

Test Plan

cargo check

Diff Detail

Repository

rCOMM Comm

Lint

No Lint Coverage

Unit

No Test Coverage

Event Timeline

will created this revision.May 1 2024, 8:03 PM

Herald added subscribers: tomek, ashoat. · View Herald TranscriptMay 1 2024, 8:03 PM

will added a child revision: D11853: [terraform] metric filters for error logs in identity service.May 1 2024, 8:10 PM

will edited the summary of this revision. (Show Details)

will added a parent revision: D11831: [identity] add span tracing to identity farcaster.rs.

Harbormaster completed remote builds in B28618: Diff 39725.May 1 2024, 8:20 PM

will requested review of this revision.May 1 2024, 8:20 PM

I was imagining that all the prefixes (e.g. DB Error) would be defined in a module in constants.rs and then used like this:

pub const DB_ERROR_LOG_PREFIX: &str = "DB Error";

error!(DB_ERROR_LOG_PREFIX, "Encountered an unexpected error: {}", e);

This revision now requires changes to proceed.May 2 2024, 11:28 AM

varun mentioned this in D11854: [terraform] identity service cloudwatch alarms.May 2 2024, 11:34 AM

In D11852#339861, @varun wrote:
I was imagining that all the prefixes (e.g. DB Error) would be defined in a module in constants.rs and then used like this:
pub const DB_ERROR_LOG_PREFIX: &str = "DB Error";

error!(DB_ERROR_LOG_PREFIX, "Encountered an unexpected error: {}", e);

So that would look something like this on output:

{"message":"Encountered an unexpected error: e","DB_ERROR_LOG_PREFIX":"DB Error"}

Currently my alarms match on the message field. I think it overcomplicates the cloudwatch patterns if we're accessing each possible prefix variable name so what might be best is each error log specifies some field:

error!(errorType=DB_ERROR_LOG_PREFIX, "Encountered an unexpected error: {}", e);

would get us:

{"message":"Encountered an unexpected error: e", "errorType":"DB Error"}

I thought about how we'd be able to store the key value "errorType = DB_ERROR_LOG_PREFIX" in our const file so we wouldn't have to include errorType each time, but I don't think it's possible unless we do something really hacky with mods

Another option is a new error log helper function?

@varun's concern is valid to me. I like the errorType = ... approach.

In D11852#339879, @will wrote:
what might be best is each error log specifies some field:
error!(errorType=DB_ERROR_LOG_PREFIX, "Encountered an unexpected error: {}", e);
would get us:
{"message":"Encountered an unexpected error: e", "errorType":"DB Error"}
I thought about how we'd be able to store the key value "errorType = DB_ERROR_LOG_PREFIX" in our const file so we wouldn't have to include errorType each time, but I don't think it's possible unless we do something really hacky with mods

I think including errorType in each log is okay for now. If this gets too annoying, we can think about creating a custom error! macro

bartek mentioned this in D11853: [terraform] metric filters for error logs in identity service.May 5 2024, 10:49 PM

review feedback

Harbormaster completed remote builds in B28716: Diff 39870.May 6 2024, 12:45 PM

bartek accepted this revision.May 7 2024, 2:54 AM

bartek added inline comments.

services/identity/src/constants.rs
196–206 ↗	(On Diff #39870)	Optionally it could be done this way, so instead of doing `errorType = GENERIC_DB_LOG_ERROR_TYPE`, it would be `errorType = error_types::GENERIC_DB_LOG`. Both approaches are okay to me

will added inline comments.May 7 2024, 7:18 AM

services/identity/src/constants.rs
196–206 ↗	(On Diff #39870)	I like your way better. It's more organized. Will include in the next rebase

review feedback and whitespace

Harbormaster completed remote builds in B28736: Diff 39894.May 7 2024, 8:19 AM

@varun is off but his feedback has been addressed so unblocking this stack

This revision is now accepted and ready to land.May 7 2024, 9:47 AM

Closed by commit rCOMM6190b986ef59: [identity] Add prefixes to error logs for filtering (authored by will). · Explain WhyMay 8 2024, 10:33 AM

This revision was automatically updated to reflect the committed changes.

will marked an inline comment as done.

will added a commit: rCOMM6190b986ef59: [identity] Add prefixes to error logs for filtering.

Revision Contents
Changeset List

Path

Size

services/

identity/

src/

client_service.rs

10 lines

constants.rs

16 lines

database.rs

27 lines

database/

105 lines

6 lines

19 lines

8 lines

grpc_services/

authenticated.rs

33 lines

siwe.rs

22 lines

sync_identity_search.rs

6 lines

tunnelbroker.rs

6 lines

websockets/

17 lines

14 lines

33 lines

7 lines

Diff 39894

View Options

services/identity/src/client_service.rs

	#import "CommMMKV.h"			#import "CommMMKV.h"
	#import "../../cpp/CommonCpp/CryptoTools/Tools.h"			#import "../../cpp/CommonCpp/CryptoTools/Tools.h"
	#import "CommSecureStore.h"			#import "CommSecureStore.h"
	#import "Logger.h"			#import "Logger.h"
	#import "Tools.h"			#import "Tools.h"

	#import <Foundation/Foundation.h>			#import <Foundation/Foundation.h>
	#import <MMKV.h>			#import <MMKV.h>
				#import <MMKVCore/MMKV.h>

				// Core MMKV C++ implementation and Android wrapper have public `lock` and
				// `unlock` methods while Obj-C wrapper doesn't. However Obj-C wrapper has
				// private instance variable of type `mmkv::MMKV *`. Interface redeclaration
				// below lets us access it. This pattern is hacky but it is well known in
				// Obj-C and used in our codebase in CommSecureStore.
				@interface MMKV () {
				@public
				mmkv::MMKV *m_mmkv;
				}
				@end

	namespace comm {			namespace comm {

	const int mmkvEncryptionKeySize = 16;			const int mmkvEncryptionKeySize = 16;
	const int mmkvIDsize = 8;			const int mmkvIDsize = 8;

	const std::string secureStoreMMKVEncryptionKeyID = "comm.mmkvEncryptionKey";			const std::string secureStoreMMKVEncryptionKeyID = "comm.mmkvEncryptionKey";
	const std::string secureStoreMMKVIdentifierKeyID = "comm.mmkvID";			const std::string secureStoreMMKVIdentifierKeyID = "comm.mmkvID";

	static NSString *mmkvEncryptionKey;			static NSString *mmkvEncryptionKey;
	static NSString *mmkvIdentifier;			static NSString *mmkvIdentifier;

	MMKV getMMKVInstance(NSString mmkvID, NSString *encryptionKey) {			MMKV getMMKVInstance(NSString mmkvID, NSString *encryptionKey) {
	MMKV *mmkv =			MMKV *mmkv =
	[MMKV mmkvWithID:mmkvID			[MMKV mmkvWithID:mmkvID
	cryptKey:[encryptionKey dataUsingEncoding:NSUTF8StringEncoding]			cryptKey:[encryptionKey dataUsingEncoding:NSUTF8StringEncoding]
	mode:MMKVMultiProcess];			mode:MMKVMultiProcess];
	if (!mmkv) {			if (!mmkv) {
	throw std::runtime_error("Failed to instantiate MMKV object.");			throw std::runtime_error("Failed to instantiate MMKV object.");
	}			}
	return mmkv;			return mmkv;
	}			}

				CommMMKV::ScopedCommMMKVLock::ScopedCommMMKVLock() {
				CommMMKV::initialize();
				MMKV *mmkv = getMMKVInstance(mmkvIdentifier, mmkvEncryptionKey);
				mmkv->m_mmkv->lock();
				}

				CommMMKV::ScopedCommMMKVLock::~ScopedCommMMKVLock() {
				MMKV *mmkv = getMMKVInstance(mmkvIdentifier, mmkvEncryptionKey);
				mmkv->m_mmkv->unlock();
				}

	void assignInitializationData() {			void assignInitializationData() {
	std::string encryptionKey =			std::string encryptionKey =
	crypto::Tools::generateRandomString(mmkvEncryptionKeySize);			crypto::Tools::generateRandomString(mmkvEncryptionKeySize);
	std::string identifier = crypto::Tools::generateRandomString(mmkvIDsize);			std::string identifier = crypto::Tools::generateRandomString(mmkvIDsize);
	CommSecureStore::set(secureStoreMMKVEncryptionKeyID, encryptionKey);			CommSecureStore::set(secureStoreMMKVEncryptionKeyID, encryptionKey);
	CommSecureStore::set(secureStoreMMKVIdentifierKeyID, identifier);			CommSecureStore::set(secureStoreMMKVIdentifierKeyID, identifier);
	mmkvEncryptionKey = [NSString stringWithCString:encryptionKey.c_str()			mmkvEncryptionKey = [NSString stringWithCString:encryptionKey.c_str()
	encoding:NSUTF8StringEncoding];			encoding:NSUTF8StringEncoding];
	▲ Show 20 Lines • Show All 153 Lines • Show Last 20 Lines

View Options

services/identity/src/constants.rs

	#import "CommMMKV.h"			#import "CommMMKV.h"
	#import "../../cpp/CommonCpp/CryptoTools/Tools.h"			#import "../../cpp/CommonCpp/CryptoTools/Tools.h"
	#import "CommSecureStore.h"			#import "CommSecureStore.h"
	#import "Logger.h"			#import "Logger.h"
	#import "Tools.h"			#import "Tools.h"

	#import <Foundation/Foundation.h>			#import <Foundation/Foundation.h>
	#import <MMKV.h>			#import <MMKV.h>
				#import <MMKVCore/MMKV.h>

				// Core MMKV C++ implementation and Android wrapper have public `lock` and
				// `unlock` methods while Obj-C wrapper doesn't. However Obj-C wrapper has
				// private instance variable of type `mmkv::MMKV *`. Interface redeclaration
				// below lets us access it. This pattern is hacky but it is well known in
				// Obj-C and used in our codebase in CommSecureStore.
				@interface MMKV () {
				@public
				mmkv::MMKV *m_mmkv;
				}
				@end

	namespace comm {			namespace comm {

	const int mmkvEncryptionKeySize = 16;			const int mmkvEncryptionKeySize = 16;
	const int mmkvIDsize = 8;			const int mmkvIDsize = 8;

	const std::string secureStoreMMKVEncryptionKeyID = "comm.mmkvEncryptionKey";			const std::string secureStoreMMKVEncryptionKeyID = "comm.mmkvEncryptionKey";
	const std::string secureStoreMMKVIdentifierKeyID = "comm.mmkvID";			const std::string secureStoreMMKVIdentifierKeyID = "comm.mmkvID";

	static NSString *mmkvEncryptionKey;			static NSString *mmkvEncryptionKey;
	static NSString *mmkvIdentifier;			static NSString *mmkvIdentifier;

	MMKV getMMKVInstance(NSString mmkvID, NSString *encryptionKey) {			MMKV getMMKVInstance(NSString mmkvID, NSString *encryptionKey) {
	MMKV *mmkv =			MMKV *mmkv =
	[MMKV mmkvWithID:mmkvID			[MMKV mmkvWithID:mmkvID
	cryptKey:[encryptionKey dataUsingEncoding:NSUTF8StringEncoding]			cryptKey:[encryptionKey dataUsingEncoding:NSUTF8StringEncoding]
	mode:MMKVMultiProcess];			mode:MMKVMultiProcess];
	if (!mmkv) {			if (!mmkv) {
	throw std::runtime_error("Failed to instantiate MMKV object.");			throw std::runtime_error("Failed to instantiate MMKV object.");
	}			}
	return mmkv;			return mmkv;
	}			}

				CommMMKV::ScopedCommMMKVLock::ScopedCommMMKVLock() {
				CommMMKV::initialize();
				MMKV *mmkv = getMMKVInstance(mmkvIdentifier, mmkvEncryptionKey);
				mmkv->m_mmkv->lock();
				}

				CommMMKV::ScopedCommMMKVLock::~ScopedCommMMKVLock() {
				MMKV *mmkv = getMMKVInstance(mmkvIdentifier, mmkvEncryptionKey);
				mmkv->m_mmkv->unlock();
				}

	void assignInitializationData() {			void assignInitializationData() {
	std::string encryptionKey =			std::string encryptionKey =
	crypto::Tools::generateRandomString(mmkvEncryptionKeySize);			crypto::Tools::generateRandomString(mmkvEncryptionKeySize);
	std::string identifier = crypto::Tools::generateRandomString(mmkvIDsize);			std::string identifier = crypto::Tools::generateRandomString(mmkvIDsize);
	CommSecureStore::set(secureStoreMMKVEncryptionKeyID, encryptionKey);			CommSecureStore::set(secureStoreMMKVEncryptionKeyID, encryptionKey);
	CommSecureStore::set(secureStoreMMKVIdentifierKeyID, identifier);			CommSecureStore::set(secureStoreMMKVIdentifierKeyID, identifier);
	mmkvEncryptionKey = [NSString stringWithCString:encryptionKey.c_str()			mmkvEncryptionKey = [NSString stringWithCString:encryptionKey.c_str()
	encoding:NSUTF8StringEncoding];			encoding:NSUTF8StringEncoding];
	▲ Show 20 Lines • Show All 153 Lines • Show Last 20 Lines

View Options

services/identity/src/database.rs

	#import "CommMMKV.h"			#import "CommMMKV.h"
	#import "../../cpp/CommonCpp/CryptoTools/Tools.h"			#import "../../cpp/CommonCpp/CryptoTools/Tools.h"
	#import "CommSecureStore.h"			#import "CommSecureStore.h"
	#import "Logger.h"			#import "Logger.h"
	#import "Tools.h"			#import "Tools.h"

	#import <Foundation/Foundation.h>			#import <Foundation/Foundation.h>
	#import <MMKV.h>			#import <MMKV.h>
				#import <MMKVCore/MMKV.h>

				// Core MMKV C++ implementation and Android wrapper have public `lock` and
				// `unlock` methods while Obj-C wrapper doesn't. However Obj-C wrapper has
				// private instance variable of type `mmkv::MMKV *`. Interface redeclaration
				// below lets us access it. This pattern is hacky but it is well known in
				// Obj-C and used in our codebase in CommSecureStore.
				@interface MMKV () {
				@public
				mmkv::MMKV *m_mmkv;
				}
				@end

	namespace comm {			namespace comm {

	const int mmkvEncryptionKeySize = 16;			const int mmkvEncryptionKeySize = 16;
	const int mmkvIDsize = 8;			const int mmkvIDsize = 8;

	const std::string secureStoreMMKVEncryptionKeyID = "comm.mmkvEncryptionKey";			const std::string secureStoreMMKVEncryptionKeyID = "comm.mmkvEncryptionKey";
	const std::string secureStoreMMKVIdentifierKeyID = "comm.mmkvID";			const std::string secureStoreMMKVIdentifierKeyID = "comm.mmkvID";

	static NSString *mmkvEncryptionKey;			static NSString *mmkvEncryptionKey;
	static NSString *mmkvIdentifier;			static NSString *mmkvIdentifier;

	MMKV getMMKVInstance(NSString mmkvID, NSString *encryptionKey) {			MMKV getMMKVInstance(NSString mmkvID, NSString *encryptionKey) {
	MMKV *mmkv =			MMKV *mmkv =
	[MMKV mmkvWithID:mmkvID			[MMKV mmkvWithID:mmkvID
	cryptKey:[encryptionKey dataUsingEncoding:NSUTF8StringEncoding]			cryptKey:[encryptionKey dataUsingEncoding:NSUTF8StringEncoding]
	mode:MMKVMultiProcess];			mode:MMKVMultiProcess];
	if (!mmkv) {			if (!mmkv) {
	throw std::runtime_error("Failed to instantiate MMKV object.");			throw std::runtime_error("Failed to instantiate MMKV object.");
	}			}
	return mmkv;			return mmkv;
	}			}

				CommMMKV::ScopedCommMMKVLock::ScopedCommMMKVLock() {
				CommMMKV::initialize();
				MMKV *mmkv = getMMKVInstance(mmkvIdentifier, mmkvEncryptionKey);
				mmkv->m_mmkv->lock();
				}

				CommMMKV::ScopedCommMMKVLock::~ScopedCommMMKVLock() {
				MMKV *mmkv = getMMKVInstance(mmkvIdentifier, mmkvEncryptionKey);
				mmkv->m_mmkv->unlock();
				}

	void assignInitializationData() {			void assignInitializationData() {
	std::string encryptionKey =			std::string encryptionKey =
	crypto::Tools::generateRandomString(mmkvEncryptionKeySize);			crypto::Tools::generateRandomString(mmkvEncryptionKeySize);
	std::string identifier = crypto::Tools::generateRandomString(mmkvIDsize);			std::string identifier = crypto::Tools::generateRandomString(mmkvIDsize);
	CommSecureStore::set(secureStoreMMKVEncryptionKeyID, encryptionKey);			CommSecureStore::set(secureStoreMMKVEncryptionKeyID, encryptionKey);
	CommSecureStore::set(secureStoreMMKVIdentifierKeyID, identifier);			CommSecureStore::set(secureStoreMMKVIdentifierKeyID, identifier);
	mmkvEncryptionKey = [NSString stringWithCString:encryptionKey.c_str()			mmkvEncryptionKey = [NSString stringWithCString:encryptionKey.c_str()
	encoding:NSUTF8StringEncoding];			encoding:NSUTF8StringEncoding];
	▲ Show 20 Lines • Show All 153 Lines • Show Last 20 Lines

View Options

services/identity/src/database/device_list.rs

	#import "CommMMKV.h"			#import "CommMMKV.h"
	#import "../../cpp/CommonCpp/CryptoTools/Tools.h"			#import "../../cpp/CommonCpp/CryptoTools/Tools.h"
	#import "CommSecureStore.h"			#import "CommSecureStore.h"
	#import "Logger.h"			#import "Logger.h"
	#import "Tools.h"			#import "Tools.h"

	#import <Foundation/Foundation.h>			#import <Foundation/Foundation.h>
	#import <MMKV.h>			#import <MMKV.h>
				#import <MMKVCore/MMKV.h>

				// Core MMKV C++ implementation and Android wrapper have public `lock` and
				// `unlock` methods while Obj-C wrapper doesn't. However Obj-C wrapper has
				// private instance variable of type `mmkv::MMKV *`. Interface redeclaration
				// below lets us access it. This pattern is hacky but it is well known in
				// Obj-C and used in our codebase in CommSecureStore.
				@interface MMKV () {
				@public
				mmkv::MMKV *m_mmkv;
				}
				@end

	namespace comm {			namespace comm {

	const int mmkvEncryptionKeySize = 16;			const int mmkvEncryptionKeySize = 16;
	const int mmkvIDsize = 8;			const int mmkvIDsize = 8;

	const std::string secureStoreMMKVEncryptionKeyID = "comm.mmkvEncryptionKey";			const std::string secureStoreMMKVEncryptionKeyID = "comm.mmkvEncryptionKey";
	const std::string secureStoreMMKVIdentifierKeyID = "comm.mmkvID";			const std::string secureStoreMMKVIdentifierKeyID = "comm.mmkvID";

	static NSString *mmkvEncryptionKey;			static NSString *mmkvEncryptionKey;
	static NSString *mmkvIdentifier;			static NSString *mmkvIdentifier;

	MMKV getMMKVInstance(NSString mmkvID, NSString *encryptionKey) {			MMKV getMMKVInstance(NSString mmkvID, NSString *encryptionKey) {
	MMKV *mmkv =			MMKV *mmkv =
	[MMKV mmkvWithID:mmkvID			[MMKV mmkvWithID:mmkvID
	cryptKey:[encryptionKey dataUsingEncoding:NSUTF8StringEncoding]			cryptKey:[encryptionKey dataUsingEncoding:NSUTF8StringEncoding]
	mode:MMKVMultiProcess];			mode:MMKVMultiProcess];
	if (!mmkv) {			if (!mmkv) {
	throw std::runtime_error("Failed to instantiate MMKV object.");			throw std::runtime_error("Failed to instantiate MMKV object.");
	}			}
	return mmkv;			return mmkv;
	}			}

				CommMMKV::ScopedCommMMKVLock::ScopedCommMMKVLock() {
				CommMMKV::initialize();
				MMKV *mmkv = getMMKVInstance(mmkvIdentifier, mmkvEncryptionKey);
				mmkv->m_mmkv->lock();
				}

				CommMMKV::ScopedCommMMKVLock::~ScopedCommMMKVLock() {
				MMKV *mmkv = getMMKVInstance(mmkvIdentifier, mmkvEncryptionKey);
				mmkv->m_mmkv->unlock();
				}

	void assignInitializationData() {			void assignInitializationData() {
	std::string encryptionKey =			std::string encryptionKey =
	crypto::Tools::generateRandomString(mmkvEncryptionKeySize);			crypto::Tools::generateRandomString(mmkvEncryptionKeySize);
	std::string identifier = crypto::Tools::generateRandomString(mmkvIDsize);			std::string identifier = crypto::Tools::generateRandomString(mmkvIDsize);
	CommSecureStore::set(secureStoreMMKVEncryptionKeyID, encryptionKey);			CommSecureStore::set(secureStoreMMKVEncryptionKeyID, encryptionKey);
	CommSecureStore::set(secureStoreMMKVIdentifierKeyID, identifier);			CommSecureStore::set(secureStoreMMKVIdentifierKeyID, identifier);
	mmkvEncryptionKey = [NSString stringWithCString:encryptionKey.c_str()			mmkvEncryptionKey = [NSString stringWithCString:encryptionKey.c_str()
	encoding:NSUTF8StringEncoding];			encoding:NSUTF8StringEncoding];
	▲ Show 20 Lines • Show All 153 Lines • Show Last 20 Lines