Page MenuHomePhabricator
Differential D11115

[web] Check backup client wasm integrity buildkite
AbandonedPublic
Actions

Authored by michal on Feb 20 2024, 2:40 AM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, Apr 7, 8:26 PM
Unknown Object (File)
Sat, Apr 5, 1:28 AM
Unknown Object (File)
Sun, Mar 30, 8:05 AM
Unknown Object (File)
Thu, Mar 27, 8:09 PM
Unknown Object (File)
Tue, Mar 25, 6:06 PM
Unknown Object (File)
Mar 12 2025, 5:24 PM
Unknown Object (File)
Mar 9 2025, 12:50 AM
Unknown Object (File)
Mar 6 2025, 4:39 AM
View All Files
Subscribers
ashoat
tomek

Details

Reviewers
kamil
atul
Summary

ENG-6873 : Add a buildkite job for checking integrity of backup client wasm file

Similarly to comm_query_executor we want a buildkite job to build the wasm file from source and make sure that it's the same. In this diff I'm modifiying the existing pipeline to also check for the backup client.

Test Plan

Check that the CI passes.

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

michal created this revision.Feb 20 2024, 2:40 AM
michal held this revision as a draft.
Herald added subscribers: tomek, ashoat. · View Herald TranscriptFeb 20 2024, 2:40 AM
michal added a parent revision: D11041: [backup-client] Compile to wasm.Feb 20 2024, 2:40 AM
Harbormaster failed remote builds in B26980: Diff 37339!Feb 20 2024, 2:54 AM
michal updated this revision to Diff 37340.Feb 20 2024, 3:00 AM

Install wasm-pack on CI

Harbormaster completed remote builds in B26981: Diff 37340.Feb 20 2024, 3:15 AM
michal updated this revision to Diff 37345.Feb 20 2024, 3:50 AM

Check if CI will fail if wasm file is not updated, but the code is changed

Harbormaster completed remote builds in B26986: Diff 37345.Feb 20 2024, 4:05 AM
michal updated this revision to Diff 37350.Feb 20 2024, 4:32 AM

Debugging

Harbormaster completed remote builds in B26990: Diff 37350.Feb 20 2024, 4:47 AM
michal updated this revision to Diff 37353.Feb 20 2024, 5:20 AM

Debugging

michal updated this revision to Diff 37355.Feb 20 2024, 5:32 AM

Check if my changes broke the pipeline

Harbormaster completed remote builds in B26993: Diff 37353.Feb 20 2024, 5:35 AM
Harbormaster completed remote builds in B26995: Diff 37355.Feb 20 2024, 6:00 AM
michal updated this revision to Diff 37539.Feb 23 2024, 9:09 AM

Update the new emscripten pipeline

michal updated this revision to Diff 37544.Feb 23 2024, 9:23 AM

Rebase

Harbormaster failed remote builds in B27122: Diff 37539!Feb 23 2024, 9:23 AM
Harbormaster failed remote builds in B27127: Diff 37544!Feb 23 2024, 9:46 AM
michal published this revision for review.Feb 23 2024, 10:10 AM
atul added a comment.Feb 23 2024, 11:35 AM

Passing two command flags seems weird, can we do like yarn build-db-wasm && yarn build-db-wasm?

michal updated this revision to Diff 37590.Feb 26 2024, 3:29 AM

Switch to one command. Check that CI fails if wasm wasn't updated.

Harbormaster failed remote builds in B27170: Diff 37590!Feb 26 2024, 4:56 AM
michal updated this revision to Diff 37591.Feb 26 2024, 5:02 AM

Switch to two command, as I had problems with doing both builds in one command.

Harbormaster failed remote builds in B27171: Diff 37591!Feb 26 2024, 5:09 AM
michal updated this revision to Diff 37596.Feb 26 2024, 5:20 AM

Check that CI passes if wasm is current.

michal updated this revision to Diff 37598.Feb 26 2024, 5:32 AM

Try cleaning before building

Harbormaster failed remote builds in B27172: Diff 37596!Feb 26 2024, 5:34 AM
Harbormaster failed remote builds in B27174: Diff 37598!Feb 26 2024, 5:53 AM
michal updated this revision to Diff 37611.Feb 26 2024, 6:58 AM

Remap paths

Harbormaster failed remote builds in B27179: Diff 37611!Feb 26 2024, 7:05 AM
atul requested changes to this revision.Feb 26 2024, 1:58 PM

Looks like the Emscripten workflow is failing? Sending back to your queue, feel free to re-request review if I'm missing something

This revision now requires changes to proceed.Feb 26 2024, 1:58 PM
Harbormaster completed remote builds in B27127: Diff 37544.Feb 26 2024, 2:18 PM
ashoat added inline comments.Feb 26 2024, 6:10 PM
.buildkite/emscripten.yml
5 ↗(On Diff #37611)

We should avoid committing commented-out code

michal added a comment.Feb 27 2024, 5:16 AM

Oh sorry, I forgot this was in review, sorry for the confusion. I'm having issues with generated WASM reproducibility on CI so this diff isn't ready yet and I'm still testing.

michal updated this revision to Diff 37653.Feb 27 2024, 6:06 AM

Rebase

Harbormaster failed remote builds in B27204: Diff 37653!Feb 27 2024, 6:18 AM
ashoat added a comment.Feb 27 2024, 8:53 PM

It's good practice to "Plan Changes" to take diffs off of your reviewers' queues if they're not ready for review

michal planned changes to this revision.Feb 28 2024, 5:44 AM

Sorry, I keep forgetting that for this diff, when testing CI I have to plan changes after every new revision.

michal abandoned this revision.Feb 28 2024, 7:01 AM

Canceling this in favour of just adding another build step to cleaninstall. See discussions in ENG-6873

Revision Contents
Changeset List

PathSize
native/
cpp/
CommonCpp/
DatabaseManagers/
2 lines
shared/
backup_client/
src/
2 lines

Diff 37355

View Options

native/cpp/CommonCpp/DatabaseManagers/SQLiteQueryExecutor.cpp

// @flow // @flow
import { unsupervisedBackgroundActionType } from './lifecycle-state-reducer.js'; import { unsupervisedBackgroundActionType } from './lifecycle-state-reducer.js';
import { updateActivityActionTypes } from '../actions/activity-actions.js'; import { updateActivityActionTypes } from '../actions/activity-actions.js';
import { setClientDBStoreActionType } from '../actions/client-db-store-actions.js';
import { import {
updateLastCommunicatedPlatformDetailsActionType, updateLastCommunicatedPlatformDetailsActionType,
setDeviceTokenActionTypes, setDeviceTokenActionTypes,
} from '../actions/device-actions.js'; } from '../actions/device-actions.js';
import { import {
addKeyserverActionType, addKeyserverActionType,
removeKeyserverActionType, removeKeyserverActionType,
} from '../actions/keyserver-actions.js'; } from '../actions/keyserver-actions.js';
Show All 10 Lines
import { setNewSessionActionType } from '../keyserver-conn/keyserver-conn-types.js'; import { setNewSessionActionType } from '../keyserver-conn/keyserver-conn-types.js';
import { import {
keyserverStoreOpsHandlers, keyserverStoreOpsHandlers,
type ReplaceKeyserverOperation, type ReplaceKeyserverOperation,
type RemoveKeyserversOperation, type RemoveKeyserversOperation,
type KeyserverStoreOperation, type KeyserverStoreOperation,
} from '../ops/keyserver-store-ops.js'; } from '../ops/keyserver-store-ops.js';
import { queueActivityUpdatesActionType } from '../types/activity-types.js'; import { queueActivityUpdatesActionType } from '../types/activity-types.js';
import type { KeyserverStore } from '../types/keyserver-types.js'; import type {
KeyserverInfos,
KeyserverStore,
} from '../types/keyserver-types.js';
import type { BaseAction } from '../types/redux-types.js'; import type { BaseAction } from '../types/redux-types.js';
import { import {
fullStateSyncActionType, fullStateSyncActionType,
incrementalStateSyncActionType, incrementalStateSyncActionType,
updateConnectionStatusActionType, updateConnectionStatusActionType,
setLateResponseActionType, setLateResponseActionType,
updateKeyserverReachabilityActionType, updateKeyserverReachabilityActionType,
setConnectionIssueActionType, setConnectionIssueActionType,
} from '../types/socket-types.js'; } from '../types/socket-types.js';
import { updateTypes } from '../types/update-types-enum.js'; import { updateTypes } from '../types/update-types-enum.js';
import { processUpdatesActionType } from '../types/update-types.js'; import { processUpdatesActionType } from '../types/update-types.js';
import { getConfig } from '../utils/config.js'; import { getConfig } from '../utils/config.js';
import { getMessageForException } from '../utils/errors.js';
import { assertObjectsAreEqual } from '../utils/objects.js';
import { setURLPrefix } from '../utils/url-utils.js'; import { setURLPrefix } from '../utils/url-utils.js';
import { ashoatKeyserverID } from '../utils/validation-utils.js'; import { ashoatKeyserverID } from '../utils/validation-utils.js';
function assertKeyserverStoresAreEqual(
processedKeyserverStore: KeyserverInfos,
expectedKeyserverStore: KeyserverInfos,
location: string,
onStateDifference?: (message: string) => mixed,
) {
try {
assertObjectsAreEqual(
processedKeyserverStore,
expectedKeyserverStore,
`KeyserverInfos - ${location}`,
);
} catch (e) {
console.log(
'Error processing KeyserverStore ops',
processedKeyserverStore,
expectedKeyserverStore,
);
const message = `Error processing KeyserverStore ops ${
getMessageForException(e) ?? '{no exception message}'
}`;
onStateDifference?.(message);
}
}
const { processStoreOperations: processStoreOps } = keyserverStoreOpsHandlers; const { processStoreOperations: processStoreOps } = keyserverStoreOpsHandlers;
export default function reduceKeyserverStore( export default function reduceKeyserverStore(
state: KeyserverStore, state: KeyserverStore,
action: BaseAction, action: BaseAction,
onStateDifference?: (message: string) => mixed,
): { ): {
keyserverStore: KeyserverStore, keyserverStore: KeyserverStore,
keyserverStoreOperations: $ReadOnlyArray<KeyserverStoreOperation>, keyserverStoreOperations: $ReadOnlyArray<KeyserverStoreOperation>,
} { } {
if (action.type === addKeyserverActionType) { if (action.type === addKeyserverActionType) {
const replaceOperation: ReplaceKeyserverOperation = { const replaceOperation: ReplaceKeyserverOperation = {
type: 'replace_keyserver', type: 'replace_keyserver',
payload: { payload: {
▲ Show 20 Lines • Show All 548 Lines • ▼ Show 20 Lines ) {
}; };
return { return {
keyserverStore: { keyserverStore: {
...state, ...state,
keyserverInfos: processStoreOps(state.keyserverInfos, [operation]), keyserverInfos: processStoreOps(state.keyserverInfos, [operation]),
}, },
keyserverStoreOperations: [operation], keyserverStoreOperations: [operation],
}; };
} else if (action.type === setClientDBStoreActionType) {
// Once the functionality is confirmed to work correctly,
// we will proceed with returning keyserverInfos from the payload.
assertKeyserverStoresAreEqual(
action.payload.keyserverInfos ?? {},
state.keyserverInfos,
action.type,
onStateDifference,
);
return {
keyserverStore: state,
keyserverStoreOperations: [],
};
} }
return { return {
keyserverStore: state, keyserverStore: state,
keyserverStoreOperations: [], keyserverStoreOperations: [],
}; };
} }
View Options

shared/backup_client/src/lib.rs

// @flow // @flow
import { unsupervisedBackgroundActionType } from './lifecycle-state-reducer.js'; import { unsupervisedBackgroundActionType } from './lifecycle-state-reducer.js';
import { updateActivityActionTypes } from '../actions/activity-actions.js'; import { updateActivityActionTypes } from '../actions/activity-actions.js';
import { setClientDBStoreActionType } from '../actions/client-db-store-actions.js';
import { import {
updateLastCommunicatedPlatformDetailsActionType, updateLastCommunicatedPlatformDetailsActionType,
setDeviceTokenActionTypes, setDeviceTokenActionTypes,
} from '../actions/device-actions.js'; } from '../actions/device-actions.js';
import { import {
addKeyserverActionType, addKeyserverActionType,
removeKeyserverActionType, removeKeyserverActionType,
} from '../actions/keyserver-actions.js'; } from '../actions/keyserver-actions.js';
Show All 10 Lines
import { setNewSessionActionType } from '../keyserver-conn/keyserver-conn-types.js'; import { setNewSessionActionType } from '../keyserver-conn/keyserver-conn-types.js';
import { import {
keyserverStoreOpsHandlers, keyserverStoreOpsHandlers,
type ReplaceKeyserverOperation, type ReplaceKeyserverOperation,
type RemoveKeyserversOperation, type RemoveKeyserversOperation,
type KeyserverStoreOperation, type KeyserverStoreOperation,
} from '../ops/keyserver-store-ops.js'; } from '../ops/keyserver-store-ops.js';
import { queueActivityUpdatesActionType } from '../types/activity-types.js'; import { queueActivityUpdatesActionType } from '../types/activity-types.js';
import type { KeyserverStore } from '../types/keyserver-types.js'; import type {
KeyserverInfos,
KeyserverStore,
} from '../types/keyserver-types.js';
import type { BaseAction } from '../types/redux-types.js'; import type { BaseAction } from '../types/redux-types.js';
import { import {
fullStateSyncActionType, fullStateSyncActionType,
incrementalStateSyncActionType, incrementalStateSyncActionType,
updateConnectionStatusActionType, updateConnectionStatusActionType,
setLateResponseActionType, setLateResponseActionType,
updateKeyserverReachabilityActionType, updateKeyserverReachabilityActionType,
setConnectionIssueActionType, setConnectionIssueActionType,
} from '../types/socket-types.js'; } from '../types/socket-types.js';
import { updateTypes } from '../types/update-types-enum.js'; import { updateTypes } from '../types/update-types-enum.js';
import { processUpdatesActionType } from '../types/update-types.js'; import { processUpdatesActionType } from '../types/update-types.js';
import { getConfig } from '../utils/config.js'; import { getConfig } from '../utils/config.js';
import { getMessageForException } from '../utils/errors.js';
import { assertObjectsAreEqual } from '../utils/objects.js';
import { setURLPrefix } from '../utils/url-utils.js'; import { setURLPrefix } from '../utils/url-utils.js';
import { ashoatKeyserverID } from '../utils/validation-utils.js'; import { ashoatKeyserverID } from '../utils/validation-utils.js';
function assertKeyserverStoresAreEqual(
processedKeyserverStore: KeyserverInfos,
expectedKeyserverStore: KeyserverInfos,
location: string,
onStateDifference?: (message: string) => mixed,
) {
try {
assertObjectsAreEqual(
processedKeyserverStore,
expectedKeyserverStore,
`KeyserverInfos - ${location}`,
);
} catch (e) {
console.log(
'Error processing KeyserverStore ops',
processedKeyserverStore,
expectedKeyserverStore,
);
const message = `Error processing KeyserverStore ops ${
getMessageForException(e) ?? '{no exception message}'
}`;
onStateDifference?.(message);
}
}
const { processStoreOperations: processStoreOps } = keyserverStoreOpsHandlers; const { processStoreOperations: processStoreOps } = keyserverStoreOpsHandlers;
export default function reduceKeyserverStore( export default function reduceKeyserverStore(
state: KeyserverStore, state: KeyserverStore,
action: BaseAction, action: BaseAction,
onStateDifference?: (message: string) => mixed,
): { ): {
keyserverStore: KeyserverStore, keyserverStore: KeyserverStore,
keyserverStoreOperations: $ReadOnlyArray<KeyserverStoreOperation>, keyserverStoreOperations: $ReadOnlyArray<KeyserverStoreOperation>,
} { } {
if (action.type === addKeyserverActionType) { if (action.type === addKeyserverActionType) {
const replaceOperation: ReplaceKeyserverOperation = { const replaceOperation: ReplaceKeyserverOperation = {
type: 'replace_keyserver', type: 'replace_keyserver',
payload: { payload: {
▲ Show 20 Lines • Show All 548 Lines • ▼ Show 20 Lines ) {
}; };
return { return {
keyserverStore: { keyserverStore: {
...state, ...state,
keyserverInfos: processStoreOps(state.keyserverInfos, [operation]), keyserverInfos: processStoreOps(state.keyserverInfos, [operation]),
}, },
keyserverStoreOperations: [operation], keyserverStoreOperations: [operation],
}; };
} else if (action.type === setClientDBStoreActionType) {
// Once the functionality is confirmed to work correctly,
// we will proceed with returning keyserverInfos from the payload.
assertKeyserverStoresAreEqual(
action.payload.keyserverInfos ?? {},
state.keyserverInfos,
action.type,
onStateDifference,
);
return {
keyserverStore: state,
keyserverStoreOperations: [],
};
} }
return { return {
keyserverStore: state, keyserverStore: state,
keyserverStoreOperations: [], keyserverStoreOperations: [],
}; };
} }