Paths

Table of Contentst

-
lib/
-
facts/
1/3
reports-service.js
-
types/
1/3
report-types.js

[lib] Add reports service facts
ClosedPublic
Actions

Authored by bartek on Aug 30 2023, 11:41 PM.

Details

Reviewers

kamil
michal
tomek
inka

Commits

rCOMMcc25100ea231: [lib] Add reports service facts

Summary

Similiar to other services, add facts file with endpoint info for reports service.
We expose only single endpoint meaningful to clients. Other endpoints are staff only so I haven't added them.

Test Plan

flow

Diff Detail

Repository

rCOMM Comm

Lint

No Lint Coverage

Unit

No Test Coverage

Event Timeline

bartek created this revision.Aug 30 2023, 11:41 PM

bartek held this revision as a draft.

Herald added a subscriber: ashoat. · View Herald TranscriptAug 30 2023, 11:41 PM

bartek added a child revision: D9042: [lib] Add function to send reports.Aug 30 2023, 11:44 PM

Make types readonly

Harbormaster completed remote builds in B22289: Diff 30615.Aug 31 2023, 12:00 AM

bartek published this revision for review.Aug 31 2023, 12:07 AM

bartek added inline comments.

lib/facts/reports-service.js
5–8 ↗	(On Diff #30621)	I wish flow had TypeScript's `as const` - playground link

Harbormaster completed remote builds in B22296: Diff 30621.Aug 31 2023, 12:14 AM

Should we also update the docs with the new facts file?

This revision is now accepted and ready to land.Aug 31 2023, 5:07 AM

In D9041#266461, @tomek wrote:

Should we also update the docs with the new facts file?

Which docs? Other services' facts don't have any docs IIRC 🤔

In D9041#266850, @bartek wrote:

In D9041#266461, @tomek wrote:

Should we also update the docs with the new facts file?

Which docs? Other services' facts don't have any docs IIRC 🤔

If the app works correctly after fresh checkout without these facts, then we're fine. But if this file is required for the app to work, then we should mention it somewhere (I remember that we're mentioning web and landing facts somewhere).

Rebase

Harbormaster completed remote builds in B22469: Diff 30890.Sep 9 2023, 10:59 AM

Closed by commit rCOMMcc25100ea231: [lib] Add reports service facts (authored by bartek). · Explain WhySep 9 2023, 11:53 AM

This revision was automatically updated to reflect the committed changes.

bartek added a commit: rCOMMcc25100ea231: [lib] Add reports service facts.

Revision Contents
Changeset List

Path

Size

lib/

facts/

reports-service.js

15 lines

types/

report-types.js

9 lines

Diff 30615

View Options

lib/facts/reports-service.js

// @flow

import Tokenizer from 'tokenize-text';

import RadixTree from './radix-tree.js';

type Token = {

index: number,

match: {

[i: number]: string,

index: number,

input: string,

...

offset: number,

value: string,

...

};

type TokenizeFunc = (str: string) => Token[];

const defaultTokenize: TokenizeFunc = new Tokenizer().words();

class SearchIndex {

tokenize: (str: string) => Token[];

fullTextIndex: { [token: string]: Set<string> };

radixTree: RadixTree<string> = new RadixTree<string>();

partialTextIndex: { [token: string]: Set<string> };

constructor(inputTokenize?: TokenizeFunc) {

this.tokenize = inputTokenize ?? defaultTokenize;

this.fullTextIndex = {};

this.partialTextIndex = {};

}

addAllPrefixes(id: string, value: string): void {

if (this.fullTextIndex[value] === undefined) {

this.radixTree.add(value, id);

this.fullTextIndex[value] = new Set();

}

this.fullTextIndex[value].add(id);

let partialString = '';

for (let i = 0; i < value.length; i++) {

const char = value[i];

partialString += char;

// TODO probably should do some stopwords here

if (this.partialTextIndex[partialString] === undefined) {

this.partialTextIndex[partialString] = new Set();

}

this.partialTextIndex[partialString].add(id);

}

addEntry(id: string, rawText: string) {

const keywords = this.tokenize(rawText);

for (const keyword of keywords) {

const value = keyword.value.toLowerCase();

this.addAllPrefixes(id, value);

}

getSearchResults(query: string): string[] {

const keywords = this.tokenize(query);

if (keywords.length === 0) {

return [];

}

const lastKeyword = keywords[keywords.length - 1];

const lastKeywordValue = lastKeyword.value.toLowerCase();

const lastMatchSet = lastKeyword.match.input.match(/\S$/)

? this.partialTextIndex[lastKeywordValue]

? this.radixTree.getAllMatchingPrefix(lastKeywordValue)

: this.fullTextIndex[lastKeywordValue];

: this.radixTree.getAllMatchingExactly(lastKeywordValue);

if (!lastMatchSet) {

if (lastMatchSet.length === 0) {

return [];

}

const fullKeywords = keywords.slice(0, -1).map(k => k.value.toLowerCase());

let possibleMatches: string[] = Array.from(lastMatchSet);

let possibleMatches = lastMatchSet;

for (const keyword of fullKeywords) {

const fullMatches = this.fullTextIndex[keyword];

const fullMatches = this.radixTree.getAllMatchingExactly(keyword);

if (!fullMatches) {

if (fullMatches.length === 0) {

return [];

}

possibleMatches = possibleMatches.filter(id => fullMatches.has(id));

const fullMatchesSet = new Set(fullMatches);

possibleMatches = possibleMatches.filter(id => fullMatchesSet.has(id));

tomekUnsubmitted

Not Done

const fullMatchesSet = new Set(fullMatches);

- possibleMatches = possibleMatches.filter(id => fullMatchesSet.has(id));

+ possibleMatches = possibleMatches.filter(fullMatchesSet.has);

if (possibleMatches.length === 0) {

Can this be simplified?

tomek: Can this be simplified?

ashoatAuthorUnsubmitted

Done

I tested this approach in the Chrome debug console and found that it didn't work:

Screenshot 2023-10-30 at 2.47.26 PM.png (592×1 px, 94 KB)

ashoat: I tested this approach in the Chrome debug console and found that it didn't work: {F842952}

tomekUnsubmitted

Not Done

Ah, that makes sense. In order to make it work, we would need to bind the method which will be more complicated than the current solution.

tomek: Ah, that makes sense. In order to make it work, we would need to bind the method which will be…

if (possibleMatches.length === 0) {

return [];

}

return possibleMatches;

}

export default SearchIndex;

View Options

lib/types/report-types.js

// @flow

import Tokenizer from 'tokenize-text';

import RadixTree from './radix-tree.js';

type Token = {

index: number,

match: {

[i: number]: string,

index: number,

input: string,

...

offset: number,

value: string,

...

};

type TokenizeFunc = (str: string) => Token[];

const defaultTokenize: TokenizeFunc = new Tokenizer().words();

class SearchIndex {

tokenize: (str: string) => Token[];

fullTextIndex: { [token: string]: Set<string> };

radixTree: RadixTree<string> = new RadixTree<string>();

partialTextIndex: { [token: string]: Set<string> };

constructor(inputTokenize?: TokenizeFunc) {

this.tokenize = inputTokenize ?? defaultTokenize;

this.fullTextIndex = {};

this.partialTextIndex = {};

}

addAllPrefixes(id: string, value: string): void {

if (this.fullTextIndex[value] === undefined) {

this.radixTree.add(value, id);

this.fullTextIndex[value] = new Set();

}

this.fullTextIndex[value].add(id);

let partialString = '';

for (let i = 0; i < value.length; i++) {

const char = value[i];

partialString += char;

// TODO probably should do some stopwords here

if (this.partialTextIndex[partialString] === undefined) {

this.partialTextIndex[partialString] = new Set();

}

this.partialTextIndex[partialString].add(id);

}

addEntry(id: string, rawText: string) {

const keywords = this.tokenize(rawText);

for (const keyword of keywords) {

const value = keyword.value.toLowerCase();

this.addAllPrefixes(id, value);

}

getSearchResults(query: string): string[] {

const keywords = this.tokenize(query);

if (keywords.length === 0) {

return [];

}

const lastKeyword = keywords[keywords.length - 1];

const lastKeywordValue = lastKeyword.value.toLowerCase();

const lastMatchSet = lastKeyword.match.input.match(/\S$/)

? this.partialTextIndex[lastKeywordValue]

? this.radixTree.getAllMatchingPrefix(lastKeywordValue)

: this.fullTextIndex[lastKeywordValue];

: this.radixTree.getAllMatchingExactly(lastKeywordValue);

if (!lastMatchSet) {

if (lastMatchSet.length === 0) {

return [];

}

const fullKeywords = keywords.slice(0, -1).map(k => k.value.toLowerCase());

let possibleMatches: string[] = Array.from(lastMatchSet);

let possibleMatches = lastMatchSet;

for (const keyword of fullKeywords) {

const fullMatches = this.fullTextIndex[keyword];

const fullMatches = this.radixTree.getAllMatchingExactly(keyword);

if (!fullMatches) {

if (fullMatches.length === 0) {

return [];

}

possibleMatches = possibleMatches.filter(id => fullMatches.has(id));

const fullMatchesSet = new Set(fullMatches);

possibleMatches = possibleMatches.filter(id => fullMatchesSet.has(id));

tomekUnsubmitted

Not Done

const fullMatchesSet = new Set(fullMatches);

- possibleMatches = possibleMatches.filter(id => fullMatchesSet.has(id));

+ possibleMatches = possibleMatches.filter(fullMatchesSet.has);

if (possibleMatches.length === 0) {

Can this be simplified?

tomek: Can this be simplified?

ashoatAuthorUnsubmitted

Done

I tested this approach in the Chrome debug console and found that it didn't work:

ashoat: I tested this approach in the Chrome debug console and found that it didn't work: {F842952}

tomekUnsubmitted

Not Done

Ah, that makes sense. In order to make it work, we would need to bind the method which will be more complicated than the current solution.

tomek: Ah, that makes sense. In order to make it work, we would need to bind the method which will be…

if (possibleMatches.length === 0) {

return [];

}

return possibleMatches;

}

export default SearchIndex;

[lib] Add reports service factsClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 30615

lib/facts/reports-service.js

lib/types/report-types.js

[lib] Add reports service facts
ClosedPublic
Actions

Revision Contents
Changeset List