spiegel-keyman/common/predictive-text
Eddie Antonio Santos 7dd22b59bd
Implement looking up a wordform in a trie.
This trie implementation allows for word forms to be converted to keys.
Since keys do not have to have the exact same spelling, capitalization,
or even Unicode normalization form as the desried word form, this
enables a somewhat search for word forms based on prefixes.

The implementation was at one point Conrad Irwin's trie-ing,
<9f1d2e0858?
but has been heavily modfied since.
2019-04-26 16:28:14 -06:00
..
docs Merge pull request #1655 from keymanapp/web-lmlayer-document-fsm 2019-03-06 23:07:50 -07:00
testing Demonstrate Blob code proof-of-concept. 2018-11-20 11:12:05 -07:00
unit_tests Start tests for new Trie model. 2019-04-26 15:18:49 -06:00
worker Implement looking up a wordform in a trie. 2019-04-26 16:28:14 -06:00
.gitignore More general pattern for .gitignore. 2018-11-20 08:48:45 -07:00
build.sh Forgot the $ symbol in the if-condition. 2019-03-06 12:13:25 +07:00
embedded_worker.d.ts Merge branch 'lmlayer-init' into lmlayer-build-reorg 2018-11-29 23:35:37 +07:00
history.md Created an initial build script for use in common/predictive-text 2018-11-01 08:04:06 +07:00
index.d.ts Merge branch 'web-lmlayer-model-registration' into lmlayer-wordbreak-spec 2019-03-07 09:04:00 +07:00
index.ts Things are more work-y... 2019-03-28 10:26:34 -06:00
message.d.ts For Suggestion application: OutputTarget.restoreTo 2019-03-28 10:10:10 +07:00
package-lock.json Update dependencies based on npm audit. 2019-03-26 11:07:29 -06:00
package.json Allow LMLayer to be imported as a Node module. 2019-03-26 13:10:35 -06:00
promise-store.ts Actually includes the files this time... 2019-02-19 08:52:31 +07:00
README.md Slightly improve the README text. 2018-11-28 14:54:03 -07:00
tsconfig.json Reorganized the build structure into something simpler to -clean. 2018-11-29 09:11:26 +07:00

Language Modelling Layer (LMLayer)

Provide predictions and corrections while you type!

See Worker Communication Protocol for a semi-formal specification on how the Worker and the main thread communicate.

System dependencies

You will need Bash and Node.js >= 6.0.

Build

Run build.sh. This will also automatically install dependencies with npm.

./build.sh

Two-stage compilation process

Since the primary LMLayer code runs within a Web Worker, build.sh compiles the LMLayer in two stages:

  1. Compile the inner worker code

    1. Compile the TypeScript sources for only the Worker code.
    2. Wrap the Worker code as embedded_worker.js
  2. Compile the top-level code

    1. Include embedded_worker.js verbatim using a TypeScript directive.
    2. Compile the top-level TypeScript code.
    3. Unwrap the Worker code at runtime.

Test

This will run both headless unit tests, and in-browser unit tests and integration tests:

./build.sh -test

Test-Driven Development

I like to use entr to automatically build and re-run the unit tests anytime I change a source code file. Here's the command I run in separate window:

git ls-files | entr -c ./build.sh -tdd

Importantly, ./build.sh -tdd skips running the in-browser tests, and skips downloading/updating npm dependencies.