Commit graph

555 commits

Author SHA1 Message Date
Eddie Antonio Santos
6395ff5ce0
Merge branch 'master' into lmlayer-search-key-param 2019-05-22 23:12:33 -06:00
Eddie Antonio Santos
5237a1817e
Use const enum for word break property. 2019-05-22 16:25:06 -06:00
Eddie Antonio Santos
969ceafbd5
Revert: change default word breaker to placeholder again. 2019-05-22 11:50:48 -06:00
Eddie Antonio Santos
f205f868c4
Use array instead of object in big table. 2019-05-22 11:48:16 -06:00
Eddie Antonio Santos
dc2282dc60
Use the default word breaker consistently. 2019-05-22 11:20:52 -06:00
Eddie Antonio Santos
42ff692e00
Remove it.only... 2019-05-22 08:42:45 -06:00
Eddie Antonio Santos
c4947cfce6
Add searchTermToKey() function to TrieModelOptions. 2019-05-21 15:58:56 -06:00
Eddie Antonio Santos
a2d5c09575
Convolute test to see if the key function actually works. 2019-05-21 15:58:29 -06:00
Eddie Antonio Santos
4865c51951
Fix off-by-one error.
Both `left` and `right` must be valid indices into the
WORD_BOUNDARY_PROPERTY array.
2019-05-21 12:34:43 -06:00
Eddie Antonio Santos
d46f5e1de1
Remove generator function* and work exclusively with arrays instead. 2019-05-21 10:26:33 -06:00
Eddie Antonio Santos
dafa1f6ff3
Convert findBoundaries() from generator to array returning function. 2019-05-21 09:48:42 -06:00
Eddie Antonio Santos
ca18b3c990
Remove extendedPictographic and support for WB3c. 2019-05-21 09:26:05 -06:00
Eddie Antonio Santos
4a2d965845
Place data into its own namespace. 2019-05-21 09:18:15 -06:00
Eddie Antonio Santos
a5974013ac
Merge branch 'master' into lmlayer-unicode-default-word-boundary 2019-05-21 09:12:36 -06:00
Eddie Antonio Santos
b5443e5741
Export a function called 'default'. 2019-05-20 04:06:24 -06:00
Eddie Antonio Santos
d2d017ce53
Implement fixes from upstream. 2019-05-18 17:53:34 -06:00
Eddie Antonio Santos
4817ef6d6d
Make unit test slightly less weird. 2019-05-18 17:06:06 -06:00
Eddie Antonio Santos
6d8ce4f2bc
Treat flakiness in 'empty buffer' test. 2019-05-17 21:32:02 -06:00
Eddie Antonio Santos
bd7a2a52f4
Remove debugger and console.assert() lines... 2019-05-17 17:26:47 -06:00
Eddie Antonio Santos
8eede859c0
Split the data from the implementation. 2019-05-17 17:08:12 -06:00
Eddie Antonio Santos
ce179c772c
First version of the default word breaker. 2019-05-17 16:47:59 -06:00
Eddie Antonio Santos
fc8cd86617
Test for default Unicode word breaker. 2019-05-17 16:39:08 -06:00
jahorton
efb0350bac Adds support for rotating suggestions in KMW. 2019-05-14 15:00:48 +07:00
Eddie Antonio Santos
2356608b72
Change name to 'placeholder' to match lexical-models. 2019-05-03 08:05:30 -06:00
Eddie Antonio Santos
64073fb02d
Clean up ASCII word breaker unit test. 2019-05-02 15:11:03 -06:00
Eddie Antonio Santos
bc91ee9ec4
Use word breaking function in WordListModel. 2019-05-02 14:53:50 -06:00
Eddie Antonio Santos
dab2d0c2eb
Refactor: move placeholder word breaker to word_breakers directory/namespace. 2019-05-02 14:47:27 -06:00
Eddie Antonio Santos
89ad857210
Have TrieModel use a word-breaker internally. 2019-05-02 14:42:00 -06:00
Eddie Antonio Santos
a4db640621
Add word breaker as a parameter to TrieModel. 2019-05-02 14:33:28 -06:00
Eddie Antonio Santos
d2dd697706
Implement ASCII word breaker. 2019-05-02 13:50:47 -06:00
Eddie Antonio Santos
ed9e95f9a7
Test the ASCII word breaker. 2019-05-02 13:32:14 -06:00
Eddie Antonio Santos
b2282c0349
Add new interfaces for word breaking. 2019-05-02 13:17:42 -06:00
Eddie Antonio Santos
617d657d69
Remove previous versions of the word breaker. 2019-05-02 12:57:04 -06:00
Eddie Antonio Santos
d34cecf0ae
[LMLayer-CLI] fix argument paring bug. 2019-04-30 09:57:03 -06:00
Eddie Antonio Santos
e3d98dddd3
Merge pull request #1748 from keymanapp/lmlayer-trie
[LMLayer] Trie-backed word list model
2019-04-29 21:32:01 -06:00
Eddie Antonio Santos
75ef303422
Update english-1000 in integration test. 2019-04-29 20:46:53 -06:00
Eddie Antonio Santos
05de75229c
[LMLayer] Always accept wordlist as array of [word, count] pairs. 2019-04-29 20:39:31 -06:00
Eddie Antonio Santos
19b218cf97
Tweak current priority queue implementation. 2019-04-29 08:37:16 -06:00
Eddie Antonio Santos
dd94e4ba26
Remove checks for unsortedness. 2019-04-29 08:30:22 -06:00
Eddie Antonio Santos
258d41943e
Test suggest after a single word. 2019-04-29 08:27:42 -06:00
Eddie Antonio Santos
58f429e344
Implement trie.firstN(). 2019-04-29 08:25:31 -06:00
Eddie Antonio Santos
4382bccd95
Test empty context/transform. 2019-04-29 08:13:08 -06:00
Eddie Antonio Santos
c8913a4b41
Implement test for non-empty context. 2019-04-29 08:09:41 -06:00
Eddie Antonio Santos
7dd22b59bd
Implement looking up a wordform in a trie.
This trie implementation allows for word forms to be converted to keys.
Since keys do not have to have the exact same spelling, capitalization,
or even Unicode normalization form as the desried word form, this
enables a somewhat search for word forms based on prefixes.

The implementation was at one point Conrad Irwin's trie-ing,
<9f1d2e0858?
but has been heavily modfied since.
2019-04-26 16:28:14 -06:00
Eddie Antonio Santos
9415975765
Start tests for new Trie model. 2019-04-26 15:18:49 -06:00
Eddie Antonio Santos
6b84f16e03
Bump version, and document options in the README. 2019-04-18 12:40:38 -06:00
Eddie Antonio Santos
aa9eaaa15a
Change --string to --phrase, and allow for multiple phrases. 2019-04-18 12:34:44 -06:00
Eddie Antonio Santos
e7c87824b8
Factor out slurpLinesFromFile(). 2019-04-18 12:16:31 -06:00
Eddie Antonio Santos
6650cf3cbe
Support batch mode. 2019-04-18 12:05:48 -06:00
Eddie Antonio Santos
dd5d8eb418
Factor isatty() and short how-to to the asyncRepl(). 2019-04-18 11:08:13 -06:00