Joshua A. Horton
fed4618d02
feat(common/models): tslib use, treeshaking in lm-worker
2023-06-08 14:39:55 +07:00
Joshua A. Horton
b773cc852d
change(common/models): wordbreakers -> ES modules
2023-01-20 13:26:09 +07:00
Joshua A. Horton
26fb6da3ab
feat(common/models): adds tokenization tests for wordbreaks near the caret
2022-09-15 13:23:21 +07:00
Joshua A. Horton
94face23d0
chore(common/models): minor cleanup
2022-09-13 11:06:54 +07:00
Joshua A. Horton
261e252026
docs(common/models): adds minor doc re 'sot', 'eot'
2022-09-13 10:38:34 +07:00
Joshua A. Horton
8e076fe3fd
feat(common/models): actual wordbreaker customization
2022-09-13 10:18:24 +07:00
Joshua A. Horton
51caf533a8
feat(common/models): interface for custom breaker rules
2022-09-12 15:14:28 +07:00
Joshua A. Horton
991a5bc271
feat(common/models): wordbreaker context .match method
2022-09-12 15:14:27 +07:00
Joshua A. Horton
5bad267e11
refactor(common/models): default wordbreaker state tracking
2022-09-12 15:14:26 +07:00
Joshua A. Horton
91448c344f
feat(web): more default-breaker unit tests
2022-09-06 14:52:16 +07:00
Joshua A. Horton
a6e65333c5
fix(common/models): wordbreaker test require paths
2022-09-06 08:54:56 +07:00
Eddie Antonio Santos
d2249ad2d7
change(common/models/wordbreakers): remove extendedPictographic regexp (unused)
2020-07-17 15:35:37 -06:00
Eddie Antonio Santos
37e9b034e4
fix(common/models/wordbreakers): use ES5 compatible regexp
2020-07-17 15:23:18 -06:00
Eddie Antonio Santos
dbb2908aa0
change(common/models/wordbreakers): update word boundary props to 13.0
...
This updates the Word_Boundary table to Unicode 13.0.0.
The `searchForProperty()` function has also been changed so that the END
property of a range is no longer needed; all code points are explicitly
covered in the table. Consequently, ranges containing unmapped code points
(WordBreakProperty.Other) are explicit.
2020-07-17 15:03:44 -06:00
Eddie Antonio Santos
a4dae62817
refactor(common/lmlayer): move default word breaker to models-wordbreakers package
2020-05-19 15:07:06 -06:00
Eddie Antonio Santos
6fc39df5d0
Revert "refactor(common/lmlayer): move default word-breaker to @keymanapp/lexical-model-word-breakers"
...
This reverts commit 1f4de1ba93
2020-05-19 14:32:19 -06:00
Eddie Antonio Santos
73b5d520e2
change(common/lmlayer): move to common/models/wordbreakers
2020-05-19 14:08:56 -06:00
