Commit graph

22 commits

Author SHA1 Message Date
Joshua A. Horton
26fb6da3ab feat(common/models): adds tokenization tests for wordbreaks near the caret 2022-09-15 13:23:21 +07:00
Joshua A. Horton
94face23d0 chore(common/models): minor cleanup 2022-09-13 11:06:54 +07:00
Joshua A. Horton
261e252026 docs(common/models): adds minor doc re 'sot', 'eot' 2022-09-13 10:38:34 +07:00
Joshua A. Horton
8e076fe3fd feat(common/models): actual wordbreaker customization 2022-09-13 10:18:24 +07:00
Joshua A. Horton
51caf533a8 feat(common/models): interface for custom breaker rules 2022-09-12 15:14:28 +07:00
Joshua A. Horton
991a5bc271 feat(common/models): wordbreaker context .match method 2022-09-12 15:14:27 +07:00
Joshua A. Horton
5bad267e11 refactor(common/models): default wordbreaker state tracking 2022-09-12 15:14:26 +07:00
Joshua A. Horton
91448c344f feat(web): more default-breaker unit tests 2022-09-06 14:52:16 +07:00
Joshua A. Horton
a6e65333c5 fix(common/models): wordbreaker test require paths 2022-09-06 08:54:56 +07:00
Marc Durdin
e0b308b983 chore(web): cleanup for ts projects 2022-04-13 15:19:55 +10:00
Marc Durdin
4d94786852 chore(web): replace lerna with npm workspaces and ts projects
Fixes #6320.

Part 1 of moving from lerna to a simpler, maintained monorepo solution,
using TypeScript Projects and NPM Workspaces.

There is more work to be done here. At this point, KeymanWeb builds and
runs without errors, but the built file is substantially different,
mostly in include order.

Using TypeScript Projects, we move away from the need to run build
scripts in various locations for almost all the Typescript modules.

TODO: Embedded versions and tests have not been verified.

TODO: developer/server is not yet verified.

TODO: developer/js (needs a rename!) is not yet verified.

TODO: Currently, the predictive-text folder needs refactoring to move
the construction of the worker wrapper out of the Predictive Text build
and into the final assembly of keymanweb.js (as it should be valid to
run it as a separate .js anyway).

TODO: Most of the `<reference>` paths need to be re-verified. Ideally
there should be no references outside the current module for any given
.ts.

TODO: The embedded vs browser vs node (headless) builds should be tidied
up for consistency so that it's obvious what depends on what. This is
currently messiest in the predictive-text folder, where the output names
diverge from the filenames and the various files are mixed in the same
folder (as evidenced by the exclusions listed in each tsconfig.json).

TODO: `npm install` should be removed from most build scripts and
instead `npm ci` (#6196) should be run only once from the top-level
folder for any given build. I've had eliminated side-effects from the
`install` action for npm, which makes it easier to reason about state.

TODO: verify_npm_setup and related functions can probably be eliminated.

TODO: most of the build scripts should be largely eliminated for web.

TODO: several ts projects use inconsistent output folders.

TODO: it may be possible to generate a .d.ts for models/types so that
we can use a consistent reference for those as well.

TODO: build.sh, tsconfig.json should always be in the module's top-level
folder, not in a subfolder such as src (e.g. see input-processor/src,
keyboard-processor/src, web/source).

TODO: resources/web-environment should be in common/web.

TODO: other js node_modules imports should be wrapped like es6-shim.

TODO: fix up the publish code for npm modules

TODO: eliminate version numbers from package.json if possible?

Whew, that's most of the stuff I noticed!
2022-04-12 21:02:33 +10:00
Eddie Antonio Santos
d2249ad2d7
change(common/models/wordbreakers): remove extendedPictographic regexp (unused) 2020-07-17 15:35:37 -06:00
Eddie Antonio Santos
37e9b034e4
fix(common/models/wordbreakers): use ES5 compatible regexp 2020-07-17 15:23:18 -06:00
Eddie Antonio Santos
dbb2908aa0
change(common/models/wordbreakers): update word boundary props to 13.0
This updates the Word_Boundary table to Unicode 13.0.0.

The `searchForProperty()` function has also been changed so that the END
property of a range is no longer needed; all code points are explicitly
covered in the table. Consequently, ranges containing unmapped code points
(WordBreakProperty.Other) are explicit.
2020-07-17 15:03:44 -06:00
Eddie Antonio Santos
a4dae62817
refactor(common/lmlayer): move default word breaker to models-wordbreakers package 2020-05-19 15:07:06 -06:00
Eddie Antonio Santos
0baf04ae06
refactor(common/lmlayer): move placeholder wordbreaker to models-wordbreakers 2020-05-19 15:04:29 -06:00
Eddie Antonio Santos
e98ae7d7a1
fix(common/lmlayer): add @types/node to @keymanapp/models-wordbreaker to prevent failure 2020-05-19 14:57:55 -06:00
Eddie Antonio Santos
7b1f86b399
fix(common/lmlayer): export wordBreakers namespace 2020-05-19 14:46:33 -06:00
Eddie Antonio Santos
863b83883b
chore(common/lmlayer): get ASCII word breaker compiling 2020-05-19 14:36:03 -06:00
Eddie Antonio Santos
374f0dfd8e
Revert "refactor(common/lmlayer): move placeholder word breaker to @keymanapp/lexical-model-word-breakers"
This reverts commit 73b16ae33e.
2020-05-19 14:34:11 -06:00
Eddie Antonio Santos
6fc39df5d0
Revert "refactor(common/lmlayer): move default word-breaker to @keymanapp/lexical-model-word-breakers"
This reverts commit 1f4de1ba93.
2020-05-19 14:32:19 -06:00
Eddie Antonio Santos
73b5d520e2
change(common/lmlayer): move to common/models/wordbreakers 2020-05-19 14:08:56 -06:00