Joshua Horton
aaec2837ca
Merge pull request #14825 from keymanapp/feat/web/interim-legacy-keyer
...
feat(web): adds legacy keyer that matches existing transform-tokenization behavior 🚂
2025-10-16 21:49:54 +07:00
Joshua Horton
ebc264ac15
Merge branch 'epic/autocorrect' into feat/web/evaluate-precomputed-tokenization
2025-10-16 09:17:22 -05:00
Joshua Horton
66209db351
Merge pull request #14876 from keymanapp/feat/web/compute-removed-token-count
...
feat(web): compute number of tokens deleted in tokenization transition 🚂
2025-10-16 21:09:35 +07:00
Joshua Horton
1a78cc5a25
Merge pull request #14875 from keymanapp/fix/web/empty-context-tail-handling
...
fix(web): do not ignore empty context-tail tokens when computing tokenization transition 🚂
2025-10-16 21:09:26 +07:00
Keyman Server
0753cc548c
Merge branch 'epic/autocorrect' into chore/A19S13-merge-master-into-autocorrect
2025-10-10 05:04:36 +02:00
Joshua Horton
41619fed1f
Merge branch 'feat/web/evaluate-precomputed-tokenization' into feat/web/interim-legacy-keyer
2025-10-08 11:41:39 -05:00
Joshua Horton
8dd5748fec
docs(web): "assumption", not "assertion"
2025-10-08 11:37:36 -05:00
Joshua Horton
b28bb2deb7
Merge pull request #14841 from keymanapp/feat/web/merge-context-tokens
...
feat(web): add method for merging multiple ContextTokens into one 🚂
2025-10-08 22:35:38 +07:00
Joshua Horton
da444878ff
Merge pull request #14840 from keymanapp/feat/web/split-context-tokens
...
feat(web): add class method for splitting ContextToken instances 🚂
2025-10-08 22:35:29 +07:00
Joshua Horton
d70d7a75c9
Merge pull request #14823 from keymanapp/feat/web/tokenization-subset-builder
...
feat(web): build subsets of prior-state + input pairings with compatible tokenization results 🚂
2025-10-08 22:35:20 +07:00
Joshua Horton
e95422565d
Merge pull request #14822 from keymanapp/feat/web/key-common-result-tokenizations
...
feat(web): define keying function that matches paths that result in compatible context tokenizations 🚂
2025-10-08 22:35:09 +07:00
Joshua Horton
05a89a07a4
Merge pull request #14820 from keymanapp/feat/web/track-boundary-token-source
...
feat(web): track source input ranges for the edit-boundary source token 🚂
2025-10-08 22:34:58 +07:00
Joshua Horton
eb3a1d86bb
Merge pull request #14790 from keymanapp/feat/web/boundary-shifting-transform-tokenization
...
feat(web): precompute transform tokenization while handling when existing boundaries shift 🚂
2025-10-08 22:34:48 +07:00
Joshua Horton
4698f22c86
Merge branch 'feat/web/compute-removed-token-count' into feat/web/evaluate-precomputed-tokenization
2025-10-08 08:52:04 -05:00
Joshua Horton
b2f460d0be
Merge branch 'fix/web/empty-context-tail-handling' into feat/web/compute-removed-token-count
2025-10-08 08:51:22 -05:00
Joshua Horton
96fd4b6a6f
fix(web): update unit test use of method with name changed to mapWhitespacedTokenization
2025-10-08 08:50:57 -05:00
Joshua Horton
30f0eee46b
Merge branch 'feat/web/merge-context-tokens' into fix/web/empty-context-tail-handling
2025-10-08 08:50:08 -05:00
Joshua Horton
886642c67d
Merge branch 'feat/web/split-context-tokens' into feat/web/merge-context-tokens
2025-10-08 08:49:25 -05:00
Joshua Horton
2b290955a4
Merge branch 'feat/web/tokenization-subset-builder' into feat/web/split-context-tokens
2025-10-08 08:45:19 -05:00
Joshua Horton
97789c008b
fix(web): update unit test use of method with name changed to mapWhitespacedTokenization
2025-10-08 08:44:51 -05:00
Joshua Horton
8a69cd4689
Merge branch 'feat/web/key-common-result-tokenizations' into feat/web/tokenization-subset-builder
2025-10-08 08:43:28 -05:00
Joshua Horton
8ebfab10fc
docs(web): remove duplicate words in file header comment
2025-10-08 08:43:02 -05:00
Joshua Horton
9c8747d9c8
Merge branch 'feat/web/track-boundary-token-source' into feat/web/key-common-result-tokenizations
2025-10-08 08:42:01 -05:00
Joshua Horton
164b6306ef
Merge branch 'feat/web/boundary-shifting-transform-tokenization' into feat/web/track-boundary-token-source
2025-10-08 08:40:55 -05:00
Joshua Horton
01a0eeec19
change(web): renames precomputeTokenizationAfterInput to mapWhitespacedTokenization
2025-10-08 08:40:15 -05:00
Joshua Horton
98a40a3f64
change(web): clarify role of sliceIndex in token-merge, token-split tests
2025-10-06 16:47:51 -05:00
Joshua Horton
199c962f6f
Merge pull request #14807 from keymanapp/feat/web/tokenized-transform-assembly
...
change(web): provide alternate helper for assembly of tokenized transforms 🚂
2025-10-06 21:52:03 +07:00
Joshua Horton
df2a868605
Merge pull request #14806 from keymanapp/feat/web/compute-merge-split-effects
...
feat(web): add helper for aligning tokens after merges, splits 🚂
2025-10-06 21:51:52 +07:00
Joshua Horton
e8f28fdf36
Merge pull request #14805 from keymanapp/feat/web/trace-insert-edits
...
feat(web): add helper for tokenizing Transform.insert via tokenization of applied context 🚂
2025-10-06 21:51:38 +07:00
Joshua Horton
a9a761443f
Merge pull request #14754 from keymanapp/feat/web/backward-slide-tokenization
...
feat(web): add helper method for updating context tokenization before applying inputs 🚂
2025-10-06 21:51:20 +07:00
Joshua Horton
6800734d39
Merge pull request #14800 from keymanapp/feat/web/build-token-aligned-edge-windows
...
feat(web): build token-aligned edge windows 🚂
2025-10-06 21:51:10 +07:00
Joshua Horton
e681dc46ed
Merge pull request #14751 from keymanapp/feat/web/context-slide-deltas
...
feat(web): add method to determine the change to context resulting from a context window slide 🚂
2025-10-06 21:50:59 +07:00
Joshua Horton
2db1e3fd20
Merge pull request #14756 from keymanapp/refactor/web/source-input-vs-example-input
...
refactor(web): differentiate between true and represented inputs for context tokens 🚂
2025-10-06 21:50:46 +07:00
Eberhard Beilharz
229bdf8270
maint(resources): nest unit tests under parent block
...
This adds a modified `mocha-teamcity-reporter` that allows to nest the
unit tests under the parent block.
Fixes : #14839
Test-bot: skip
2025-10-06 16:09:47 +02:00
Joshua Horton
b48bf4334d
feat(web): adds legacy keyer that matches existing transform-tokenization behavior
...
Build-bot: skip test:web
Test-bot: skip
The new transform-tokenization "similarity" check added in #14822 is more limiting when only one tokenization is supported. As an interim measure, this commit and PR are designed to be reverted once the engine can handle multiple tokenizations properly and thus can swap fully to use of the new keying strategy.
2025-10-02 11:40:39 -05:00
Joshua Horton
7ca6f27962
feat(web): add method that evaluates precomputed tokenizations
...
Relates-to: #14679
Build-bot: skip build:web
Test-bot: skip
2025-10-02 11:40:09 -05:00
Joshua Horton
b7a1f8d73b
feat(web): compute number of tokens deleted in tokenization transition
...
Build-bot: skip build:web
Test-bot: skip
2025-10-02 11:34:27 -05:00
Joshua Horton
f9b80db5e3
fix(web): do not ignore empty context-tail tokens when computing tokenization transition
...
Build-bot: skip build:web
Test-bot: skip
2025-10-02 11:33:42 -05:00
Joshua Horton
95bda69e36
feat(web): add token-merge non-BMP string unit test
2025-10-02 11:33:03 -05:00
Joshua Horton
153415dd90
feat(web): add method for merging multiple ContextTokens into one
...
Build-bot: skip build:web
Test-bot: skip
2025-10-02 11:33:03 -05:00
Joshua Horton
050edaca21
fix(web): improve token-split handling of non-BMP text
2025-10-02 11:32:07 -05:00
Joshua Horton
ace4fefc0f
feat(web): add class method for splitting ContextToken instances
...
Relates-to: #14679
Build-bot: skip build:web
Test-bot: skip
2025-10-02 11:32:07 -05:00
Joshua Horton
9f5928eea0
change(web): tokenizedInputs -> inputs
2025-10-02 11:30:57 -05:00
Joshua Horton
5d1885bbff
change(web): fix minor nits from review
2025-10-01 11:48:39 -05:00
Joshua Horton
dc9744a374
feat(web): copies some context-slide unit tests from transitionTo
...
They'll eventually replace the originals, but we still need `transitionTo` at this time.
2025-09-30 10:42:42 -05:00
Eberhard Beilharz
34c43b1e15
refactor(web): cleanup in test scripts
...
Also use common function for running web types tests.
Test-bot: skip
2025-09-30 17:40:15 +02:00
Joshua Horton
f1b8cb2457
docs(web): assertion -> assumption
2025-09-30 09:18:56 -05:00
Joshua Horton
e94a2ffe8b
docs(web): assertion -> assumption
2025-09-30 09:10:34 -05:00
Joshua Horton
d1c70c5dd2
docs(web): assertion -> assumption
2025-09-30 08:54:19 -05:00
Joshua Horton
17618176c6
change(web): assertion -> assumption, ternary if-condition rework
2025-09-30 08:50:54 -05:00
