Joshua Horton
cf5b2d91fc
Merge branch 'change/web/remove-old-alignment-and-transform-tokenization' into change/web/remove-tracker-analyze-state
2025-10-16 11:19:28 -05:00
Joshua Horton
0b34fde05e
Merge branch 'refactor/web/delayed-reversion-via-tokenization-analysis' into change/web/remove-old-alignment-and-transform-tokenization
2025-10-16 11:17:29 -05:00
Joshua Horton
646fdc1661
fix(web): adjust unit tests to match change in ContextTokenization constructor parameterization
...
Build-bot: build
Test-bot: test
2025-10-16 11:09:35 -05:00
Joshua Horton
1ce8b7589a
Merge branch 'refactor/web/preservation-transforms' into refactor/web/delayed-reversion-via-tokenization-analysis
2025-10-16 10:18:50 -05:00
Joshua Horton
1942f8f8b1
Merge branch 'feat/web/link-new-tokenization-transitioner' into refactor/web/preservation-transforms
2025-10-16 10:18:06 -05:00
Joshua Horton
d376c777a4
Merge branch 'fix/web/base-context-state-validation' into feat/web/link-new-tokenization-transitioner
2025-10-16 10:17:32 -05:00
Joshua Horton
4612b8cd2a
Merge branch 'epic/autocorrect' into fix/web/base-context-state-validation
2025-10-16 09:54:55 -05:00
Joshua Horton
aaec2837ca
Merge pull request #14825 from keymanapp/feat/web/interim-legacy-keyer
...
feat(web): adds legacy keyer that matches existing transform-tokenization behavior 🚂
2025-10-16 21:49:54 +07:00
Joshua Horton
ebc264ac15
Merge branch 'epic/autocorrect' into feat/web/evaluate-precomputed-tokenization
2025-10-16 09:17:22 -05:00
Joshua Horton
66209db351
Merge pull request #14876 from keymanapp/feat/web/compute-removed-token-count
...
feat(web): compute number of tokens deleted in tokenization transition 🚂
2025-10-16 21:09:35 +07:00
Joshua Horton
1a78cc5a25
Merge pull request #14875 from keymanapp/fix/web/empty-context-tail-handling
...
fix(web): do not ignore empty context-tail tokens when computing tokenization transition 🚂
2025-10-16 21:09:26 +07:00
Keyman Server
0753cc548c
Merge branch 'epic/autocorrect' into chore/A19S13-merge-master-into-autocorrect
2025-10-10 05:04:36 +02:00
Joshua Horton
490c5e6751
Merge branch 'feat/web/interim-legacy-keyer' into fix/web/base-context-state-validation
2025-10-08 11:43:06 -05:00
Joshua Horton
41619fed1f
Merge branch 'feat/web/evaluate-precomputed-tokenization' into feat/web/interim-legacy-keyer
2025-10-08 11:41:39 -05:00
Joshua Horton
8dd5748fec
docs(web): "assumption", not "assertion"
2025-10-08 11:37:36 -05:00
Joshua Horton
b28bb2deb7
Merge pull request #14841 from keymanapp/feat/web/merge-context-tokens
...
feat(web): add method for merging multiple ContextTokens into one 🚂
2025-10-08 22:35:38 +07:00
Joshua Horton
da444878ff
Merge pull request #14840 from keymanapp/feat/web/split-context-tokens
...
feat(web): add class method for splitting ContextToken instances 🚂
2025-10-08 22:35:29 +07:00
Joshua Horton
d70d7a75c9
Merge pull request #14823 from keymanapp/feat/web/tokenization-subset-builder
...
feat(web): build subsets of prior-state + input pairings with compatible tokenization results 🚂
2025-10-08 22:35:20 +07:00
Joshua Horton
e95422565d
Merge pull request #14822 from keymanapp/feat/web/key-common-result-tokenizations
...
feat(web): define keying function that matches paths that result in compatible context tokenizations 🚂
2025-10-08 22:35:09 +07:00
Joshua Horton
05a89a07a4
Merge pull request #14820 from keymanapp/feat/web/track-boundary-token-source
...
feat(web): track source input ranges for the edit-boundary source token 🚂
2025-10-08 22:34:58 +07:00
Joshua Horton
eb3a1d86bb
Merge pull request #14790 from keymanapp/feat/web/boundary-shifting-transform-tokenization
...
feat(web): precompute transform tokenization while handling when existing boundaries shift 🚂
2025-10-08 22:34:48 +07:00
Joshua Horton
4698f22c86
Merge branch 'feat/web/compute-removed-token-count' into feat/web/evaluate-precomputed-tokenization
2025-10-08 08:52:04 -05:00
Joshua Horton
b2f460d0be
Merge branch 'fix/web/empty-context-tail-handling' into feat/web/compute-removed-token-count
2025-10-08 08:51:22 -05:00
Joshua Horton
96fd4b6a6f
fix(web): update unit test use of method with name changed to mapWhitespacedTokenization
2025-10-08 08:50:57 -05:00
Joshua Horton
30f0eee46b
Merge branch 'feat/web/merge-context-tokens' into fix/web/empty-context-tail-handling
2025-10-08 08:50:08 -05:00
Joshua Horton
886642c67d
Merge branch 'feat/web/split-context-tokens' into feat/web/merge-context-tokens
2025-10-08 08:49:25 -05:00
Joshua Horton
2b290955a4
Merge branch 'feat/web/tokenization-subset-builder' into feat/web/split-context-tokens
2025-10-08 08:45:19 -05:00
Joshua Horton
97789c008b
fix(web): update unit test use of method with name changed to mapWhitespacedTokenization
2025-10-08 08:44:51 -05:00
Joshua Horton
8a69cd4689
Merge branch 'feat/web/key-common-result-tokenizations' into feat/web/tokenization-subset-builder
2025-10-08 08:43:28 -05:00
Joshua Horton
8ebfab10fc
docs(web): remove duplicate words in file header comment
2025-10-08 08:43:02 -05:00
Joshua Horton
9c8747d9c8
Merge branch 'feat/web/track-boundary-token-source' into feat/web/key-common-result-tokenizations
2025-10-08 08:42:01 -05:00
Joshua Horton
164b6306ef
Merge branch 'feat/web/boundary-shifting-transform-tokenization' into feat/web/track-boundary-token-source
2025-10-08 08:40:55 -05:00
Joshua Horton
01a0eeec19
change(web): renames precomputeTokenizationAfterInput to mapWhitespacedTokenization
2025-10-08 08:40:15 -05:00
Joshua Horton
df7fec06c9
change(web): remove ContextTracker.analyzeState
...
WIth the changes in #14883 , we are now safe to remove the fallback check that was the one place still utilizing ContextTracker.analyzeState. As a result... time for more dead code removal.
Build-bot: skip build:web
Test-bot: skip
2025-10-07 12:01:13 -05:00
Joshua Horton
a3f1a89c97
change(web): remove isSubstitutionAlignable
2025-10-07 12:00:26 -05:00
Joshua Horton
24e82f42ce
change(web): remove computeAlignment
...
While handy, this method did not adequately account for 'split' / 'merge' edit cases and also required special handling to avoid certain degenerate edit-path cases. The newer method put in place (see #14874 ) handles such edits and avoids the degeneracy problem that resulted when relying on edit paths for the edited portion of context.
Build-bot: skip build:web
Test-bot: skip
2025-10-07 12:00:26 -05:00
Joshua Horton
c6cf11afb4
change(web): remove prior transform-tokenization methods
...
The removed methods did not properly support cases adjacent to 'split' / 'merge' edits.
2025-10-07 12:00:26 -05:00
Joshua Horton
aa477a9ee3
fix(web): restore unit tests
2025-10-07 11:30:13 -05:00
Joshua Horton
604b2526a1
refactor(web): use tokenization analysis to maintain delayed reversion feature
2025-10-07 11:29:47 -05:00
Joshua Horton
b2cb13741a
refactor(web): rework preservation transforms to prep for multi-tokenization
...
Build-bot: skip build:web
Test-bot: skip
2025-10-07 11:25:13 -05:00
Joshua Horton
c3e5c66a6e
docs(web): thoughts toward upcoming changes for preservation transforms
2025-10-07 11:19:15 -05:00
Joshua Horton
39dd3b50a3
change(web): clean up computation of preservationTransform
2025-10-07 09:20:37 -05:00
Joshua Horton
98a40a3f64
change(web): clarify role of sliceIndex in token-merge, token-split tests
2025-10-06 16:47:51 -05:00
Joshua Horton
199c962f6f
Merge pull request #14807 from keymanapp/feat/web/tokenized-transform-assembly
...
change(web): provide alternate helper for assembly of tokenized transforms 🚂
2025-10-06 21:52:03 +07:00
Joshua Horton
df2a868605
Merge pull request #14806 from keymanapp/feat/web/compute-merge-split-effects
...
feat(web): add helper for aligning tokens after merges, splits 🚂
2025-10-06 21:51:52 +07:00
Joshua Horton
e8f28fdf36
Merge pull request #14805 from keymanapp/feat/web/trace-insert-edits
...
feat(web): add helper for tokenizing Transform.insert via tokenization of applied context 🚂
2025-10-06 21:51:38 +07:00
Joshua Horton
a9a761443f
Merge pull request #14754 from keymanapp/feat/web/backward-slide-tokenization
...
feat(web): add helper method for updating context tokenization before applying inputs 🚂
2025-10-06 21:51:20 +07:00
Joshua Horton
6800734d39
Merge pull request #14800 from keymanapp/feat/web/build-token-aligned-edge-windows
...
feat(web): build token-aligned edge windows 🚂
2025-10-06 21:51:10 +07:00
Joshua Horton
e681dc46ed
Merge pull request #14751 from keymanapp/feat/web/context-slide-deltas
...
feat(web): add method to determine the change to context resulting from a context window slide 🚂
2025-10-06 21:50:59 +07:00
Joshua Horton
2db1e3fd20
Merge pull request #14756 from keymanapp/refactor/web/source-input-vs-example-input
...
refactor(web): differentiate between true and represented inputs for context tokens 🚂
2025-10-06 21:50:46 +07:00
