Joshua Horton
2c1c46e0dd
feat(web): add Web engine support for auto-reverting whitespace appended to Suggestions on punctuation input
...
Relates-to: #7163
Relates-to: #12013
This does not outright _fix_ them because we still need to add the ability
to set language-specific punctuation mark sets within the model, and the
model needs to use those to return an appropriate configuration. (This
commit sets defaults that are English-centric and do not generalize to
all languages.)
2025-08-15 11:54:10 -05:00
Joshua Horton
c5892ab843
change(web): trim cached tokens, token data sliding out of context window
2025-08-15 11:21:51 -05:00
Joshua Horton
7515124c41
fix(web): prevent preserved leading-token text from breaking alignment
...
Prevents scenarios in which a word sliding out of the predictive-text engine's available context window causes the engine's context alignment to fail.
Our cached tokenization data preserves all text for tokens, even those partially slid out of the active context window. We could either adjust/replace the token or allow the leading token to align with notably decreased restrictions. ... or both, to be extra robust (in case of large transform input).
This commit allows alignment with notably decreased restrictions.
2025-08-15 11:21:49 -05:00
Joshua Horton
3e3e5713e1
change(web): do not cache context states that will no longer be referenced
2025-08-15 11:14:40 -05:00
Joshua Horton
01389555d2
docs(web): more minor doc polishing
2025-08-15 11:12:33 -05:00
Joshua Horton
0562eca3e0
docs(web): improve file header
2025-08-15 11:11:31 -05:00
Joshua Horton
8f920c4b51
docs(web): improve analyzeTransition documentation
2025-08-15 11:10:39 -05:00
Joshua Horton
6a6c9e4923
refactor(web): move attemptMatchContext to ContextState.analyzeTransition
2025-08-15 10:44:42 -05:00
Joshua Horton
b73f6d3a52
fix(web): prevent post-application suggestions from overwriting applied suggestion set
2025-08-15 10:28:44 -05:00
Joshua Horton
7be3bfd226
feat(web): add unit tests for ContextTokenization.transitionTo
2025-08-15 10:28:44 -05:00
Joshua Horton
54c19e36cb
refactor(web): refactor aligned tokenization-transition operations into separate method
...
I've really been looking forward to figuring this one out. This refactor spins off the bulk of `attemptMatchContext` into a _class_ method on `ContextTokenization` - `.transitionTo` - which uses alignment data to build the new tokenization for incoming context based on the old tokenization in a clear manner.
By factorizing the method in this manner, we pave the way to allow this operation to be performed for different tokenization patterns based on either the initial or the final context (or both!) in the future - something epic-dict-breaker models will likely need to leverage. It also makes the roles of the two halves of the code _far_ clearer than it was before, in my opinion.
2025-08-15 10:28:44 -05:00
Joshua Horton
b200f19d46
refactor(web): hoist applySuggestion refactor from child branch
2025-08-15 10:21:26 -05:00
Joshua Horton
eebf85a888
docs(web): documents parameter added to computeAlignment
2025-08-15 10:21:26 -05:00
Joshua Horton
5804bc3586
change(web): copy applied-transition ID when cloning tokens
2025-08-15 10:21:26 -05:00
Joshua Horton
3cc19c22d7
feat(web): adds ContextTransition unit tests
2025-08-15 10:21:26 -05:00
Joshua Horton
bbb7dc68a4
change(web): annotate tokens affected by applied suggestions
2025-08-15 10:21:26 -05:00
Joshua Horton
ca1205ad39
refactor(web): refactor reconstruction of transitions for reverted suggestions
2025-08-15 10:21:26 -05:00
Joshua Horton
79b33c4f1c
fix(web): add special handling for alignment when applying suggestions
2025-08-15 10:21:26 -05:00
Joshua Horton
c67ea99484
change(web): rework applied-suggestion tracking + reversions
2025-08-15 10:21:26 -05:00
Joshua Horton
2490ad322d
Revert "chore(web): blocks broken unit test components"
...
This reverts commit 2152b4c3c1
2025-08-15 10:21:26 -05:00
Joshua Horton
2152b4c3c1
chore(web): blocks broken unit test components
2025-08-15 10:18:43 -05:00
Joshua Horton
2341d14b75
docs(web): add headers to new source files
2025-08-14 14:25:57 -05:00
Joshua Horton
dc9276979e
change(web): cache context transition objects, not just context state
2025-08-14 14:22:25 -05:00
Joshua Horton
f224b8bfa5
change(web): add standard header, reorder imports
2025-08-14 14:09:59 -05:00
Joshua Horton
36defc42f3
Merge branch 'refactor/web/distribution-tokenization' into refactor/web/pred-context-transition
2025-08-14 14:08:05 -05:00
Joshua Horton
37721c0d6a
docs(web): add a doc-header
2025-08-14 13:56:00 -05:00
Joshua Horton
3fe1b17239
Merge branch 'refactor/web/init-state-from-reset' into refactor/web/distribution-tokenization
2025-08-14 13:55:46 -05:00
Joshua Horton
87f0bdb32b
docs(web): add common header to new test file
2025-08-14 13:45:58 -05:00
Joshua Horton
8a1ae2d460
Merge branch 'refactor/web/pred-context-state' into refactor/web/init-state-from-reset
2025-08-14 13:38:52 -05:00
Joshua Horton
d3fabb4848
docs(web): adds standard header to new file, reorders imports
2025-08-14 13:32:20 -05:00
Joshua Horton
a4ac1a2f4d
Merge branch 'refactor/web/tokenization-alignment' into refactor/web/pred-context-state
2025-08-14 13:28:27 -05:00
Joshua Horton
c153dc73c4
Merge branch 'refactor/web/context-alignment-helpers' into refactor/web/tokenization-alignment
2025-08-14 13:19:21 -05:00
Joshua Horton
15e8c1acfb
docs(web): add header comments to new files
2025-08-14 13:13:27 -05:00
Joshua Horton
dce15a62bb
Merge branch 'refactor/web/pred-context-tokenization' into refactor/web/context-alignment-helpers
2025-08-14 13:10:12 -05:00
Joshua Horton
e9377bfc9a
Merge branch 'refactor/web/pred-context-tokens' into refactor/web/pred-context-tokenization
2025-08-14 12:59:57 -05:00
Joshua Horton
0e003edb08
Merge branch 'chore/merge-master-into-autocorrect' into refactor/web/pred-context-tokens
2025-08-14 12:51:36 -05:00
Joshua Horton
891bcaa3ef
Merge branch 'epic/autocorrect' into chore/merge-master-into-autocorrect
2025-08-14 12:26:50 -05:00
Joshua Horton
6d27c3468d
Merge branch 'refactor/web/apply-suggestion-casing' into maint/web/convert-worker-thread-tests
2025-08-14 11:01:39 -05:00
Joshua Horton
51fc6f636f
Merge branch 'fix/web/suggestion-typing' into refactor/web/apply-suggestion-casing
2025-08-14 11:00:11 -05:00
Joshua Horton
aa77e4ca0b
fix(web): remove unneeded sort from unit test
2025-08-14 10:59:45 -05:00
Joshua Horton
65885a4669
chore(web): add displayAs strings where previously unset
2025-08-14 08:32:11 -05:00
Joshua Horton
981905717a
change(web): add MAX_PROB constant for use in 'keep'-suggestion construction
2025-08-14 08:23:40 -05:00
Joshua Horton
ae469ed1b4
change(web): apply suggestion from code-review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-08-14 20:19:53 +07:00
Joshua Horton
87eb476ecd
change(web): eliminate extra tsconfig
...
I tried to get a multi-config setup to split off the tests that caused compilation issues in a prior commit, but I never got it working. It's best to simplify and remove the thing that didn't work.
2025-08-13 11:00:52 -05:00
Joshua Horton
c3124ec685
change(web): use contextTracker getter internally
2025-08-13 10:57:16 -05:00
Joshua Horton
ec888dd751
Merge branch 'refactor/web/apply-suggestion-casing' into maint/web/convert-worker-thread-tests
2025-08-13 10:52:57 -05:00
Joshua Horton
519e39e327
refactor(web): move applySuggestionCasing to standalone exportable method
2025-08-13 10:45:15 -05:00
Joshua Horton
e24deefdb8
fix(web): improve typing accuracy for suggestions & suggestion-generation
...
Also adds doc-comments to a couple of the related types
2025-08-13 10:27:13 -05:00
Joshua Horton
a01211efa4
change(web): improve exposure of once-private members
2025-08-13 08:49:41 -05:00
Joshua Horton
e9a1f6c349
maint(web): converts worker-thread tests to TS
2025-08-12 15:38:53 -05:00
