Joshua Horton
fae2dec1fe
Merge branch 'fix/web/sliding-tokenization-alignment' into refactor/web/independent-compute-alignment
2025-08-27 15:04:29 -05:00
Joshua Horton
0937d0da4c
Merge branch 'fix/web/post-bksp-applies' into fix/web/sliding-tokenization-alignment
2025-08-26 08:54:45 -05:00
Joshua Horton
1d2b9be778
Merge remote-tracking branch 'origin/fix/web/post-bksp-applies' into fix/web/post-bksp-applies
2025-08-26 08:53:26 -05:00
Joshua Horton
5546843254
change(web): clarify path-last-match assertions
2025-08-26 08:52:50 -05:00
Joshua Horton
1044d90e47
Merge branch 'fix/web/sliding-tokenization-alignment' into refactor/web/independent-compute-alignment
2025-08-25 15:55:29 -05:00
Joshua Horton
d35bd80a90
fix(web): fix last commit's fix - missed the return keyword a few times
2025-08-25 15:48:05 -05:00
Joshua Horton
e3510b9e51
Merge branch 'fix/web/sliding-tokenization-alignment' into refactor/web/independent-compute-alignment
2025-08-25 15:31:53 -05:00
Joshua Horton
4680295108
fix(web): fix error that occurs when applying suggestion to empty token
2025-08-25 15:22:45 -05:00
Joshua Horton
31e4a932d1
docs(web): Fix comment in alignment-helpers.ts
...
Co-authored-by: Darcy Wong <darcy_wong@sil.org>
2025-08-21 20:04:41 +07:00
Joshua Horton
625f734223
refactor(web): move computeAlignment core to standalone method
...
Relates-to: #14480
This is being done to prepare for a fix to #14480 , which is needed to better handle complex input transforms and tokenization scenarios. The easiest way to consistently index the tokenized transforms... is to perform context alignment on the results of transform application.
2025-08-20 10:18:29 -05:00
Joshua Horton
e794f34dc9
Merge branch 'fix/web/post-bksp-applies' into fix/web/sliding-tokenization-alignment
2025-08-20 09:45:01 -05:00
Joshua Horton
9fbe0f4c40
fix(web): handle suggestion application after backspacing
2025-08-20 09:39:47 -05:00
Joshua Horton
dc230f68bc
fix(web): improve logic check for deleted token before attempting correction-search
2025-08-19 12:39:43 -05:00
Joshua Horton
307da5fef1
Merge remote-tracking branch 'origin/fix/web/sliding-tokenization-alignment' into fix/web/sliding-tokenization-alignment
2025-08-19 12:06:21 -05:00
Joshua Horton
9b28fb8e63
Merge branch 'change/web/cache-states-selectively' into fix/web/sliding-tokenization-alignment
2025-08-19 12:05:20 -05:00
Joshua Horton
fc0348585c
Merge branch 'refactor/web/analyze-state-transition' into change/web/cache-states-selectively
2025-08-19 12:04:54 -05:00
Joshua Horton
0b9d1018e5
Merge branch 'refactor/web/tokenization-transition' into refactor/web/analyze-state-transition
2025-08-19 12:00:21 -05:00
Joshua Horton
cc5a5fc38d
Merge branch 'change/web/suggestion-tracking' into refactor/web/tokenization-transition
2025-08-19 11:58:08 -05:00
Joshua Horton
b714dc37fb
Merge branch 'change/web/cache-context-transitions' into change/web/suggestion-tracking-alt
2025-08-19 11:52:49 -05:00
Joshua Horton
5b379a82f4
chore(web): remove extra duplicated line
2025-08-19 10:52:48 -05:00
Joshua Horton
59158cb9f7
chore(web): fix count in comment
...
Co-authored-by: Eberhard Beilharz <ermshiperete@users.noreply.github.com>
2025-08-19 20:26:04 +07:00
Joshua Horton
109f216c7b
change(web): adjustments per PR review
...
Removes duplicate line, fixes doc-comment.
2025-08-19 08:17:10 -05:00
Joshua Horton
6dd1ff39a2
fix(web): restore temp-disabled unit test
2025-08-18 14:03:47 -05:00
Joshua Horton
08670c2e68
Merge branch 'change/web/cache-context-transitions-alt' into change/web/suggestion-tracking-alt
2025-08-18 13:56:24 -05:00
Joshua Horton
e3c4983031
fix(web): temporarily disable unit test
2025-08-18 13:51:00 -05:00
Joshua Horton
d6d44c4d1d
change(web): remove .get, force key type to number, base rewind on key value
2025-08-18 13:50:49 -05:00
Joshua Horton
4d66ff671d
change(web): drop redundant ordering-set (per review)
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-08-18 12:05:39 -05:00
Joshua Horton
3e9e638f9d
change(web): adjusts unit tests per review
2025-08-18 12:00:23 -05:00
Joshua Horton
222fa814ac
change(web): apply suggestion from code-review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-08-18 23:11:45 +07:00
Joshua Horton
f520a98406
change(web): simplify unit tests, per code review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-08-18 22:59:55 +07:00
Joshua Horton
470bf6a54f
Merge branch 'change/web/cache-states-selectively' into fix/web/sliding-tokenization-alignment
2025-08-15 14:06:09 -05:00
Joshua Horton
55b6b8bc5d
Merge branch 'refactor/web/analyze-state-transition' into change/web/cache-states-selectively
2025-08-15 14:05:42 -05:00
Joshua Horton
26444e7291
fix(web): fix empty context handling
2025-08-15 14:04:52 -05:00
Joshua Horton
c5892ab843
change(web): trim cached tokens, token data sliding out of context window
2025-08-15 11:21:51 -05:00
Joshua Horton
7515124c41
fix(web): prevent preserved leading-token text from breaking alignment
...
Prevents scenarios in which a word sliding out of the predictive-text engine's available context window causes the engine's context alignment to fail.
Our cached tokenization data preserves all text for tokens, even those partially slid out of the active context window. We could either adjust/replace the token or allow the leading token to align with notably decreased restrictions. ... or both, to be extra robust (in case of large transform input).
This commit allows alignment with notably decreased restrictions.
2025-08-15 11:21:49 -05:00
Joshua Horton
3e3e5713e1
change(web): do not cache context states that will no longer be referenced
2025-08-15 11:14:40 -05:00
Joshua Horton
01389555d2
docs(web): more minor doc polishing
2025-08-15 11:12:33 -05:00
Joshua Horton
0562eca3e0
docs(web): improve file header
2025-08-15 11:11:31 -05:00
Joshua Horton
8f920c4b51
docs(web): improve analyzeTransition documentation
2025-08-15 11:10:39 -05:00
Joshua Horton
6a6c9e4923
refactor(web): move attemptMatchContext to ContextState.analyzeTransition
2025-08-15 10:44:42 -05:00
Joshua Horton
b73f6d3a52
fix(web): prevent post-application suggestions from overwriting applied suggestion set
2025-08-15 10:28:44 -05:00
Joshua Horton
7be3bfd226
feat(web): add unit tests for ContextTokenization.transitionTo
2025-08-15 10:28:44 -05:00
Joshua Horton
54c19e36cb
refactor(web): refactor aligned tokenization-transition operations into separate method
...
I've really been looking forward to figuring this one out. This refactor spins off the bulk of `attemptMatchContext` into a _class_ method on `ContextTokenization` - `.transitionTo` - which uses alignment data to build the new tokenization for incoming context based on the old tokenization in a clear manner.
By factorizing the method in this manner, we pave the way to allow this operation to be performed for different tokenization patterns based on either the initial or the final context (or both!) in the future - something epic-dict-breaker models will likely need to leverage. It also makes the roles of the two halves of the code _far_ clearer than it was before, in my opinion.
2025-08-15 10:28:44 -05:00
Joshua Horton
b200f19d46
refactor(web): hoist applySuggestion refactor from child branch
2025-08-15 10:21:26 -05:00
Joshua Horton
eebf85a888
docs(web): documents parameter added to computeAlignment
2025-08-15 10:21:26 -05:00
Joshua Horton
5804bc3586
change(web): copy applied-transition ID when cloning tokens
2025-08-15 10:21:26 -05:00
Joshua Horton
3cc19c22d7
feat(web): adds ContextTransition unit tests
2025-08-15 10:21:26 -05:00
Joshua Horton
bbb7dc68a4
change(web): annotate tokens affected by applied suggestions
2025-08-15 10:21:26 -05:00
Joshua Horton
ca1205ad39
refactor(web): refactor reconstruction of transitions for reverted suggestions
2025-08-15 10:21:26 -05:00
Joshua Horton
79b33c4f1c
fix(web): add special handling for alignment when applying suggestions
2025-08-15 10:21:26 -05:00
