Joshua Horton
307da5fef1
Merge remote-tracking branch 'origin/fix/web/sliding-tokenization-alignment' into fix/web/sliding-tokenization-alignment
2025-08-19 12:06:21 -05:00
Joshua Horton
9b28fb8e63
Merge branch 'change/web/cache-states-selectively' into fix/web/sliding-tokenization-alignment
2025-08-19 12:05:20 -05:00
Joshua Horton
fc0348585c
Merge branch 'refactor/web/analyze-state-transition' into change/web/cache-states-selectively
2025-08-19 12:04:54 -05:00
Joshua Horton
0b9d1018e5
Merge branch 'refactor/web/tokenization-transition' into refactor/web/analyze-state-transition
2025-08-19 12:00:21 -05:00
Joshua Horton
cc5a5fc38d
Merge branch 'change/web/suggestion-tracking' into refactor/web/tokenization-transition
2025-08-19 11:58:08 -05:00
Joshua Horton
b714dc37fb
Merge branch 'change/web/cache-context-transitions' into change/web/suggestion-tracking-alt
2025-08-19 11:52:49 -05:00
Joshua Horton
5b379a82f4
chore(web): remove extra duplicated line
2025-08-19 10:52:48 -05:00
Joshua Horton
59158cb9f7
chore(web): fix count in comment
...
Co-authored-by: Eberhard Beilharz <ermshiperete@users.noreply.github.com>
2025-08-19 20:26:04 +07:00
Joshua Horton
109f216c7b
change(web): adjustments per PR review
...
Removes duplicate line, fixes doc-comment.
2025-08-19 08:17:10 -05:00
Joshua Horton
6dd1ff39a2
fix(web): restore temp-disabled unit test
2025-08-18 14:03:47 -05:00
Joshua Horton
08670c2e68
Merge branch 'change/web/cache-context-transitions-alt' into change/web/suggestion-tracking-alt
2025-08-18 13:56:24 -05:00
Joshua Horton
e3c4983031
fix(web): temporarily disable unit test
2025-08-18 13:51:00 -05:00
Joshua Horton
d6d44c4d1d
change(web): remove .get, force key type to number, base rewind on key value
2025-08-18 13:50:49 -05:00
Joshua Horton
4d66ff671d
change(web): drop redundant ordering-set (per review)
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-08-18 12:05:39 -05:00
Joshua Horton
3e9e638f9d
change(web): adjusts unit tests per review
2025-08-18 12:00:23 -05:00
Joshua Horton
222fa814ac
change(web): apply suggestion from code-review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-08-18 23:11:45 +07:00
Joshua Horton
f520a98406
change(web): simplify unit tests, per code review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-08-18 22:59:55 +07:00
Joshua Horton
470bf6a54f
Merge branch 'change/web/cache-states-selectively' into fix/web/sliding-tokenization-alignment
2025-08-15 14:06:09 -05:00
Joshua Horton
55b6b8bc5d
Merge branch 'refactor/web/analyze-state-transition' into change/web/cache-states-selectively
2025-08-15 14:05:42 -05:00
Joshua Horton
26444e7291
fix(web): fix empty context handling
2025-08-15 14:04:52 -05:00
Joshua Horton
c5892ab843
change(web): trim cached tokens, token data sliding out of context window
2025-08-15 11:21:51 -05:00
Joshua Horton
7515124c41
fix(web): prevent preserved leading-token text from breaking alignment
...
Prevents scenarios in which a word sliding out of the predictive-text engine's available context window causes the engine's context alignment to fail.
Our cached tokenization data preserves all text for tokens, even those partially slid out of the active context window. We could either adjust/replace the token or allow the leading token to align with notably decreased restrictions. ... or both, to be extra robust (in case of large transform input).
This commit allows alignment with notably decreased restrictions.
2025-08-15 11:21:49 -05:00
Joshua Horton
3e3e5713e1
change(web): do not cache context states that will no longer be referenced
2025-08-15 11:14:40 -05:00
Joshua Horton
01389555d2
docs(web): more minor doc polishing
2025-08-15 11:12:33 -05:00
Joshua Horton
0562eca3e0
docs(web): improve file header
2025-08-15 11:11:31 -05:00
Joshua Horton
8f920c4b51
docs(web): improve analyzeTransition documentation
2025-08-15 11:10:39 -05:00
Joshua Horton
6a6c9e4923
refactor(web): move attemptMatchContext to ContextState.analyzeTransition
2025-08-15 10:44:42 -05:00
Joshua Horton
b73f6d3a52
fix(web): prevent post-application suggestions from overwriting applied suggestion set
2025-08-15 10:28:44 -05:00
Joshua Horton
7be3bfd226
feat(web): add unit tests for ContextTokenization.transitionTo
2025-08-15 10:28:44 -05:00
Joshua Horton
54c19e36cb
refactor(web): refactor aligned tokenization-transition operations into separate method
...
I've really been looking forward to figuring this one out. This refactor spins off the bulk of `attemptMatchContext` into a _class_ method on `ContextTokenization` - `.transitionTo` - which uses alignment data to build the new tokenization for incoming context based on the old tokenization in a clear manner.
By factorizing the method in this manner, we pave the way to allow this operation to be performed for different tokenization patterns based on either the initial or the final context (or both!) in the future - something epic-dict-breaker models will likely need to leverage. It also makes the roles of the two halves of the code _far_ clearer than it was before, in my opinion.
2025-08-15 10:28:44 -05:00
Joshua Horton
b200f19d46
refactor(web): hoist applySuggestion refactor from child branch
2025-08-15 10:21:26 -05:00
Joshua Horton
eebf85a888
docs(web): documents parameter added to computeAlignment
2025-08-15 10:21:26 -05:00
Joshua Horton
5804bc3586
change(web): copy applied-transition ID when cloning tokens
2025-08-15 10:21:26 -05:00
Joshua Horton
3cc19c22d7
feat(web): adds ContextTransition unit tests
2025-08-15 10:21:26 -05:00
Joshua Horton
bbb7dc68a4
change(web): annotate tokens affected by applied suggestions
2025-08-15 10:21:26 -05:00
Joshua Horton
ca1205ad39
refactor(web): refactor reconstruction of transitions for reverted suggestions
2025-08-15 10:21:26 -05:00
Joshua Horton
79b33c4f1c
fix(web): add special handling for alignment when applying suggestions
2025-08-15 10:21:26 -05:00
Joshua Horton
c67ea99484
change(web): rework applied-suggestion tracking + reversions
2025-08-15 10:21:26 -05:00
Joshua Horton
2490ad322d
Revert "chore(web): blocks broken unit test components"
...
This reverts commit 2152b4c3c1
2025-08-15 10:21:26 -05:00
Joshua Horton
2152b4c3c1
chore(web): blocks broken unit test components
2025-08-15 10:18:43 -05:00
Joshua Horton
2341d14b75
docs(web): add headers to new source files
2025-08-14 14:25:57 -05:00
Joshua Horton
dc9276979e
change(web): cache context transition objects, not just context state
2025-08-14 14:22:25 -05:00
Joshua Horton
f224b8bfa5
change(web): add standard header, reorder imports
2025-08-14 14:09:59 -05:00
Joshua Horton
36defc42f3
Merge branch 'refactor/web/distribution-tokenization' into refactor/web/pred-context-transition
2025-08-14 14:08:05 -05:00
Joshua Horton
37721c0d6a
docs(web): add a doc-header
2025-08-14 13:56:00 -05:00
Joshua Horton
3fe1b17239
Merge branch 'refactor/web/init-state-from-reset' into refactor/web/distribution-tokenization
2025-08-14 13:55:46 -05:00
Joshua Horton
87f0bdb32b
docs(web): add common header to new test file
2025-08-14 13:45:58 -05:00
Joshua Horton
8a1ae2d460
Merge branch 'refactor/web/pred-context-state' into refactor/web/init-state-from-reset
2025-08-14 13:38:52 -05:00
Joshua Horton
d3fabb4848
docs(web): adds standard header to new file, reorders imports
2025-08-14 13:32:20 -05:00
Joshua Horton
a4ac1a2f4d
Merge branch 'refactor/web/tokenization-alignment' into refactor/web/pred-context-state
2025-08-14 13:28:27 -05:00
