Commit graph

488 commits

Author SHA1 Message Date
Steven R. Loomis
8787073c40 feat(developer): errs on \uXXXX escapes 🙀
- note, some older test cases had to be fixed also.

For: #10389
2024-02-13 12:26:46 -06:00
Steven R. Loomis
da4b544471 feat(developer): errs on unparseable regex 🙀
- also, make sure NFD-incompliant ranges get mapped to escaped chars
- also, propagate UnicodeSet errs out

For: #10389
2024-02-12 18:16:34 -06:00
Marc Durdin
44abf2c41e
Merge pull request #10651 from markcsinclair/fix/developer/ensure-project-populate-files-actually-adds-files
fix(developer): ensure project populate files actually adds files
2024-02-07 16:01:56 +11:00
Steven R. Loomis
5360cff8e5 Merge branch 'feat/developer/10554-sometimes-no-norm' into feat/developer/10316-range-warning 2024-02-06 16:14:54 -06:00
Steven R. Loomis
9085a4fcda feat(developer): support normalization=disabled 🙀
- assume normalization is enabled

#10554
2024-02-06 16:11:20 -06:00
Steven R. Loomis
50e0b31e9e feat(developer): support normalization=disabled 🙀
- fix check normalization state

#10554
2024-02-06 14:54:26 -06:00
Dr Mark C. Sinclair
29b6601018 fix(developer): ensure project populate files actually adds files and test 2024-02-06 10:39:47 +00:00
Steven R. Loomis
4b22d3a527
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-02-05 17:17:29 -06:00
Steven R. Loomis
10940be7e0 feat(developer): hint/warn on range check 🙀
- analyzer

#10316
2024-02-02 15:18:08 -06:00
Steven R. Loomis
107e8e98f9 feat(developer): hint/warn on range check 🙀
- analyzer

#10316
2024-02-02 12:09:23 -06:00
Steven R. Loomis
d77ed9099b feat(developer): support normalization=disabled 🙀
#10554
2024-01-31 23:16:51 -06:00
Steven R. Loomis
6c1447bb4b feat(core,developer): simplify markers 🙀
- clean up removing multi-segment processing
2024-01-31 18:17:17 -06:00
Steven R. Loomis
32d2e6ea2c feat(core,developer): simplify markers 🙀
- maybe don't need NFD table

#10317
2024-01-31 18:15:37 -06:00
Steven R. Loomis
1988c43a05 feat(common): updates to marker parsing 🙀
- clear the marker map at the end of remove_markers if no markers were found.
- add some suggested tests from @jahorton and @mcdurdin

For: #10317
2024-01-31 18:04:29 -06:00
Steven R. Loomis
722131dcf3 feat(developer): double markers once again 🙀
- add another greek double reorder.  with chips.

Fixes: #10516
2024-01-31 17:47:21 -06:00
Steven R. Loomis
1cfeeed3e4 feat(developer): double markers once again 🙀
- fix off by one in adding denormalized glue chars

Fixes: #10516
2024-01-31 17:47:21 -06:00
Steven R. Loomis
75aa04a4ee feat(developer): double markers once again 🙀
- revert comment
- reorder tests

Fixes: #10516
2024-01-31 17:47:21 -06:00
Steven R. Loomis
462cdf3f30 feat(developer): double markers once again 🙀
- use an 'end' marker for each NFD

Fixes: #10516
2024-01-31 17:47:21 -06:00
Marc Durdin
f546c44048
Merge pull request #10390 from keymanapp/epic/core/9999-normalization
epic: Keyman Core normalization 🌱
2024-01-31 10:45:51 +11:00
Steven R. Loomis
1e3ae9b8f6 feat(developer): dev side norm 🙀
- normalize in key.to, tran.from, tran.to
- add as an option in the strs pipeline

- also, assertCodePoints utility for asserting with escapes

For: #10317
2024-01-26 19:10:00 -06:00
Steven R. Loomis
6ca5a29675 chore(developer): ldml add tests expecting dev side norm 🙀
- also, change this.X to Class.X for statics

For: #10317
2024-01-26 18:06:31 -06:00
Steven R. Loomis
44d8ba7198 feat(developer): ldml marker normalization fix 🙀
- ts side

For: #10516
2024-01-26 14:54:15 -06:00
Marc Durdin
07d3e9981d chore: test preserveConstEnums in common/web/types only 2024-01-24 10:55:31 +07:00
Steven R. Loomis
ea59f0d596 fix(common): ldml: fix typo in kmc error message 🙀
- stray dot in the pathname, it's not a regex!
2024-01-23 09:44:52 -06:00
Steven R. Loomis
52d9c48128 feat(developer): document marker algo 🙀
For: #10317
2024-01-22 21:51:01 -06:00
Steven R. Loomis
c3bae838ec feat(developer): ldml marker normalization in ts 🙀
- use Intl.Segmenter instead.

For: #10317
2024-01-19 18:32:08 -06:00
Steven R. Loomis
f2082d79a6 feat(developer): ldml marker normalization in ts 🙀
- cleanup and move marker_no_index into the shared area

For: #10317
2024-01-19 18:07:51 -06:00
Steven R. Loomis
20e2d811eb feat(developer): ldml marker normalization in ts 🙀
- normalize_nfd is working

For: #10317
2024-01-19 18:00:27 -06:00
Steven R. Loomis
e05549eaf9 feat(developer): ldml marker normalization in ts 🙀
- add_back_markers

For: #10317
2024-01-19 17:44:36 -06:00
Steven R. Loomis
2bb4747c2f feat(developer): ldml marker normalization in ts 🙀
- add nfd table from ICU

For: #10317
2024-01-19 16:26:01 -06:00
Steven R. Loomis
9e9bbf8b44 feat(developer): ldml marker normalization in ts 🙀
- add remove_markers

For: #10317
2024-01-19 14:30:55 -06:00
Steven R. Loomis
6e2030b3b8 feat(developer): ldml marker normalization in ts 🙀
- intial parsing

For: #10317
2024-01-18 12:39:19 -06:00
Steven R. Loomis
16d641fc3b feat(developer): ldml marker normalization in ts 🙀
- scaffolding and tests

For: #10317
2024-01-18 05:43:44 -06:00
Steven R. Loomis
9e828b1369
Merge branch 'master' into feat/core/10319-escaping-you-epic-ldml 2024-01-17 18:49:26 -06:00
Steven R. Loomis
763bc96739 feat(core): ldml updates per code review 🙀
- remove stray change to kmx_processevent.cpp
- fix escaping of syntax chars
- fix comment per review

For: #10319
2024-01-17 18:05:28 -06:00
Marc Durdin
d2580c6a73
Merge pull request #10258 from keymanapp/feat/developer/10150-control-message-severity
feat(developer): add `-m` parameter to kmc to control message severity
2024-01-16 14:01:27 +11:00
Steven R. Loomis
a5ae4e3b25 feat(developer): don't re-escape non-syntax chars 🙀
- \u{0300} needs to turn back into an actual char, so normalization gets an opportunity to run.

#10319
2024-01-15 16:39:38 -06:00
Steven R. Loomis
04bce67d03 feat(developer): 32 bit escapades 🙀
- match sentinels as \\u…
- support \Uxxxxxxxx for >0xffff
- more escaping stuff

#10319
2024-01-12 23:21:38 -06:00
Steven R. Loomis
d662f67955 feat(developer,common): all unescaping as \uXXXX 🙀
- update unescaping from \u{…} to \uXXXX in regex
- re-unescape in the string analyzr (empty-compiler.ts) so that we catch PUA etc.
- fix some XML files that had bad DTDs
- common; updates to string util api

#10319
2024-01-11 13:07:26 -06:00
Steven R. Loomis
c49dab2103 feat(core): unscape u 🙀
- update unescaping from \u{…} to \uXXXX

#10319
2024-01-11 11:43:14 -06:00
Joshua Horton
8a7394dd23
Merge pull request #10257 from keymanapp/feat/web/es6-target-artifact
feat(web): alternate artifact - es6-bundled Web 📜
2024-01-08 15:53:43 +07:00
Steven R. Loomis
8b65010892
Merge pull request #10306 from keymanapp/feat/developer/9121-reject-bad-regex-epic-ldml
feat(developer): escape bad markers 🙀
2024-01-03 11:47:11 -06:00
Steven R. Loomis
54c77e2de9
Merge pull request #10300 from keymanapp/fix/developer/10291-layer-errs-epic-ldml
fix(developer): quell internal error when  a section fails 🙀
2024-01-02 17:05:00 -06:00
Steven R. Loomis
a6151e681b
Merge branch 'chore/core/ldml-test-robust-epic-ldml' into feat/developer/9121-reject-bad-regex-epic-ldml 2024-01-02 17:04:37 -06:00
Steven R. Loomis
24cd664502
Update common/web/types/src/ldml-keyboard/pattern-parser.ts
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-02 17:04:14 -06:00
Marc Durdin
edc950727f
Merge pull request #10208 from keymanapp/feat/developer/9473-kmc-module-api-consolidation
feat(developer): Consolidate public APIs for kmc modules
2024-01-02 15:35:47 +11:00
Steven R. Loomis
80ba5aa0ce fix(developer): escape markers as regex 🙀
- escape markers to prevent the n'th marker from having some other meaning
2023-12-30 11:18:51 -06:00
Joshua A. Horton
dc7bfc1344 chore(web): Merge branch 'change/web/build-bundler-replacement' into feat/web/es6-target-artifact 2023-12-22 12:59:20 +07:00
Steven R. Loomis
21a7680278 fix(developer): quell internal error when a section fails 🙀
- when a required section is missing, only throw an internal error IF we haven't already flagged a problem.  for example, if 'vars' fails to compile because of a problem, and we already have an error on file, then don't throw an internal error that 'bksp' depends on uninitialized 'vars'.

- mimic this flow in the test helpers.

- add a test case for this in the test helpers, in test-tran

- add a strictError flag to say that no additional errs are allowed.

Fixes: #10291
2023-12-21 22:14:54 -06:00
Joshua A. Horton
5eb787bac6 fix(web): default multitap logic conditions on and affects both default and shift layers 2023-12-19 09:50:38 +07:00