Commit graph

477 commits

Author SHA1 Message Date
Steven R. Loomis
6c1447bb4b feat(core,developer): simplify markers 🙀
- clean up removing multi-segment processing
2024-01-31 18:17:17 -06:00
Steven R. Loomis
32d2e6ea2c feat(core,developer): simplify markers 🙀
- maybe don't need NFD table

#10317
2024-01-31 18:15:37 -06:00
Steven R. Loomis
1988c43a05 feat(common): updates to marker parsing 🙀
- clear the marker map at the end of remove_markers if no markers were found.
- add some suggested tests from @jahorton and @mcdurdin

For: #10317
2024-01-31 18:04:29 -06:00
Steven R. Loomis
722131dcf3 feat(developer): double markers once again 🙀
- add another greek double reorder.  with chips.

Fixes: #10516
2024-01-31 17:47:21 -06:00
Steven R. Loomis
1cfeeed3e4 feat(developer): double markers once again 🙀
- fix off by one in adding denormalized glue chars

Fixes: #10516
2024-01-31 17:47:21 -06:00
Steven R. Loomis
75aa04a4ee feat(developer): double markers once again 🙀
- revert comment
- reorder tests

Fixes: #10516
2024-01-31 17:47:21 -06:00
Steven R. Loomis
462cdf3f30 feat(developer): double markers once again 🙀
- use an 'end' marker for each NFD

Fixes: #10516
2024-01-31 17:47:21 -06:00
Marc Durdin
f546c44048
Merge pull request #10390 from keymanapp/epic/core/9999-normalization
epic: Keyman Core normalization 🌱
2024-01-31 10:45:51 +11:00
Steven R. Loomis
1e3ae9b8f6 feat(developer): dev side norm 🙀
- normalize in key.to, tran.from, tran.to
- add as an option in the strs pipeline

- also, assertCodePoints utility for asserting with escapes

For: #10317
2024-01-26 19:10:00 -06:00
Steven R. Loomis
6ca5a29675 chore(developer): ldml add tests expecting dev side norm 🙀
- also, change this.X to Class.X for statics

For: #10317
2024-01-26 18:06:31 -06:00
Steven R. Loomis
44d8ba7198 feat(developer): ldml marker normalization fix 🙀
- ts side

For: #10516
2024-01-26 14:54:15 -06:00
Marc Durdin
07d3e9981d chore: test preserveConstEnums in common/web/types only 2024-01-24 10:55:31 +07:00
Steven R. Loomis
ea59f0d596 fix(common): ldml: fix typo in kmc error message 🙀
- stray dot in the pathname, it's not a regex!
2024-01-23 09:44:52 -06:00
Steven R. Loomis
52d9c48128 feat(developer): document marker algo 🙀
For: #10317
2024-01-22 21:51:01 -06:00
Steven R. Loomis
c3bae838ec feat(developer): ldml marker normalization in ts 🙀
- use Intl.Segmenter instead.

For: #10317
2024-01-19 18:32:08 -06:00
Steven R. Loomis
f2082d79a6 feat(developer): ldml marker normalization in ts 🙀
- cleanup and move marker_no_index into the shared area

For: #10317
2024-01-19 18:07:51 -06:00
Steven R. Loomis
20e2d811eb feat(developer): ldml marker normalization in ts 🙀
- normalize_nfd is working

For: #10317
2024-01-19 18:00:27 -06:00
Steven R. Loomis
e05549eaf9 feat(developer): ldml marker normalization in ts 🙀
- add_back_markers

For: #10317
2024-01-19 17:44:36 -06:00
Steven R. Loomis
2bb4747c2f feat(developer): ldml marker normalization in ts 🙀
- add nfd table from ICU

For: #10317
2024-01-19 16:26:01 -06:00
Steven R. Loomis
9e9bbf8b44 feat(developer): ldml marker normalization in ts 🙀
- add remove_markers

For: #10317
2024-01-19 14:30:55 -06:00
Steven R. Loomis
6e2030b3b8 feat(developer): ldml marker normalization in ts 🙀
- intial parsing

For: #10317
2024-01-18 12:39:19 -06:00
Steven R. Loomis
16d641fc3b feat(developer): ldml marker normalization in ts 🙀
- scaffolding and tests

For: #10317
2024-01-18 05:43:44 -06:00
Steven R. Loomis
9e828b1369
Merge branch 'master' into feat/core/10319-escaping-you-epic-ldml 2024-01-17 18:49:26 -06:00
Steven R. Loomis
763bc96739 feat(core): ldml updates per code review 🙀
- remove stray change to kmx_processevent.cpp
- fix escaping of syntax chars
- fix comment per review

For: #10319
2024-01-17 18:05:28 -06:00
Marc Durdin
d2580c6a73
Merge pull request #10258 from keymanapp/feat/developer/10150-control-message-severity
feat(developer): add `-m` parameter to kmc to control message severity
2024-01-16 14:01:27 +11:00
Steven R. Loomis
a5ae4e3b25 feat(developer): don't re-escape non-syntax chars 🙀
- \u{0300} needs to turn back into an actual char, so normalization gets an opportunity to run.

#10319
2024-01-15 16:39:38 -06:00
Steven R. Loomis
04bce67d03 feat(developer): 32 bit escapades 🙀
- match sentinels as \\u…
- support \Uxxxxxxxx for >0xffff
- more escaping stuff

#10319
2024-01-12 23:21:38 -06:00
Steven R. Loomis
d662f67955 feat(developer,common): all unescaping as \uXXXX 🙀
- update unescaping from \u{…} to \uXXXX in regex
- re-unescape in the string analyzr (empty-compiler.ts) so that we catch PUA etc.
- fix some XML files that had bad DTDs
- common; updates to string util api

#10319
2024-01-11 13:07:26 -06:00
Steven R. Loomis
c49dab2103 feat(core): unscape u 🙀
- update unescaping from \u{…} to \uXXXX

#10319
2024-01-11 11:43:14 -06:00
Joshua Horton
8a7394dd23
Merge pull request #10257 from keymanapp/feat/web/es6-target-artifact
feat(web): alternate artifact - es6-bundled Web 📜
2024-01-08 15:53:43 +07:00
Steven R. Loomis
8b65010892
Merge pull request #10306 from keymanapp/feat/developer/9121-reject-bad-regex-epic-ldml
feat(developer): escape bad markers 🙀
2024-01-03 11:47:11 -06:00
Steven R. Loomis
54c77e2de9
Merge pull request #10300 from keymanapp/fix/developer/10291-layer-errs-epic-ldml
fix(developer): quell internal error when  a section fails 🙀
2024-01-02 17:05:00 -06:00
Steven R. Loomis
a6151e681b
Merge branch 'chore/core/ldml-test-robust-epic-ldml' into feat/developer/9121-reject-bad-regex-epic-ldml 2024-01-02 17:04:37 -06:00
Steven R. Loomis
24cd664502
Update common/web/types/src/ldml-keyboard/pattern-parser.ts
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-02 17:04:14 -06:00
Marc Durdin
edc950727f
Merge pull request #10208 from keymanapp/feat/developer/9473-kmc-module-api-consolidation
feat(developer): Consolidate public APIs for kmc modules
2024-01-02 15:35:47 +11:00
Steven R. Loomis
80ba5aa0ce fix(developer): escape markers as regex 🙀
- escape markers to prevent the n'th marker from having some other meaning
2023-12-30 11:18:51 -06:00
Joshua A. Horton
dc7bfc1344 chore(web): Merge branch 'change/web/build-bundler-replacement' into feat/web/es6-target-artifact 2023-12-22 12:59:20 +07:00
Steven R. Loomis
21a7680278 fix(developer): quell internal error when a section fails 🙀
- when a required section is missing, only throw an internal error IF we haven't already flagged a problem.  for example, if 'vars' fails to compile because of a problem, and we already have an error on file, then don't throw an internal error that 'bksp' depends on uninitialized 'vars'.

- mimic this flow in the test helpers.

- add a test case for this in the test helpers, in test-tran

- add a strictError flag to say that no additional errs are allowed.

Fixes: #10291
2023-12-21 22:14:54 -06:00
Joshua A. Horton
5eb787bac6 fix(web): default multitap logic conditions on and affects both default and shift layers 2023-12-19 09:50:38 +07:00
Steven R. Loomis
6364adceeb fix(web): unbreak some ts paths 2023-12-18 17:23:39 -06:00
Joshua A. Horton
f6e77a3736 feat(web): alternate artifact - es6-bundled Web
Does not affect the worker.

Bundling is in a temp-script and not properly connected to our actual build infrastructure.
2023-12-18 10:32:35 +07:00
Marc Durdin
0ebc5d6aab feat(developer): add -m parameter to control message display
Fixes #10150.

This feature adds a command line parameter to kmc to control the
severity of INFO, HINT and WARN level messages. The severity of any of
these messages can be reduced all the way to INFO, or even disabled, or
increased to ERROR. The compiler and the reporter will honour the new
code and fail builds when failure-level codes are encountered.

Inputs are validated against all the available compiler messages in all
modules. This also provides a good pathway for automated future
documentation generation of compiler messages.

In order to process the input parameters cleanly, have refactored some
of kmc's build command logic.

This commit increases code coverage for kmc to roughly 50%.
2023-12-14 13:17:29 +07:00
Joshua Horton
535bfec1b3
Merge pull request #10242 from keymanapp/chore/merge-master-into-feature-gestures
chore: merge master into feature-gestures 🐵
2023-12-13 14:09:05 +07:00
Marc Durdin
60bb1e4d9e chore(developer): Merge branch 'master' into feat/developer/9473-kmc-module-api-consolidation 2023-12-13 05:01:20 +07:00
Marc Durdin
52038e22aa chore(developer): remove unintended usage of callbacks.path/callbacks.fs in kmc 2023-12-11 11:43:24 +07:00
Marc Durdin
f9c3be09a6 refactor(developer): move outFile out of CompilerBaseOptions
`outFile` is not a compiler option, and so it should not be a part of
the `CompilerBaseOptions` interface. This formally separates the
command-line options from the compiler options, which eliminates a
number of restrictive structures.
2023-12-11 09:47:23 +07:00
Marc Durdin
6bebefda10 feat(developer): move kmc-kmn to KeymanCompiler interface
Part of #9473.

KmnCompiler now implements KeymanCompiler, including the returned
artifacts. Also establishes the common types for the compiler interfaces
and consolidates and renames various API surfaces for kmc-kmn.
2023-12-07 11:05:31 +07:00
Marc Durdin
d693bbd457 chore: address review comments 2023-12-06 13:00:04 +07:00
Marc Durdin
a330666515 fix(common): kpj loader needs to handle empty Files element
Fixes #10143.
2023-12-06 12:37:25 +07:00
Joshua A. Horton
e9dd7a1216 chore(web): marks new default key's ID as private use 2023-12-05 08:47:40 +07:00