9 commits

Author	SHA1	Message	Date
Steven R. Loomis	263d0e32f2	chore(core): update comments and remove a raw numeric literal ... - per review comments Fixes: #9467 Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>	2024-06-13 12:17:55 -05:00
Steven R. Loomis	f156a7264d	feat(core): generator in core/src for util_normalize_table.h ... - temporary header file generated by wasm during build - built using icu - test_unicode verifies the contents and synchronization with running ICU. - this is used by util_normalize to provide normalization properties under wasm without needing to include ICU. Fixes: #9467	2024-06-05 15:47:20 -05:00
Steven R. Loomis	e9673867fa	feat(core): speedup NFD boundary table ... - use RLE encoding, thanks @mcdurdin - much smaller table and faster lookup Fixes: #9467	2024-06-04 12:23:16 -05:00
Steven R. Loomis	53a6638a70	feat(core): generate and use static table in wasm for NFD boundary ... - add core/tools build tree with custom targets - add to core/build.sh to generate nfd_table.h - test_unicode to validate Unicode version and compare NFD to actual ICU - currently, linear search of the table.	2024-06-04 11:15:17 -05:00
Steven R. Loomis	5d420248eb	feat(core): move more normalization logic into JS ... - major redo of actions_normalize - into UTF-32 and not using ICU directly - add some utilities: u32len, u32dup, context_items_from_utf32 #9467	2024-05-24 18:20:30 -05:00
Steven R. Loomis	a80a0a7dcb	feat(core): move more normalization logic into JS ... - add a normalize_nfd() which takes a single codepoint - temporarily keep ICU in actions_normalize.cpp and ldml_transforms.cpp - expand wasm opts in unit tests	2024-05-24 10:42:57 -05:00
Steven R. Loomis	465c4bfdc0	feat(core): add a KMN_NO_ICU internal switch to start being able to turn off ICU ... - always set to 0 for now (keep ICU around) - set KMN_IN_LDML_TESTS in tests to keep ICU there for test and comparison - add core_icu.cpp and put some utils there. #9467	2024-05-24 09:24:53 -05:00
Steven R. Loomis	43cd828e9d	feat(core): change normalize_nfd() to use JS native call instead of ICU ... - does not hit all ICU usage yet #9467	2024-05-23 17:09:00 -05:00
Steven R. Loomis	d8b5c25954	chore(core): refactor to move most normalization calls into util_normalize.cpp ... - fold ldml_utils.hpp into core_icu.h - refactor to use km::core::util::normalize_nfd() functions where simple to do so For #9467	2024-05-23 14:47:38 -05:00