Commit graph

19875 commits

Author SHA1 Message Date
Marc Durdin
71df68cd13 refactor(developer): move fs for kmn load to caller 2023-05-30 13:09:35 +07:00
Marc Durdin
f36a908d94 chore(developer): move keyboard repo fixtures
Relocates the keyboard-repo fixtures so we can add other local fixtures
for unit tests.
2023-05-30 13:08:53 +07:00
Marc Durdin
05fe88bb3a chore(developer): remove usekmcmplib flag from kmcomp 2023-05-30 10:57:57 +07:00
Marc Durdin
d763cf315a chore(developer): refactor kmcmplib interfaces
Relates to #8493.

* Removes kmcmplib calls from kmcmpdll (now that we have kmc)
* Removes old kmcmp_CompileKeyboardFile and
  kmcmp_CompileKeyboardFileToBuffer functions in preference for a much
  cleaner kmcmp_CompileKeyboard function
* Removes json validation helper from kmcmplib (we'll use js-native
  json schema validation instead)

This change means that we no longer need to keep compfile.h consistent
between kmcmplib and kmcmpdll. This will simplify upcoming refactoring
of kmcmplib.

kmcmplib no longer writes files, but it does still read them. The next
refactor will move file load responsibility into the caller.
2023-05-30 08:14:23 +07:00
Marc Durdin
a92917f947 chore(developer): replace cwrap wasm bindings
This moves the remainder of the WASM interfaces in kmcmplib to using
emscripten bind. It is a little bit of a step backwards at present for
parseUnicodeSet, because I've changed the output buffer type to an int
for the purposes of simplifying the binding just now. But that can be
improved later, and at least we are consistent with the binding methods.

Next step is to move the filesystem access out of kmcmplib.
2023-05-25 12:54:27 +07:00
Marc Durdin
50186e986e chore(developer): remove SetError macro
The `SetError` macro emitted `return FALSE;` which was confusing for
reviewing code flow. Replaced with explicit `AddCompileMessage()` and
`return FALSE;`.
2023-05-25 09:03:22 +07:00
Marc Durdin
ef101d746c chore(developer): address review comments 2023-05-25 08:24:09 +07:00
Marc Durdin
cdf5f7a7c4 chore(developer): consolidate 'extra' field for FILE_KEYBOARD
FILE_KEYBOARD is a structure used in multiple libraries and languages.
It is passed around between kmcmplib, kmcmpdll, and kmcomp, with three
separate definitions (kmcmplib/compfile.h, kmcmpdll/compfile.h, and
kmcomp/compile.pas). These duplicate definitions must be maintained for
now because of type issues between the legacy kmcmpdll and kmcmplib,
which means that the definitions must be kept in sync.

Because FILE_KEYBOARD_EXTRA has a C++ class in it, it will be
initialized in CompileKeyboardHandle rather than in the caller, so
FILE_KEYBOARD.extra is now a pointer rather than a struct.

When original kmcomp and kmcmpdll are eventually removed, these
duplications will disappear and some of the memory management and
lifecycle will become easier to manage.
2023-05-24 12:14:41 +07:00
Marc Durdin
d1d6dc5d99 refactor(developer): compiler interface part 1 for wasm
This starts the refactor of the .kmn compiler interfaces for WASM,
moving more file writes into typescript and out of the compiler. Adds a
cleaner binding and structures for passing compile metadata back to
typescript for additional build steps (e.g. kvk, js builds).

Will rebase the Typescript-based KMW compiler onto this so it can start
to benefit from the refactoring.

* Fixes a minor issue with loading a .kvks file with an empty flags set.
* Adds a writeFileSync method to the compiler callback interfaces.
* Adds an e2e compiler test to verify that .kmx and .kvk are both
  correctly compiled.
2023-05-23 15:26:28 +07:00
Marc Durdin
15142e9a4b chore(developer): cleanup 2023-05-23 11:52:37 +07:00
Marc Durdin
ac0cd831f8 refactor(developer): CompileKeyboard merge 2023-05-14 17:14:51 +07:00
Marc Durdin
e7a6d3293d refactor(developer): move file write out of WriteCompiledKeyboard
Stage one of the kmcmplib filesystem refactoring, as we move to writing
to buffers and allowing caller to write to disk. Moves responsibility
for writing data out of WriteCompiledKeyboard and into
kmcmp_CompileKeyboardFile. This means that there are now no file writes
in Compiler.cpp, except for the temp UTF16fromUTF8 (which I will also
change to a in-memory buffer "real soon now").

Note that responsibility for writing in kmcmp_CompileKeyboardFile is
very much temporary as we'll move it out of the library altogether in
a subsequent commit.
2023-05-14 16:02:18 +07:00
Keyman Server
0fb6b164ab
Merge pull request #8785 from keymanapp/auto/version-master-17.0.107
auto: increment master version to 17.0.107
2023-05-13 01:03:19 +07:00
Keyman Build Agent
db619deb66 auto: increment master version to 17.0.107 2023-05-12 14:02:41 -04:00
Steven R. Loomis
d1fb5bc205
Merge pull request #8716 from keymanapp/feat/developer-234-uset-wasm-epic-ldml
feat(developer): uset api from wasm! 🙀
2023-05-12 10:14:35 -05:00
Steven R. Loomis
e795287b93 feat(developer): kmc-kmn: merge conflict updates 🙀
for: #7234
2023-05-12 08:44:50 -05:00
Steven R. Loomis
1937552f39 Merge remote-tracking branch 'upstream/master' into feat/developer-234-uset-wasm-epic-ldml 2023-05-12 08:44:19 -05:00
Steven R. Loomis
02a3b41222
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-05-12 08:37:32 -05:00
Marc Durdin
3b86754314
Merge pull request #8726 from keymanapp/chore/developer/disambiguate-compilers
chore(developer): rename Compiler and related classes
2023-05-12 17:58:14 +10:00
Marc Durdin
2528731b58 chore(developer): fixup missing rename 2023-05-12 14:15:24 +07:00
Marc Durdin
187a25240a chore(developer): fixup missing rename 2023-05-12 14:14:32 +07:00
Marc Durdin
27aa806282 chore(developer): Merge branch 'master' into chore/developer/disambiguate-compilers 2023-05-12 14:09:50 +07:00
Marc Durdin
b3436b8aff chore(developer): fix kmc-ldml unit tests 2023-05-12 13:31:56 +07:00
Marc Durdin
3ee1132200
Merge pull request #8757 from keymanapp/feat/developer/validate-package-name
feat(developer): validate package name in compiler
2023-05-12 13:22:58 +10:00
Marc Durdin
34ba77ff2b
Merge pull request #8755 from keymanapp/feat/developer/validate-content-file-names-in-packages
feat(developer): validate content file names in packages
2023-05-12 13:22:50 +10:00
Marc Durdin
25f3a42bd5
Merge pull request #8751 from keymanapp/feat/developer/validate-package-filename
feat(developer): add validation of package filenames
2023-05-12 12:25:33 +10:00
Marc Durdin
090170ce8d
Merge pull request #8740 from keymanapp/feat/developer/package-validation-in-kmc-package
feat(developer): add package validation
2023-05-12 12:25:26 +10:00
Marc Durdin
7982bd955c
Merge pull request #8729 from keymanapp/chore/developer/move-package-formats-to-common-types
chore(developer): move package formats to common/web/types
2023-05-12 12:25:19 +10:00
Keyman Server
e183f93be1
Merge pull request #8760 from keymanapp/auto/version-master-17.0.106
auto: increment master version to 17.0.106
2023-05-12 01:02:36 +07:00
Keyman Build Agent
7a63284dd2 auto: increment master version to 17.0.106 2023-05-11 14:01:52 -04:00
Steven R. Loomis
6317583528 feat(developer): kmc-kmn: wasm and error message updates 🙀
developer:
- update wasm machinery in kmc-kmn to be more self contained
- improve exception situation in wasm functions

common:
- compilerErrorFormatCode() for formatting the raw code such as for tests
- compilerExceptionToString() for formatting exceptions in messages

for: #7234
2023-05-11 10:37:46 -05:00
Eberhard Beilharz
b759e25a05
Merge pull request #8733 from keymanapp/chore/linux/config-build.sh
chore(linux): Add build.sh for keyman-config
2023-05-11 14:50:47 +02:00
Eberhard Beilharz
032a123347
chore(linux): Properly fix shellcheck SC1091 warnings
The secret is to disable the "shellcheck.useWorkspaceRootAsCwd".
2023-05-11 11:59:15 +02:00
Eberhard Beilharz
1b76f98ec8
chore(linux): Add .shellcheckrc instead of inline directives
Also improve output specification.

Co-authored-by: Marc Durdin <marc@durdin.net>
2023-05-11 10:31:12 +02:00
Marc Durdin
70279914a8 feat(developer): validate package name in compiler 2023-05-11 15:03:53 +07:00
Marc Durdin
b2be12c258 feat(developer): validate content file names in packages 2023-05-11 14:42:26 +07:00
Darcy Wong
d796104775
Merge pull request #8749 from keymanapp/chore/android/cleanup-build-play-store-notes
chore(android/app): Cleanup build-play-store-notes script
2023-05-11 13:42:12 +07:00
Darcy Wong
840c82c209 chore(android): Revert script header 2023-05-11 13:31:30 +07:00
Darcy Wong
c169b5ca10 chore(android/app): Remove stray test code 2023-05-11 13:29:34 +07:00
Darcy Wong
e19f6fbde4
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-05-11 13:28:34 +07:00
Darcy Wong
ffc71239aa
Merge pull request #8748 from keymanapp/chore/common/crowdin-am
chore(common): Update crowdin strings for Amharic
2023-05-11 13:27:22 +07:00
Marc Durdin
88b31ba49e feat(developer): add validation of package filenames
Adds `WARN_PackageNameDoesNotFollowKeyboardConventions` and
`WARN_PackageNameDoesNotFollowLexicalModelConventions`, refactoring as
required to test filenames, and adds unit tests for same.
2023-05-11 13:23:33 +07:00
Darcy Wong
f5d43482c1 fix(android/app): Address review comments 2023-05-11 13:08:34 +07:00
Marc Durdin
a77aa8b54c chore(developer): refactor checkForModelsAndKeyboardsInSamePackage into separate func 2023-05-11 12:44:45 +07:00
Darcy Wong
aac63118ac chore(android/app): Cleanup build-play-store-notes script 2023-05-11 11:08:30 +07:00
Darcy Wong
3ea8cc26f6 chore(common): Update crowdin strings for Amharic 2023-05-11 10:10:13 +07:00
Keyman Server
25ed2b7eeb
Merge pull request #8747 from keymanapp/auto/version-master-17.0.105
auto: increment master version to 17.0.105
2023-05-11 01:03:16 +07:00
Keyman Build Agent
7b0b01b1a4 auto: increment master version to 17.0.105 2023-05-10 14:02:29 -04:00
Eberhard Beilharz
ac8517ad3a
chore(linux): Make us of new builder_run_action 2023-05-10 18:12:10 +02:00
Eberhard Beilharz
a8e0b3666e
chore(linux): Merge remote-tracking branch 'origin/master' into chore/linux/config-build.sh 2023-05-10 18:09:11 +02:00