Commit graph

1756 commits

Author SHA1 Message Date
Marc Durdin
62b41d385c
Merge pull request #8856 from keymanapp/chore/developer/remove-seterror
chore(developer): remove SetError macro
2023-05-26 18:43:33 +10:00
Marc Durdin
18c4ac5bb2
Merge pull request #8850 from keymanapp/refactor/developer/wasm-compiler-interface
refactor(developer): compiler interface part 1 for wasm
2023-05-26 18:43:24 +10:00
Marc Durdin
d0083e984d
Merge pull request #8796 from keymanapp/refactor/developer/kmcmplib-fs-agnostic
refactor(developer): move file write out of WriteCompiledKeyboard and CompileKeyboard merge
2023-05-25 14:06:51 +10:00
Marc Durdin
50186e986e chore(developer): remove SetError macro
The `SetError` macro emitted `return FALSE;` which was confusing for
reviewing code flow. Replaced with explicit `AddCompileMessage()` and
`return FALSE;`.
2023-05-25 09:03:22 +07:00
Marc Durdin
ef101d746c chore(developer): address review comments 2023-05-25 08:24:09 +07:00
Marc Durdin
cdf5f7a7c4 chore(developer): consolidate 'extra' field for FILE_KEYBOARD
FILE_KEYBOARD is a structure used in multiple libraries and languages.
It is passed around between kmcmplib, kmcmpdll, and kmcomp, with three
separate definitions (kmcmplib/compfile.h, kmcmpdll/compfile.h, and
kmcomp/compile.pas). These duplicate definitions must be maintained for
now because of type issues between the legacy kmcmpdll and kmcmplib,
which means that the definitions must be kept in sync.

Because FILE_KEYBOARD_EXTRA has a C++ class in it, it will be
initialized in CompileKeyboardHandle rather than in the caller, so
FILE_KEYBOARD.extra is now a pointer rather than a struct.

When original kmcomp and kmcmpdll are eventually removed, these
duplications will disappear and some of the memory management and
lifecycle will become easier to manage.
2023-05-24 12:14:41 +07:00
Marc Durdin
d1d6dc5d99 refactor(developer): compiler interface part 1 for wasm
This starts the refactor of the .kmn compiler interfaces for WASM,
moving more file writes into typescript and out of the compiler. Adds a
cleaner binding and structures for passing compile metadata back to
typescript for additional build steps (e.g. kvk, js builds).

Will rebase the Typescript-based KMW compiler onto this so it can start
to benefit from the refactoring.

* Fixes a minor issue with loading a .kvks file with an empty flags set.
* Adds a writeFileSync method to the compiler callback interfaces.
* Adds an e2e compiler test to verify that .kmx and .kvk are both
  correctly compiled.
2023-05-23 15:26:28 +07:00
Marc Durdin
15142e9a4b chore(developer): cleanup 2023-05-23 11:52:37 +07:00
Marc Durdin
b6173fda37 fix(developer): 'no keyboard version' no longer emitted
The `Info_KeyboardFileHasNoKeyboardVersion` message is now only emitted
if the package uses `FollowKeyboardVersion`, which broke a unit test in
kmc.
2023-05-23 11:39:44 +07:00
Marc Durdin
e293d7608c chore(developer): add test for missing .js file 2023-05-23 09:28:26 +07:00
Marc Durdin
3828a5c16b refactor(developer): move message tests into test-messages.ts 2023-05-23 08:31:29 +07:00
Ross
1d3fc99b28 chore(windows): Merge branch 'master' into chore/windows/5442/remove-legacy-keyman-core 2023-05-17 10:27:56 +10:00
Marc Durdin
4ad975d3a8
Merge pull request #8793 from keymanapp/chore/developer/verify-package-has-a-model-or-keyboard
chore(developer): verify that package has at least a model or keyboard
2023-05-16 17:21:41 +10:00
Marc Durdin
ced54e260a
Merge pull request #8792 from keymanapp/chore/developer/verify-included-files-in-package
chore(developer): verify file types of content files in package
2023-05-16 17:21:24 +10:00
Marc Durdin
1be22862cf
Merge pull request #8783 from keymanapp/feat/developer/verify-at-least-one-language-in-package
feat(developer): verify at least one language in package
2023-05-16 17:21:01 +10:00
Marc Durdin
888786e0f4
Merge pull request #8778 from keymanapp/feat/developer/verify-package-lang-tags
feat(developer): verify bcp47 tags are valid and minimal in kmc-package
2023-05-16 17:20:46 +10:00
Marc Durdin
bd0db9c7da
Merge pull request #8769 from keymanapp/feat/developer/verify-keyboard-versions
feat(developer): verify keyboard versions in kmc-package
2023-05-16 17:20:26 +10:00
Marc Durdin
b2e293892d
Update developer/src/kmc-package/test/fixtures/invalid/keyboardcontentfilenotfound.kps 2023-05-16 13:21:31 +10:00
Marc Durdin
c31118cd6d chore(developer): rename fixture 2023-05-15 20:17:22 +07:00
Marc Durdin
663e4195ff
chore: Apply suggestions from code review 2023-05-15 23:16:36 +10:00
Marc Durdin
ac0cd831f8 refactor(developer): CompileKeyboard merge 2023-05-14 17:14:51 +07:00
Marc Durdin
e7a6d3293d refactor(developer): move file write out of WriteCompiledKeyboard
Stage one of the kmcmplib filesystem refactoring, as we move to writing
to buffers and allowing caller to write to disk. Moves responsibility
for writing data out of WriteCompiledKeyboard and into
kmcmp_CompileKeyboardFile. This means that there are now no file writes
in Compiler.cpp, except for the temp UTF16fromUTF8 (which I will also
change to a in-memory buffer "real soon now").

Note that responsibility for writing in kmcmp_CompileKeyboardFile is
very much temporary as we'll move it out of the library altogether in
a subsequent commit.
2023-05-14 16:02:18 +07:00
Marc Durdin
135d6f9c0f
chore(developer): rename misnamed error message 2023-05-14 17:23:21 +10:00
Marc Durdin
7f1c05cba4
chore: rename Error_PackageMustContainAModelOrAKeyboard
Co-authored-by: Darcy Wong <darcy_wong@sil.org>
2023-05-14 17:18:59 +10:00
Marc Durdin
abd174557c chore(developer): verify that package has at least a model or keyboard
Adds `ERROR_PackageMustContainAPackageOrAKeyboard` and corresponding
unit tests. A couple of unit tests tweaked as their fixtures were no
longer valid!
2023-05-14 09:53:36 +07:00
Marc Durdin
3c9492dd56 chore(developer): verify file types of content files in package
This is a transfer of the functionality in the legacy package compiler;
we could go much further in verifying file types and excluding certain
files, but that's a big design session. For now, just refreshed to
include the set of Keyman for Windows and Keyman Engine for Windows
files which are most likely to be accidentally included.

Adds WARN_RedistFileShouldNotBeInPackage and WARN_DocFileDangerous and
corresponding unit tests and constant declarations.
2023-05-14 09:31:32 +07:00
Steven R. Loomis
e795287b93 feat(developer): kmc-kmn: merge conflict updates 🙀
for: #7234
2023-05-12 08:44:50 -05:00
Steven R. Loomis
1937552f39 Merge remote-tracking branch 'upstream/master' into feat/developer-234-uset-wasm-epic-ldml 2023-05-12 08:44:19 -05:00
Steven R. Loomis
02a3b41222
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-05-12 08:37:32 -05:00
Marc Durdin
8446775dd5 feat(developer): verify at least one language in package
Adds `ERROR_MustHaveAtLeastOneLanguage` and corresponding unit test.
2023-05-12 14:42:07 +07:00
Marc Durdin
2528731b58 chore(developer): fixup missing rename 2023-05-12 14:15:24 +07:00
Marc Durdin
187a25240a chore(developer): fixup missing rename 2023-05-12 14:14:32 +07:00
Marc Durdin
27aa806282 chore(developer): Merge branch 'master' into chore/developer/disambiguate-compilers 2023-05-12 14:09:50 +07:00
Marc Durdin
b3436b8aff chore(developer): fix kmc-ldml unit tests 2023-05-12 13:31:56 +07:00
Marc Durdin
e4ad95550e feat(developer): verify bcp47 tags are valid and minimal in kmc-package
Adds checks for bcp47 tag metadata for keyboards and lexical models --
both validity and minimality. Adds `ERROR_LanguageTagIsNotValid` and
`WARN_LanguageTagIsNotMinimal` messages and corresponding unit tests.

This was implemented by extending the existing duplicate id check, so
renamed that function accordingly.
2023-05-12 13:21:13 +07:00
Marc Durdin
3ee1132200
Merge pull request #8757 from keymanapp/feat/developer/validate-package-name
feat(developer): validate package name in compiler
2023-05-12 13:22:58 +10:00
Marc Durdin
34ba77ff2b
Merge pull request #8755 from keymanapp/feat/developer/validate-content-file-names-in-packages
feat(developer): validate content file names in packages
2023-05-12 13:22:50 +10:00
Marc Durdin
25f3a42bd5
Merge pull request #8751 from keymanapp/feat/developer/validate-package-filename
feat(developer): add validation of package filenames
2023-05-12 12:25:33 +10:00
Marc Durdin
090170ce8d
Merge pull request #8740 from keymanapp/feat/developer/package-validation-in-kmc-package
feat(developer): add package validation
2023-05-12 12:25:26 +10:00
Marc Durdin
7982bd955c
Merge pull request #8729 from keymanapp/chore/developer/move-package-formats-to-common-types
chore(developer): move package formats to common/web/types
2023-05-12 12:25:19 +10:00
Marc Durdin
d3e4762b52 chore(developer): Info_KeyboardFileHasNoKeyboardVersion
Downgrades `Warn_KeyboardFileHasNoKeyboardVersion` to
`Info_KeyboardFileHasNoKeyboardVersion`, because this is not an error,
or even something wrong necessarily; it's just something it's good to be
aware of. Updates unit test for kmc to cater for the extra message.
2023-05-12 09:07:46 +07:00
Marc Durdin
53c51b79e6 feat(developer): add tests for keyboard and package version matching
Adds `WARN_KeyboardVersionsDoNotMatch` and
`WARN_KeyboardVersionsDoNotMatchPackageVersion`.

Adds unit tests. Several other 'invalid' packages needed corrections
after adding this validation. Note that one unit test was deleted
because it could never be satisfied after adding this validation step.
2023-05-12 08:26:54 +07:00
Marc Durdin
3c3a363478 feat(developer): verify keyboard versions in kmc-package
Refactor the `extractKeyboardVersionFromKmx` function into a new class
and add extra version checking. This caused a bit of a cascade of test
failures due to some of the 'invalid' fixtures actually being invalid
in multiple ways, so the fixtures have been corrected to only be wrong
in a single way -- the way they are supposed to be broken. This means
additional fixture files in the 'invalid' folder.

Now updates keyboard version metadata for all keyboards in the package.

Upgrades `WARN_FollowKeyboardVersionButNoKeyboards` to
`ERROR_FollowKeyboardVersionButNoKeyboards`, as this leads to invalid
package metadata on build.

Renames `ERROR_KeyboardFileNotFound` to
`ERROR_KeyboardContentFileNotFound` to better reflect that no `<File>`
in the package is found to match a given `<Keyboard>` entry.

Adds `ERROR_KeyboardFileNotFound` when a referenced .kmx does not exist.

Adds and updates corresponding unit tests.
2023-05-12 07:19:33 +07:00
Steven R. Loomis
6317583528 feat(developer): kmc-kmn: wasm and error message updates 🙀
developer:
- update wasm machinery in kmc-kmn to be more self contained
- improve exception situation in wasm functions

common:
- compilerErrorFormatCode() for formatting the raw code such as for tests
- compilerExceptionToString() for formatting exceptions in messages

for: #7234
2023-05-11 10:37:46 -05:00
Marc Durdin
70279914a8 feat(developer): validate package name in compiler 2023-05-11 15:03:53 +07:00
Marc Durdin
b2be12c258 feat(developer): validate content file names in packages 2023-05-11 14:42:26 +07:00
Marc Durdin
88b31ba49e feat(developer): add validation of package filenames
Adds `WARN_PackageNameDoesNotFollowKeyboardConventions` and
`WARN_PackageNameDoesNotFollowLexicalModelConventions`, refactoring as
required to test filenames, and adds unit tests for same.
2023-05-11 13:23:33 +07:00
Marc Durdin
a77aa8b54c chore(developer): refactor checkForModelsAndKeyboardsInSamePackage into separate func 2023-05-11 12:44:45 +07:00
Ross
1f3e8f768d chore(windows): remove RefreshPreservedKeys pt2 2023-05-11 14:57:01 +10:00
Steven R. Loomis
3d8e3e2500 Merge remote-tracking branch 'upstream/master' into feat/developer-234-uset-wasm-epic-ldml 2023-05-10 08:34:59 -05:00