Commit graph

105 commits

Author SHA1 Message Date
Marc Durdin
3468a36100 chore(common): add baseline.kpj
This makes it possible to run the common/test/keyboards build.sh and
makes all the baseline keyboards available as a build artifact (but
currently only in .kmx, not .kmp; this may come as a future update.)
2023-03-28 06:50:00 +07:00
Marc Durdin
c18ef54029 refactor(common): move core baseline tests into common
Decided to use the name `baseline` for this set of tests. They can be
used by any project in the repo now. Tests will be treated as shared,
and not modified (except in the case of a buggy test).

* Moves .kmn files, add `&NAME` line (don't use meson templating for
  these files)
* Generates reference .kmx files using kmcomp for use as a baseline with
  kmcmplib (kmc) tests, and for linux tests
* Updates references in core, kmcmplib, and linux builds
* Renames all the .kmn / .kmx to use a standardized, clean name format
  (this was needed for building packages in the core tests)
2023-03-28 05:20:45 +07:00
Marc Durdin
0c07a6d7a3 chore(developer): Merge branch 'feature-kmcompx' into chore/developer/master-to-kmcompx-A17S7 2023-03-02 12:37:37 +07:00
SabineSIL
048448a2a4
feat(developer): cross-platform kmx compiler 🚑 (#6980)
* get keymans version



Merge branch 'master' of https://github.com/keymanapp/keyman into refactor/developer/6026-kmcompx-compiler-cross-platform-part-2-NEW_VERSION

# Conflicts:
#	common/core/desktop/src/kmx/kmx_context.h
#	common/core/desktop/src/kmx/kmx_processor.cpp
#	common/windows/cpp/include/keymanversion.h
#	common/windows/cpp/include/kkmnkbd/Compfile.h
#	common/windows/cpp/include/kmtip_guids.h
#	common/windows/cpp/src/kmtip_guids.cpp
#	core/src/kmx/kmx_conversion.cpp
#	core/src/kmx/kmx_conversion.h
#	core/src/kmx/kmx_file.cpp
#	core/src/kmx/kmx_processevent.cpp
#	core/src/kmx/kmx_processor.hpp
#	core/src/kmx/kmx_xstring.cpp
#	developer/js/.gitignore
#	developer/js/package-lock.json
#	developer/js/tests/tsconfig.json
#	developer/src/kmcmpdll/CasedKeys.cpp
#	developer/src/kmcmpdll/CheckFilenameConsistency.cpp
#	developer/src/kmcmpdll/CheckFilenameConsistency.h
#	developer/src/kmcmpdll/CheckForDuplicates.cpp
#	developer/src/kmcmpdll/CheckForDuplicates.h
#	developer/src/kmcmpdll/CheckNCapsConsistency.cpp
#	developer/src/kmcmpdll/CheckNCapsConsistency.h
#	developer/src/kmcmpdll/Compiler.cpp
#	developer/src/kmcmpdll/DeprecationChecks.cpp
#	developer/src/kmcmpdll/NamedCodeConstants.cpp
#	developer/src/kmcmpdll/UnreachableRules.cpp
#	developer/src/kmcmpdll/UnreachableRules.h
#	developer/src/kmcmpdll/compfile.h
#	developer/src/kmcmpdll/kcframe/kcframe.cpp
#	developer/src/kmcmpdll/kcframe/kcframe.vcxproj
#	developer/src/kmcmpdll/kcframe/kcframe.vcxproj.filters
#	developer/src/kmcmpdll/kmcmpdll.vcxproj
#	developer/src/kmcmpdll/kmcmpdll.vcxproj.filters
#	developer/src/kmcmpdll/version.rc
#	developer/src/kmcmpdll/versioning.cpp
#	developer/src/kmlmc/bundle.sh
#	developer/src/kmlmc/tests/test-join-word-breaker.ts
#	developer/src/kmlmc/tests/test-override-script-defaults.ts
#	developer/src/server/tsconfig.json
#	windows/src/developer/kmcmpdll/Makefile
#	windows/src/developer/kmcmpdll/vkeys.h
#	windows/src/engine/keyman32/keymanengine.h
#	windows/src/global/inc/Vkeys.h
#	windows/src/global/inc/rc4.h
#	windows/src/global/vc/rc4.cpp

* changes in meson + include-paths of several files

* chore: remove common/core

* chore: remove unnecessary files

* chore: move headers and refs

* refactor(developer): fixup path to tests

* refactor(developer): fixup path to tests
tidied up +removed unnecessary files
checked contents of files+included a few new functions needed for myVersion
changed datatypes in CheckForDuplicates
tidy up more paths
CheckFileConsistency: not able to fully adapt for use of char16_t on non windows p.
	-> used char16_t + commented out some AddWarning().

* sorted out some #includes
renamed some functions to reduce ambiguity

* fix(developer): solved linker errors

* chore(developer): remove some comments

* chore(developer): introduced KMX_UCHAR

* chore(developer): restored whitespace, comments

* chore(developer): tidied up vkeys.H / .cpp

* chore(developer): remove keyboardprocessor...

* chore(common): refactor shared types into km_types.h

* chore(core): remove duplicate types

* chore(core): move kmx_file.h to common

* chore(developper): removed typedefs of kmcompcx.h

* chore(developer): removed kmcompx/include/kmx_file.h

* chore(developer): chanched include of pch.h

* chore(developer): removed files from kmcompx/include/

* chore(developer): removed commented sections ( ** old **) if files

* chore(developer): removed doubled/overwritten functions

* chore(developer): removed doubled/overwritten functions

* chore(developer): changed #includes ".." -> <..>

* chore(developer): changed #includes ".." -> <..>

* chore(developer): enable tests for keyboards repo for kmcompx

* chore(developer): new functions e.g. string_from_wstring...

* chore(developer): changed printf->u16sprintf

* chore(developer): fopen

* chore(developer): fixed errors on running kmcompxtest

* chore(developer): changef u16fmt

* chore(developer): fixup u16ncat

* chore(developer): added return Msgs for different types of errors while compiling

* chore(developer): tests using RTL run Ok ( changed RTu->RTL)

* chore(developer): replaced u16ncmp with u16nicmp at most places

* chore(developer): changed Version number for tests 16.046 to 15.0265

* chore(developer): mostly whitespace changes

* chore(developer): tidied up code (not finished yet)

* chore(developer): removed #pragma once from wrong places, edited #includes

* chore(developer): removed #pragma once from wrong places, edited #includes

* chore(developer): CheckFileNameConsistency changed to take KMX_WCHART const* (= wchar_t* const); used in compiler.cpp now

* chore(developer): excluded some tests from kmcompxtest (use get_test_source_exclude_some_kmn.bat now)

* chore(developer): Win-functions in FileNameConsistency

* chore(developer): Win-functions in FileNameConsistency

* chore(developer): CheckFNConsuistency: called u16printf with f´different par.

* chore(developer): tidied up code

* chore(developer): tidied up code

* chore(developer): tiedied up code (compiler.cpp)

* chore(developer): tiedied up code

* chore(developer): tidied up code

* chore(developer): removed all commentaries _S2

* chore(developer): tidied up code

* chore(developer): tidied up code

* chore(developer): tidied up code

* chore(developer): removed several Windows-specific functions

* chore(developer): removed unused datatype typedefs

* chore(developer): new GetCompilerErrorString() used in AddCompileMessage()

* chore(developer): new GetCompilerErrorString() used in AddCompileMessage()

* chore(developer): removed CWARN/CHINT out of CompMsg

* chore(developer): adapted kmcompxtest to test kmn-files that have errors included

* chore(developer): changed gitignore

* chore(developer): changed gitignore

* chore(developer): changed Version Number fot Tests 15.0.265.0 -> 15.0.270.0

* chore(developer): added tests for kmcompx

* chore(developer): move tests (for ErrorMessages) to common/test/keyboards

* chore(developer): move CERR-tests into test/keyboards/invalid

* chore(developer): change mostly whitespace changes

* chore(developer): change mostly whitespace changes

* chore(developer): delete files of keyboards/kmcompx_tests

* chore(developer): include check for correct Filename + rename var + change whitespace changes

* chore(developer): revert changes in AddCompileMessage, tidy up CreateTempFile

* chore(developer): remove python from meson.build

* chore(developer): change ErrExtra, change CompileMsg, change _finddata in FileExists

* chore(developer): check for '_' after Errornumber in Filename

* chore(developer): re-include AddCompilerVersionStore in Compiler.cpp

* chore(developer): add Message CERR_InvalidCharacter to UTF16TempFromUTF8() ; change #include <> -> ""

* chore(developer): add some more keyboardtests

* chore(developer): add some more keyboardtests

* chore(developer): change remaining old datatype-names to new names ( e.g. char -> KMX_CHAR )

* chore(developer): edit + add CERR_tests

* chore(developer): edit + add CERR_tests

* chore(developer): use char16_t in CheckFileNameConsistency

* chore(developer): comments: converting functions in kmx_u16

* chore: reorganize

* chore: commit kmcompx-files

* chore: use ifdef for vkeys

* chore: copy files back

* chore: more cleanup

* chore: meson build for kmcmplib

* chore: tweak unit test, error messages, and paths

* chore(developer): make kmcmplib a static library

* chore(developer): start kmcmpdll link to kmcmplib

* chore(developer): add build.sh for kmcmplib

* chore(developer): link kmcmpdll to kmcmplib

* chore(common): avoid building 'invalid' shared keyboards

* chore(developer): include BEGIN_LINE (repeated Begin)

* chore(developer): add more CERR_tests and move some to other folder, edit readme

* chore(developer): addCompilerMsg for RepeatedBegin

* chore(developer):repeatedBEgin plus rename functions to prevent same names in kmcmpdll and kmcmplib

* chore(developer): add flag flag_use_kmcompx as switch to use old code (kmcmpdll) or new code kmcmplib

* chore(developer): introduce namespace kmcmp instead of different filenames

* chore(developer): more changes for using namespaces

* chore(developer): solve merge conflicts in namespaces

* chore(developer): use namespaces in kmcmplib (except for in dllexport-functions)

* chore(developer): remove some unnecessary CERR_ tests

* chore(developer): remove CERR_CallIsProfessionalFeature, CERR_IncludeCodesIsProfessionalFeature, CERR_MnemonicLayoutIsProfessionalFeature

* chore(developer): note about func.call_js; change all "VESION x.y " to "VESION x.y or higher" ; rename flag for kmcompx to flag_use_new_kmcomp

* chore(developer): use PKMX_DWORD/PKMX_WCHAR   instead of LPKMX_DWORD/LPKMX_WCHAR; use PKMX_STR instead of char* in CompilerMessageProc

* chore(developer): remove kmx-file; NamedCodeConstants: rename variables, set const variables for buf sizes, brackets in for-loops

* chore(developer): change include paths ../

* chore(developer): get rid of reopened namespaces of kmcmp

* chore(developer): restored kmcompxtest

* chore(developer): change #include paths ../ ; excange km_kbp_cp with KMX_WCHAR; change some variable names

* chore(developer): use environment var KEYMAN_ROOT in path for tests; remove try-catch from json-validation.cpp

* chore(developer): use environment var KEYMAN_ROOT in path for tests; remove try-catch from json-validation.cpp

* chore(developer): revert change in NamedCodeConstants

* chore(developer): restored kmcompxtest, adapted readme

* chore(developer): use res-fils/version.rc for kmcmplib

* chore(developer): rename some functions/variables

* chore(developer): typos, comments, add function u16rchr()

* chore(developer): typos, comments, add function u16rchr()

* chore(developer): rename Functions, version rc-files

* chore(developer): rename Functions, version rc-files, set flag_use_new_kmcomp to use (old) kmcmpdll

* chore(developer): add BeginLine in kmcmpdll/Compiler.cpp; fread() in kmcmp_CompileKeyboardFileToBuffer

* chore(developer): set flag_use_new_kmcomp to only use kmcmpdll

* chore(developer): set flag_use_new_kmcomp to only use kmcmpdll...and back to kmcmplib

* (chore(developer): set flag_use_new_kmcomp to only use kmcmpdll...and back

* (chore(developer): changes in NamedCodeConstants::reindex() to run hieroglyphic.kmn properly

* chore(developer): changes in UnrechableRules::VerifyUnreachableRules() to run test_6440_unreachable_code..kmn

* chore(developer): remove unneccessary files, get ErrorMsg for INCLUDECODES, VISUALKEYBOARD, UnreachableRules, ErrExrta

* chore(developer): remove unneccessary files, get ErrorMsg for INCLUDECODES, VISUALKEYBOARD, UnreachableRules, ErrExrta, CWARN_ OK

* chore(developer): after stash:remove unneccessary files, get ErrorMsg for INCLUDECODES, VISUALKEYBOARD, UnreachableRules, ErrExrta, CWARN_ OK

* chore(developer): replace some #include <> with "" ; rename ErrExtra -> ErrExtraW

* chore(developer): replace some #include <> with "" ; rename ErrExtra -> ErrExtraW

* chore(developer): new converting function convert_pchar16T_To_pwcharT in kmx_u16

* chore(developer): CheckFilenameConsistency added fclose()

* chore(developer): --

* chore(developer): removed printf (---> started in... / °°-> changed to...)

* chore(developer): AddDeprecatedCode to get CWARN when in kmcomp.exe -s-w is set

* chore(developer): change MakeHashKeyFromFileKey16() to get correct behaviour of CHINT

* chore(developer): change MakeHashKeyFromFileKey16() to get correct behaviour of CHINT

* chore(developer): move converting function to kmx_u16

* chore(developer): add more CERR_ tests

* chore(developer): Not ready yet: changes in CheckFilenameConsistency to use on non-windows

* chore(developer): Not ready yet: changes in CheckFilenameConsistency to use on non-windows

* chore(developer): copied contents of 3 files of commit fcfc6c698a to here to achieve state of fcfc6c698a

* chore(developer): copied contents of 3 files of commit fcfc6c698a to here to achieve state of fcfc6c698a

* chore(developer): move/rename converting functions

* chore(developer) : update .gitignore, remove converting function

* chore(developer) : update gitignore; sort code in CheckFilenameConsistency::CheckFilenameConsistency()

* chore(developer) : change Version number 15.0.270 -> 17.0.51

* chore(developer): change #includes, path in meson.build

* chore(developer): new function Open_File() to open files on windows and non-windows platforms

* chore(developer): create Function to replace fopen/_wfsopen

* chore(developer): exclude kmx, kvk from git

* chore(developer): exclude kmx, kvk from git

* chore(developer): change #includes to include from kmcmplib

* chore(developer): remove stray fopen

* chore(developer): include Open_File() (as comment for now)

* chore(developer): include Open_File() (as comment for now)

* chore(developer): remove path to kmcmpdll in include_directories

* chore(developer): use functin Open_File()

* chore(developer): remove #ifdef for '\\' and '/' in both IsRelativePath; remove Function Open_File again

* chore(developer): new fun  u16rchr_LinWin, strrchr_LinWin to exchange #ifndef for '\\' and '/'

* chore(developer): implement u16rchr_LinWin and strrchr_LinWin

* chore(developer): implement Open_File()

* chore(developer): new MakeHashKeyFromFileKey, remove comments for Open_File, & *LinWin-functions, delete 1 converting function

* chore(developer): tidy up comments

---------

Co-authored-by: Marc Durdin <marc@durdin.net>
2023-03-01 20:00:04 +11:00
Marc Durdin
ce864ea923 chore: update standard script prolog to use builtin instead of dirname 2023-02-22 11:04:15 +07:00
Marc Durdin
463c7ff907 chore: remove greadlink from standard script prolog 2023-02-22 10:43:06 +07:00
Marc Durdin
a004978863 chore: use builder_heading, remove echo_heading 2023-02-20 09:17:51 +07:00
Marc Durdin
8f5997a3e6 chore: rename os_id to BUILDER_OS 2023-02-20 08:44:03 +07:00
Joshua A. Horton
f4591af1bc chore(web): right, stash + pop drops the exec bit too 2022-10-26 08:46:32 +07:00
Joshua A. Horton
5d411148ce chore(web): moves browserstack-termination script to common test resources 2022-10-26 08:44:48 +07:00
Marc Durdin
1724778e77 chore: try disabling concurrency for browserstack tests
We're still getting sporadic test failures with BrowserStack. So per one
discussion online, attempting to run without concurrency to see if it
improves stability.

I note that the video from tests where we "fail to capture" shows a
browser with an error: "Safari Can't Connect to the Server" "Safari
can't open the page "localhost:9879/?id=75185158" because Safari can't
connect to the server "localhost".

1971215883
2022-09-10 06:23:26 +10:00
Joshua A. Horton
bac2578039 chore(web): addresses PR review comment 2022-08-02 08:10:01 +07:00
Joshua A. Horton
205312c451 chore(web): timeout-adapter implementation 2022-08-01 13:06:15 +07:00
Joshua A. Horton
b42a87ee5b chore(web): updates target devices for BrowserStack-based CI tests 2022-08-01 12:37:19 +07:00
Joshua A. Horton
5a7840753f chore(web): centralizes test-start timeout config logic 2022-08-01 11:39:14 +07:00
Joshua A. Horton
405a553c06 chore(web): requireable module for common CI test-config 2022-08-01 10:54:57 +07:00
Marc Durdin
4f36c3c2df chore(developer): Merge branch 'master' into chore/developer/js-to-lmc 2022-06-25 05:48:57 +10:00
Marc Durdin
d5aa61fdc0 chore(common): Merge branch 'chore/web/keyman-version-for-developer' into chore/common/move-common-core-desktop-to-core 2022-05-30 09:38:27 +10:00
Marc Durdin
fcf78bcd94 chore: merge master into branch 2022-05-27 13:11:23 +10:00
Marc Durdin
91b5f2e640 chore(developer): fixup kmcomp ref in keyboards build 2022-05-23 13:58:53 +10:00
Marc Durdin
4ab3c53810 chore(common): adjust scripts for core relocation 2022-05-18 07:22:11 +10:00
Marc Durdin
dfd5fad2c7 chore(common): move common/core/web to common/web and flatten 2022-05-17 11:15:22 +10:00
Marc Durdin
833dde0ae4 chore: remove unnecessary package-lock.json files 2022-05-16 14:47:58 +10:00
Marc Durdin
0fbf53f8d7 chore(web): move version from package.json to keyman-version package
Starts to resolve the chaos of putting version numbers into every
package.json file.

Adds a new keyman-version package which mimics the
version information in build-utils.sh. This will eventually replace
resources/web-environment.

Fixes the build for the lexical model compiler tools to remove refs to
package.json version and use keyman-version instead.

More fixes coming in a follow-up.
2022-04-17 06:47:39 +10:00
Joshua A. Horton
53cd3b1b71 change(web): re-renaming the test kbd 2022-04-13 18:32:10 -05:00
Joshua A. Horton
f16f1185de fix(web): kmn typo 2022-04-12 16:31:47 -05:00
Joshua A. Horton
5e52ed4d2d change(web): renamed+moved 'vowel hater' test kbd 2022-04-12 08:30:09 -05:00
Darcy Wong
b9f6ac34e2 test(common/resources): Add Noto font to web_context_tests keyboard 2022-04-07 08:04:42 +07:00
Marc Durdin
d63bd4edde chore(web): rebuild obolo_chwerty keyboard 2022-03-17 20:47:00 +11:00
Marc Durdin
c80140d618 chore(common): update obolo_chwerty_6347 for new store names 2022-03-17 05:32:50 +11:00
Marc Durdin
7a5d1b964f chore(web): Merge branch 'beta' into feat/web/6364-new-layer-old-layer-replaces-layerchanged 2022-03-17 04:55:25 +11:00
Marc Durdin
c74c8db6fe fix(common): obolo_chwerty package had broken link 2022-03-17 04:55:05 +11:00
Marc Durdin
1038379c41
Merge pull request #6351 from keymanapp/fix/web/caps-state-tracking-for-touch
fix(web): caps state tracking for touch
2022-03-15 10:04:45 +11:00
Marc Durdin
81ec8b86e2
Merge pull request #6347 from keymanapp/fix/developer/warn-on-ncaps-consistency
fix(developer): warn on ncaps usage inconsistency
2022-03-15 10:02:35 +11:00
Marc Durdin
2fc52af719 feat(common): zip and index keyboard artifacts 2022-03-14 07:03:16 +11:00
Marc Durdin
c2b073a49b chore: fixup additional layerchanged in keyboard 2022-03-13 19:53:14 +11:00
Marc Durdin
b23df3fbc8 feat(web): &newLayer and &oldLayer
Fixes #6364.

Removes `&layerChanged` and adds `&newLayer` and `&oldLayer` system
stores with the additional nuances around values as described in #6364.
2022-03-13 08:29:15 +11:00
Marc Durdin
faff0e2aef fix(common): address review comment
Don't specify wine as kmcomp launcher if a custom kmcomp is provided on
the command line.
2022-03-11 05:42:08 +11:00
Marc Durdin
c177f142d3 fix(common): allow custom keyboard builds
This allows for custom builds of keyboards, e.g. where a keyboard is
manually constructed for testing purposes.
2022-03-11 05:23:37 +11:00
Darcy Wong
193f7bd370 chore(common): Update caps_3620_3621 to fileVersion 15.0 2022-03-10 14:03:29 +07:00
Darcy Wong
b213c3b829 fix(android/engine): Test fileVersion during package install 2022-03-10 13:15:59 +07:00
Marc Durdin
9cabb7cd66 fix(web): add obolo_chwerty_6351 test keyboard 2022-03-10 13:31:05 +11:00
Marc Durdin
0fb2a371a3 feat(common): build test keyboards via script
Relates to #5815.

build.sh is a new script that builds any or all of the keyboards in the
common/test/keyboards folder. build.bat is a wrapper for build.sh, just
for Windows CMD.

Assumes that kmcomp.exe has already been built or allows you to specify
your own kmcomp.exe if necessary.
2022-03-10 08:40:55 +11:00
Marc Durdin
6c04a11ef3 fix(developer): add obolo_chwerty_6347 for testing 2022-03-10 08:40:12 +11:00
Marc Durdin
7273028192 chore(web): add unit test for selection and matching rule 2022-02-25 15:06:11 +11:00
Marc Durdin
bf2270aa69 chore(web): add smp tests for text selection 2022-02-24 15:59:16 +11:00
Marc Durdin
6cd0f5d49a chore: add web_context_tests keyboard 2022-02-22 10:41:42 +11:00
Marc Durdin
2159df10ee chore: tweak keyboard rules for full caps
Removed the 'heuristic' test as it is probably not appropriate for
testing. Removed the mobile-specific platform from the touch layout as
it did not include a caps layer (and should have been identical to the
tablet one anyway).
2021-12-02 05:11:08 +11:00
Marc Durdin
88ea17e0ba feat(developer): support for Caps Lock layer 🍆
Relates to #3620.

Allows keyboard developers to add a Caps Lock layer (called 'caps'),
which will be accessible through double-tap of Shift key, in an upcoming
PR.

Also adds test keyboards to exercise the functionality when it is
available.
2021-12-01 08:53:23 +11:00
Marc Durdin
83eb8395d6 chore(web): cleanup sample test 3621 keyboard 2021-11-29 10:54:46 +11:00