Commit graph

39 commits

Author SHA1 Message Date
Marc Durdin
4838ba1ffe chore: rename kmc-keyboard to kmc-ldml
Part of #8719.
2023-05-09 14:08:45 +07:00
Marc Durdin
5d52932e1c chore(developer): further filesystem cleanup in kmcmplib
* fixes CheckFilenameConsistency to use Windows functions on Windows,
  to check case differences, and only test for file existence on other
  platforms which are already case sensitive.
* fixes memory leaks in usage of `wstrtostr2`, and eliminated the
  function altogether.
* avoided use of u16sprintf for error messages, as it seems not to be
  working correctly (long term, eliminate altogether, use C++ string
  formatting).
* fix memory leak and overwrite of token in compiler error message.
* remove debug logging and namespace from filesystem.cpp

Unrelated to kmcmplib:
* improved logs in kmcomp tests (these tests are running on kmcmplib),
  so it is easier to diagnose failing tests.
2023-03-09 05:39:39 +07:00
Marc Durdin
0c07a6d7a3 chore(developer): Merge branch 'feature-kmcompx' into chore/developer/master-to-kmcompx-A17S7 2023-03-02 12:37:37 +07:00
SabineSIL
048448a2a4
feat(developer): cross-platform kmx compiler 🚑 (#6980)
* get keymans version



Merge branch 'master' of https://github.com/keymanapp/keyman into refactor/developer/6026-kmcompx-compiler-cross-platform-part-2-NEW_VERSION

# Conflicts:
#	common/core/desktop/src/kmx/kmx_context.h
#	common/core/desktop/src/kmx/kmx_processor.cpp
#	common/windows/cpp/include/keymanversion.h
#	common/windows/cpp/include/kkmnkbd/Compfile.h
#	common/windows/cpp/include/kmtip_guids.h
#	common/windows/cpp/src/kmtip_guids.cpp
#	core/src/kmx/kmx_conversion.cpp
#	core/src/kmx/kmx_conversion.h
#	core/src/kmx/kmx_file.cpp
#	core/src/kmx/kmx_processevent.cpp
#	core/src/kmx/kmx_processor.hpp
#	core/src/kmx/kmx_xstring.cpp
#	developer/js/.gitignore
#	developer/js/package-lock.json
#	developer/js/tests/tsconfig.json
#	developer/src/kmcmpdll/CasedKeys.cpp
#	developer/src/kmcmpdll/CheckFilenameConsistency.cpp
#	developer/src/kmcmpdll/CheckFilenameConsistency.h
#	developer/src/kmcmpdll/CheckForDuplicates.cpp
#	developer/src/kmcmpdll/CheckForDuplicates.h
#	developer/src/kmcmpdll/CheckNCapsConsistency.cpp
#	developer/src/kmcmpdll/CheckNCapsConsistency.h
#	developer/src/kmcmpdll/Compiler.cpp
#	developer/src/kmcmpdll/DeprecationChecks.cpp
#	developer/src/kmcmpdll/NamedCodeConstants.cpp
#	developer/src/kmcmpdll/UnreachableRules.cpp
#	developer/src/kmcmpdll/UnreachableRules.h
#	developer/src/kmcmpdll/compfile.h
#	developer/src/kmcmpdll/kcframe/kcframe.cpp
#	developer/src/kmcmpdll/kcframe/kcframe.vcxproj
#	developer/src/kmcmpdll/kcframe/kcframe.vcxproj.filters
#	developer/src/kmcmpdll/kmcmpdll.vcxproj
#	developer/src/kmcmpdll/kmcmpdll.vcxproj.filters
#	developer/src/kmcmpdll/version.rc
#	developer/src/kmcmpdll/versioning.cpp
#	developer/src/kmlmc/bundle.sh
#	developer/src/kmlmc/tests/test-join-word-breaker.ts
#	developer/src/kmlmc/tests/test-override-script-defaults.ts
#	developer/src/server/tsconfig.json
#	windows/src/developer/kmcmpdll/Makefile
#	windows/src/developer/kmcmpdll/vkeys.h
#	windows/src/engine/keyman32/keymanengine.h
#	windows/src/global/inc/Vkeys.h
#	windows/src/global/inc/rc4.h
#	windows/src/global/vc/rc4.cpp

* changes in meson + include-paths of several files

* chore: remove common/core

* chore: remove unnecessary files

* chore: move headers and refs

* refactor(developer): fixup path to tests

* refactor(developer): fixup path to tests
tidied up +removed unnecessary files
checked contents of files+included a few new functions needed for myVersion
changed datatypes in CheckForDuplicates
tidy up more paths
CheckFileConsistency: not able to fully adapt for use of char16_t on non windows p.
	-> used char16_t + commented out some AddWarning().

* sorted out some #includes
renamed some functions to reduce ambiguity

* fix(developer): solved linker errors

* chore(developer): remove some comments

* chore(developer): introduced KMX_UCHAR

* chore(developer): restored whitespace, comments

* chore(developer): tidied up vkeys.H / .cpp

* chore(developer): remove keyboardprocessor...

* chore(common): refactor shared types into km_types.h

* chore(core): remove duplicate types

* chore(core): move kmx_file.h to common

* chore(developper): removed typedefs of kmcompcx.h

* chore(developer): removed kmcompx/include/kmx_file.h

* chore(developer): chanched include of pch.h

* chore(developer): removed files from kmcompx/include/

* chore(developer): removed commented sections ( ** old **) if files

* chore(developer): removed doubled/overwritten functions

* chore(developer): removed doubled/overwritten functions

* chore(developer): changed #includes ".." -> <..>

* chore(developer): changed #includes ".." -> <..>

* chore(developer): enable tests for keyboards repo for kmcompx

* chore(developer): new functions e.g. string_from_wstring...

* chore(developer): changed printf->u16sprintf

* chore(developer): fopen

* chore(developer): fixed errors on running kmcompxtest

* chore(developer): changef u16fmt

* chore(developer): fixup u16ncat

* chore(developer): added return Msgs for different types of errors while compiling

* chore(developer): tests using RTL run Ok ( changed RTu->RTL)

* chore(developer): replaced u16ncmp with u16nicmp at most places

* chore(developer): changed Version number for tests 16.046 to 15.0265

* chore(developer): mostly whitespace changes

* chore(developer): tidied up code (not finished yet)

* chore(developer): removed #pragma once from wrong places, edited #includes

* chore(developer): removed #pragma once from wrong places, edited #includes

* chore(developer): CheckFileNameConsistency changed to take KMX_WCHART const* (= wchar_t* const); used in compiler.cpp now

* chore(developer): excluded some tests from kmcompxtest (use get_test_source_exclude_some_kmn.bat now)

* chore(developer): Win-functions in FileNameConsistency

* chore(developer): Win-functions in FileNameConsistency

* chore(developer): CheckFNConsuistency: called u16printf with f´different par.

* chore(developer): tidied up code

* chore(developer): tidied up code

* chore(developer): tiedied up code (compiler.cpp)

* chore(developer): tiedied up code

* chore(developer): tidied up code

* chore(developer): removed all commentaries _S2

* chore(developer): tidied up code

* chore(developer): tidied up code

* chore(developer): tidied up code

* chore(developer): removed several Windows-specific functions

* chore(developer): removed unused datatype typedefs

* chore(developer): new GetCompilerErrorString() used in AddCompileMessage()

* chore(developer): new GetCompilerErrorString() used in AddCompileMessage()

* chore(developer): removed CWARN/CHINT out of CompMsg

* chore(developer): adapted kmcompxtest to test kmn-files that have errors included

* chore(developer): changed gitignore

* chore(developer): changed gitignore

* chore(developer): changed Version Number fot Tests 15.0.265.0 -> 15.0.270.0

* chore(developer): added tests for kmcompx

* chore(developer): move tests (for ErrorMessages) to common/test/keyboards

* chore(developer): move CERR-tests into test/keyboards/invalid

* chore(developer): change mostly whitespace changes

* chore(developer): change mostly whitespace changes

* chore(developer): delete files of keyboards/kmcompx_tests

* chore(developer): include check for correct Filename + rename var + change whitespace changes

* chore(developer): revert changes in AddCompileMessage, tidy up CreateTempFile

* chore(developer): remove python from meson.build

* chore(developer): change ErrExtra, change CompileMsg, change _finddata in FileExists

* chore(developer): check for '_' after Errornumber in Filename

* chore(developer): re-include AddCompilerVersionStore in Compiler.cpp

* chore(developer): add Message CERR_InvalidCharacter to UTF16TempFromUTF8() ; change #include <> -> ""

* chore(developer): add some more keyboardtests

* chore(developer): add some more keyboardtests

* chore(developer): change remaining old datatype-names to new names ( e.g. char -> KMX_CHAR )

* chore(developer): edit + add CERR_tests

* chore(developer): edit + add CERR_tests

* chore(developer): use char16_t in CheckFileNameConsistency

* chore(developer): comments: converting functions in kmx_u16

* chore: reorganize

* chore: commit kmcompx-files

* chore: use ifdef for vkeys

* chore: copy files back

* chore: more cleanup

* chore: meson build for kmcmplib

* chore: tweak unit test, error messages, and paths

* chore(developer): make kmcmplib a static library

* chore(developer): start kmcmpdll link to kmcmplib

* chore(developer): add build.sh for kmcmplib

* chore(developer): link kmcmpdll to kmcmplib

* chore(common): avoid building 'invalid' shared keyboards

* chore(developer): include BEGIN_LINE (repeated Begin)

* chore(developer): add more CERR_tests and move some to other folder, edit readme

* chore(developer): addCompilerMsg for RepeatedBegin

* chore(developer):repeatedBEgin plus rename functions to prevent same names in kmcmpdll and kmcmplib

* chore(developer): add flag flag_use_kmcompx as switch to use old code (kmcmpdll) or new code kmcmplib

* chore(developer): introduce namespace kmcmp instead of different filenames

* chore(developer): more changes for using namespaces

* chore(developer): solve merge conflicts in namespaces

* chore(developer): use namespaces in kmcmplib (except for in dllexport-functions)

* chore(developer): remove some unnecessary CERR_ tests

* chore(developer): remove CERR_CallIsProfessionalFeature, CERR_IncludeCodesIsProfessionalFeature, CERR_MnemonicLayoutIsProfessionalFeature

* chore(developer): note about func.call_js; change all "VESION x.y " to "VESION x.y or higher" ; rename flag for kmcompx to flag_use_new_kmcomp

* chore(developer): use PKMX_DWORD/PKMX_WCHAR   instead of LPKMX_DWORD/LPKMX_WCHAR; use PKMX_STR instead of char* in CompilerMessageProc

* chore(developer): remove kmx-file; NamedCodeConstants: rename variables, set const variables for buf sizes, brackets in for-loops

* chore(developer): change include paths ../

* chore(developer): get rid of reopened namespaces of kmcmp

* chore(developer): restored kmcompxtest

* chore(developer): change #include paths ../ ; excange km_kbp_cp with KMX_WCHAR; change some variable names

* chore(developer): use environment var KEYMAN_ROOT in path for tests; remove try-catch from json-validation.cpp

* chore(developer): use environment var KEYMAN_ROOT in path for tests; remove try-catch from json-validation.cpp

* chore(developer): revert change in NamedCodeConstants

* chore(developer): restored kmcompxtest, adapted readme

* chore(developer): use res-fils/version.rc for kmcmplib

* chore(developer): rename some functions/variables

* chore(developer): typos, comments, add function u16rchr()

* chore(developer): typos, comments, add function u16rchr()

* chore(developer): rename Functions, version rc-files

* chore(developer): rename Functions, version rc-files, set flag_use_new_kmcomp to use (old) kmcmpdll

* chore(developer): add BeginLine in kmcmpdll/Compiler.cpp; fread() in kmcmp_CompileKeyboardFileToBuffer

* chore(developer): set flag_use_new_kmcomp to only use kmcmpdll

* chore(developer): set flag_use_new_kmcomp to only use kmcmpdll...and back to kmcmplib

* (chore(developer): set flag_use_new_kmcomp to only use kmcmpdll...and back

* (chore(developer): changes in NamedCodeConstants::reindex() to run hieroglyphic.kmn properly

* chore(developer): changes in UnrechableRules::VerifyUnreachableRules() to run test_6440_unreachable_code..kmn

* chore(developer): remove unneccessary files, get ErrorMsg for INCLUDECODES, VISUALKEYBOARD, UnreachableRules, ErrExrta

* chore(developer): remove unneccessary files, get ErrorMsg for INCLUDECODES, VISUALKEYBOARD, UnreachableRules, ErrExrta, CWARN_ OK

* chore(developer): after stash:remove unneccessary files, get ErrorMsg for INCLUDECODES, VISUALKEYBOARD, UnreachableRules, ErrExrta, CWARN_ OK

* chore(developer): replace some #include <> with "" ; rename ErrExtra -> ErrExtraW

* chore(developer): replace some #include <> with "" ; rename ErrExtra -> ErrExtraW

* chore(developer): new converting function convert_pchar16T_To_pwcharT in kmx_u16

* chore(developer): CheckFilenameConsistency added fclose()

* chore(developer): --

* chore(developer): removed printf (---> started in... / °°-> changed to...)

* chore(developer): AddDeprecatedCode to get CWARN when in kmcomp.exe -s-w is set

* chore(developer): change MakeHashKeyFromFileKey16() to get correct behaviour of CHINT

* chore(developer): change MakeHashKeyFromFileKey16() to get correct behaviour of CHINT

* chore(developer): move converting function to kmx_u16

* chore(developer): add more CERR_ tests

* chore(developer): Not ready yet: changes in CheckFilenameConsistency to use on non-windows

* chore(developer): Not ready yet: changes in CheckFilenameConsistency to use on non-windows

* chore(developer): copied contents of 3 files of commit fcfc6c698a to here to achieve state of fcfc6c698a

* chore(developer): copied contents of 3 files of commit fcfc6c698a to here to achieve state of fcfc6c698a

* chore(developer): move/rename converting functions

* chore(developer) : update .gitignore, remove converting function

* chore(developer) : update gitignore; sort code in CheckFilenameConsistency::CheckFilenameConsistency()

* chore(developer) : change Version number 15.0.270 -> 17.0.51

* chore(developer): change #includes, path in meson.build

* chore(developer): new function Open_File() to open files on windows and non-windows platforms

* chore(developer): create Function to replace fopen/_wfsopen

* chore(developer): exclude kmx, kvk from git

* chore(developer): exclude kmx, kvk from git

* chore(developer): change #includes to include from kmcmplib

* chore(developer): remove stray fopen

* chore(developer): include Open_File() (as comment for now)

* chore(developer): include Open_File() (as comment for now)

* chore(developer): remove path to kmcmpdll in include_directories

* chore(developer): use functin Open_File()

* chore(developer): remove #ifdef for '\\' and '/' in both IsRelativePath; remove Function Open_File again

* chore(developer): new fun  u16rchr_LinWin, strrchr_LinWin to exchange #ifndef for '\\' and '/'

* chore(developer): implement u16rchr_LinWin and strrchr_LinWin

* chore(developer): implement Open_File()

* chore(developer): new MakeHashKeyFromFileKey, remove comments for Open_File, & *LinWin-functions, delete 1 converting function

* chore(developer): tidy up comments

---------

Co-authored-by: Marc Durdin <marc@durdin.net>
2023-03-01 20:00:04 +11:00
Steven R. Loomis
de3225a4f5 Merge remote-tracking branch 'origin/feature-ldml' into chore/core/7276-checksums-epic-ldml 2022-09-22 17:13:27 -05:00
Steven R. Loomis
93a9509302 chore(core): delete crc32.cpp and crc32.pas from some places 2022-09-21 18:13:19 -05:00
Marc Durdin
003f23d7b4 feat(developer): fixup and prepare build scripts and docs for kmc 2022-09-19 06:36:06 +10:00
Marc Durdin
561899a2a0 refactor(developer): rename kmldmlc to kmc
Fixes #7237.

Full clean, configure and build will be required for Core, Developer
after this change.

kmc is much less unwieldy a name than kmldmlc, and becomes our platform
for future compiler integration. Start of implementation for #5283 also.
2022-09-09 09:13:39 +10:00
Marc Durdin
14094c8979 feat(developer): add kmlmc and kmldmlc to make test
Also removes `-test` from normal `make build` for kmlmc, moving that
responsibility to `make test` (but uses `-tdd` to avoid rebuild).
2022-09-01 15:28:41 +10:00
Marc Durdin
9b7d44398d chore(developer): Coinit required for test suite
Removing unused dependencies also removed the coinit call. Re-adding coinit.
2022-07-28 09:24:18 +10:00
Marc Durdin
97817b8724 chore(developer): remove unused dependencies from KeymanWeb compiler 2022-07-27 14:58:13 +10:00
Marc Durdin
ca17d9e313 chore(developer): README and more path fixups 2022-06-23 10:58:58 +10:00
Marc Durdin
385045e978 chore(windows): fixup refs and move StockFileNames
* StockFileNames.pas was missed in earlier moves.
* Cleaned up references in various dproj files
* Fixed up broken references to common files
2022-06-23 09:47:59 +10:00
Marc Durdin
393083509a chore(windows): move tike files from global to developer
These files used only by TIKE so moved into that project.

Noticed obsolete refs to UMD5Hash and removed those at the same time.
2022-06-23 09:34:03 +10:00
Marc Durdin
9bb4c27e61 chore(windows): move general developer files to developer/ 2022-06-23 09:28:34 +10:00
Marc Durdin
8cbebf7ade chore(windows): move general package units to common 2022-06-23 06:36:23 +10:00
Marc Durdin
2aaa31eae2 chore(windows): move general keyboard units to common 2022-06-23 06:33:16 +10:00
Marc Durdin
96c1e10874 chore(windows): move Keyman.System.RegExGroupHelperRSP19902 unit to common 2022-06-22 14:02:03 +10:00
Marc Durdin
331c9d2d71 chore(windows): move shared general units to common 2022-06-22 13:58:58 +10:00
Marc Durdin
eba8a4ab35 chore(windows): move CRC32.pas to common 2022-06-22 10:36:55 +10:00
Marc Durdin
46c314cbb7 chore(windows): move BCP47Tag.pas to common 2022-06-22 10:35:18 +10:00
Marc Durdin
9dbe33e36e chore(windows): move KeymanVersion.pas to common and update refs 2022-06-21 09:39:04 +10:00
Marc Durdin
ad4039b5db chore(windows): move RegistryKeys.pas, ErrorControlledRegistry.pas, Glossary.pas, ScanCodeMap.pas to common 2022-06-21 09:39:03 +10:00
Marc Durdin
f163ceb6d5 chore(windows): move Unicode.pas to common 2022-06-21 09:39:01 +10:00
Marc Durdin
ff9a4c5edb chore(windows): move GetOSVersion to common 2022-06-21 09:39:00 +10:00
Marc Durdin
7ff83dcc1f chore(windows): move ExtShiftState to common 2022-06-21 09:38:52 +10:00
Marc Durdin
73c81f3a91 chore(developer): fix path refs in KeyboardPackageVersionsTestSuite 2022-06-21 06:48:10 +10:00
Marc Durdin
a295cfc8f0 chore(developer): move KMDActions component
Note this also drags TextFileFormat.pas and KMDActionInterfaces.pas into
the package so have moved those into general and components folders
respectively, alongside KMDActions.pas and updated corresponding
project includes.
2022-06-18 14:28:44 +10:00
Marc Durdin
1eff4dbca4 chore(windows): move global/delphi/visualkeyboard to common 2022-06-13 06:57:46 +10:00
Marc Durdin
6a56fca2d6 chore(windows): move global/delphi/tlb to common 2022-06-13 06:29:23 +10:00
Marc Durdin
cbda56f3e5 chore(windows): move global/delphi/standards to common 2022-06-13 06:15:47 +10:00
Marc Durdin
99b04a4240 chore(windows): move global/delphi/charmap to common
Moves character map units to common folder, and updates project refs.
2022-06-13 05:54:23 +10:00
Marc Durdin
c0ecb7e0f6 chore(windows): move Delphi developer units to developer 2022-06-12 15:38:19 +10:00
Marc Durdin
36b70758a9 chore(windows): remove unused OnlineConstants 2022-06-12 14:26:53 +10:00
Marc Durdin
e295e3ef71 chore(developer): move compfile.h to kmcmpdll
Moves compfile.h to kmcmpdll path and removes unnecessary dependencies on it.
2022-06-12 06:10:37 +10:00
Marc Durdin
cb240e259f chore(windows): move schemas to common 2022-06-10 13:26:05 +10:00
Marc Durdin
4992472561 chore(windows): update dcpcrypt refs 2022-06-02 08:51:58 +10:00
Marc Durdin
6409e51d61 chore(developer): update test build scripts 2022-05-23 06:08:36 +10:00
Marc Durdin
04b8615ed3 chore(developer): move developer tests to /developer/src/test 2022-05-23 06:07:43 +10:00