Add a KMX+ file reader and tests, including a round-trip builder:reader
test.
Move basic-17.txt, basic-19.txt, and basic.xml to common test resources
folder so they can be used by unit tests in common and developer.
Test-bot: skip
fs dependency removed from the main hextobin function, only in the
filesystem.ts version. Also supports loading a segment of a hex file
for isolated tests.
Test-bot: skip
Implement the conversion from .kvk into KMX+ format, and embed the
result into the .kmx file. This change does not yet support &displaymap.
Move some virtual key modifier state helpers out of
visual-keyboard-compiler.ts and into shared module
modifier-key-constants.ts.
This change also flags the need to update the package compiler to
rewrite the font metadata into the KMX+ tables.
Test-bot: skip
Move some modifier key conversion utilities to a shared unit and also
add touchLayoutFilename and targetVersion to the kmcmplib extra data,
in preparation for generating and embedding OSK data.
Test-bot: skip
For keyboards that are v19 or later, always reserve space for the KMX+
header that will be written out in kmc-kmn, to avoid rewriting the whole
file. At this time, zero out that space.
Test-bot: skip
Build-bot: skip build:developer
The disp and layr sections have new v19 layouts, to support the Keyman
OSK requirements for epic/embed-osk-in-kmx. This supports writing the
new versions of these sections in Developer and loading them into Core,
doing transformations where necessary so that Core always works with v19
structures after load.
This does not implement the transformations required to support the OSK
APIs; that will be implemented in a follow-up PR. Nor is support for
writing the OSK data from source .kvks and .keyman-touch-layout
supported; this is just the scaffolding for supporting the structures in
the KMX+ data.
Test-bot: skip
Adds a call to map2pdb for Delphi projects, if map2pdb is an executable
on the path. This way, we get .pdb files we can use for debugging and
for performance profiling. This also replaces tds2dbg.
If map2dbg is available, then the build and install actions will copy
the PDB to be alongside the corresponding executable, making most debug
actions relatively seamless for finding symbols.
Also fixes up setup.exe in Windows and Developer to extend setup.exe to
a 512-byte boundary after map2pdb patches the PE.
Relates-to: #14787
Build-bot: skip release:developer,windows
Test-bot: skip
This change wraps the stack walk with an exception handler, which will
enable some crash handlers to continue. However, in some circumstances,
such as the stack captured below, it appears to be an unrecoverable
crash.
So, to mitigate this scenario, we also now only capture stack traces for
crashes that we know are serious -- the set of exception types listed in
the patch.
I experienced this issue on my machine after a Windows update; it is not
clear to me if this is related to the Windows update or if it is due to
my debugging environment for Keyman. The symptoms were that Keyman
Configuration would start up, but show just a blank white dialog, and a
few seconds later would abort. Observing process history showed that
kmshell.exe and kmbrowserhost.exe would both crash, with related call
stacks -- see below for a sample. These call stacks are coming out of
keyman32 calling OutputDebugString from DllMain, which may not be a safe
thing to do, because of the exception side-effect. A separate patch will
be submitted to address that - probably just removing any
OutputDebugString calls from DllMain paths.
In any case, this change adds robustness and should mitigate other
unexpected exceptions.
```
0:026> kb
# ChildEBP RetAddr Args to Child
00 08a1d4c8 750d6e3d 750d6e20 750bd307 04447396 ucrtbase!abort+0x31
01 08a1d4d0 750bd307 04447396 051caaa0 629f52f0 ucrtbase!_purecall+0x1d
02 08a1d508 62ab2914 0495e1c8 00000000 08a1d5fc ucrtbase!__crt_state_management::wrapped_invoke<int (__cdecl*)(void),int>+0x2a
03 08a1d528 62aba00e 0495e1c8 00000000 08a1d5fc dbghelp!SymbolDataSimpleImpl<1,3>::getData+0x24
04 08a1d580 62ac649d 0495e1c8 08a1d5fc 08a1da34 dbghelp!GetData::getSymData+0x6e
05 08a1d5dc 62ac3272 08a1d5fc 05497580 08a1da5c dbghelp!CPubByAddrTrav::get+0xad
06 08a1d948 62ac7340 08a1da00 08a1da34 08a1da5c dbghelp!findBetterSymbol+0x62
07 08a1da40 62ac87eb 08a1da5c 08a1da64 62a05ea0 dbghelp!CAllSymsByAddrTrav::getEnclosingSymbol+0x80
08 08a1da6c 62abfb10 00000001 00030ed1 05497580 dbghelp!CAllSymsByAddrTrav::init+0xeb
09 08a1da88 62a876e9 00000001 00030ed1 043e9008 dbghelp!CAllSymsByAddrTrav::FInit+0xa0
0a 08a1daac 62a87846 043e9008 00000001 00030ed1 dbghelp!CDiaSession::findSymbolByAddr+0x2a9
0b 08a1dadc 62a87899 043e9008 00031ed1 00000000 dbghelp!CDiaSession::findSymbolByRVA+0x76
0c 08a1db10 62aed8f0 043e9008 00031ed1 00000000 dbghelp!CDiaSession::findSymbolByRVAEx+0x39
0d 08a1dbac 62b1a980 533b1ed1 00000000 00000000 dbghelp!FindSymbolByVA+0x117
0e 08a1dc00 629ed8da 533b1ed1 00000000 93b94d79 dbghelp!diaGetFpoFromAddr+0x6c
0f 08a1dc4c 629ed647 533b1ed1 00000000 93b94dad dbghelp!LookupFunctionEntryX86+0x23d
10 08a1dc98 629ed4c7 ffffffff 533b1ed1 00000000 dbghelp!SymFunctionTableAccess64AccessRoutines+0x167
11 08a1dcb4 629ec7a1 ffffffff 533b1ed1 00000000 dbghelp!FunctionTableAccessRoutineLocal+0x17
12 08a1dcd8 629eba25 533b1ed1 00000000 08a1dd10 dbghelp!DbhStackServices::GetFunctionEntry+0x31
13 08a1dd58 629eb126 00000001 08a1de00 08a1eda0 dbghelp!DbsStackUnwinder::StaticAdjustForNoReturn+0x4a
14 08a1ddbc 629e8a76 08a1ddf8 08a1dea8 08a1dec8 dbghelp!DbsX86StackUnwinder::UnwindAndUpdateInternalContext+0x136
15 08a1deec 629e88b4 08a1eda0 00000005 05482908 dbghelp!DbsStackUnwinder::DoDbhUnwind+0x177
16 08a1df1c 629e87b8 08a1eda0 00000005 05482908 dbghelp!DbsStackUnwinder::DbhUnwind+0x58
17 08a1e038 629e867e 08a1eda0 08a1f000 00725a60 dbghelp!PickX86Walk+0x109
18 08a1ecf4 62af77c9 08a1eda0 08a1f000 00000000 dbghelp!DoUnwindStackFrameUsingServices+0xbb
19 08a1ed4c 62b3ad4b 0000014c ffffffff fffffffe dbghelp!StackWalk2+0x1c9
1a 08a1eebc 2c7667c4 0000014c ffffffff fffffffe dbghelp!StackWalk64+0x8b
WARNING: Stack unwind information not available. Following frames may be wrong.
1b 08a1f2cc 2c765ad4 00000000 04e57318 08a1f32c kmshell+0x3667c4
1c 08a1f2dc 7726369f 08a1f2f8 08a1f428 08a1f3d8 kmshell+0x365ad4
1d 08a1f32c 7725eb04 00000000 5338cc66 08a1fbac ntdll!RtlpCallVectoredHandlers+0xf8
1e 08a1f3c0 7726b66f 08a1f3d8 08a1f428 08a1f3d8 ntdll!RtlDispatchException+0x67
1f 08a1f3c0 753e5004 08a1f3d8 08a1f428 08a1f3d8 ntdll!KiUserExceptionDispatcher+0xf
20 08a1f8ec 754c7ab0 40010006 00000000 00000002 KERNELBASE!RaiseException+0x64
21 08a1fb54 533ab6a2 08a1fbac 5338cc66 08a1fe38 KERNELBASE!OutputDebugStringA+0x50
22 08a1fcb0 533b1ba3 53673110 5338cc66 08a1fe38 keyman32!_OutputThreadDebugString+0x42 [D:\Projects\keyman\app\windows\src\engine\keyman32\K32_DBG.CPP @ 291]
23 08a1fd18 533b1ed2 00000000 5338cc66 08a1fe38 keyman32!ReleaseKeyboards+0x13 [D:\Projects\keyman\app\windows\src\engine\keyman32\keyman32.cpp @ 841]
24 08a1fd70 533b1fad 00000000 5338cc66 08a1fe38 keyman32!UninitialiseProcess+0x22 [D:\Projects\keyman\app\windows\src\engine\keyman32\keyman32.cpp @ 160]
25 08a1fdcc 53431b12 53380000 00000003 00000000 keyman32!DllMain+0x5d [D:\Projects\keyman\app\windows\src\engine\keyman32\keyman32.cpp @ 143]
26 08a1fe10 53431cf0 53380000 00000003 00000000 keyman32!dllmain_dispatch+0xb2 [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\dll_dllmain.cpp @ 281]
27 08a1fe24 77268ff6 53380000 00000003 00000000 keyman32!_DllMainCRTStartup+0x20 [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\dll_dllmain.cpp @ 334]
28 08a1fe44 772a0df3 5338cc66 53380000 00000003 ntdll!LdrxCallInitRoutine+0x16
29 08a1fe68 7722b11c 00000003 00000000 fb42ae4c ntdll!LdrpCallInitRoutineInternal+0x22
2a 08a1feb0 77229812 00000003 00000000 fb42af9c ntdll!LdrpCallInitRoutine+0xae
2b 08a1ff60 77261670 1138e500 00000000 052cbe98 ntdll!LdrShutdownThread+0x1b2
2c 08a1ff74 757c5d50 00000000 757c5d30 08a1ffdc ntdll!RtlExitUserThread+0x30
2d 08a1ff84 7725d6db 052cbe98 fb42af20 00000000 kernel32!BaseThreadInitThunk+0x20
2e 08a1ffdc 7725d661 ffffffff 772a4680 00000000 ntdll!__RtlUserThreadStart+0x2b
2f 08a1ffec 00000000 1138e500 052cbe98 00000000 ntdll!_RtlUserThreadStart+0x1b
```
The name `form` better represents the semantic meaning of the
structure, and reduces confusion with the `list` section and other uses
of `list` in KMX+.
Test-bot: skip
To reduce confusion when 'Follow Keyboard Version' is set, clear the
cached package version data from UI and .kps source when saving the
package or making changes to it.
Fixes: #11892
Test-bot: skip
While not documented as such, it appears that the repository record is
required with trusted publishing of npm packages.
Test-bot: skip
Build-bot: skip
Due to recent changes in NPM package publishing security requirements,
we have to move from TeamCity build to a GitHub Action to publish our
NPM packages, so we can take advantage of trusted publishing. This
change also consolidates and centralizes the npm publishing into
resources/build/ci/npm-publish.sh, which removes a lot of boilerplate
from each of the build.sh scripts, and ensures consistency.
Packages will be `npm pack`ed on PR and test builds, and published in
release builds.
Ref: https://docs.npmjs.com/trusted-publishers
Ref: https://github.blog/changelog/2025-09-29-strengthening-npm-security-important-changes-to-authentication-and-token-management/Fixes: #14963
Test-bot: skip
Build-bot: release:developer
