This also removes the need to have a test for `-b` parameter in
trigger-test-builds.sh, because the TC scripts calling this can already
factor that in.
Testing script trigger so only do builds of common to verify
Fixes: #14483
Test-bot: skip
Build-bot: skip:all build:common
Consolidates the node-related script functions into node.inc.sh, as part
of cleaning up the build scripts and making them easier to maintain into
the future.
Fixes: #14447
Clarifies the confusing builder.inc.sh / build-utils.sh distinction by
giving the scripts more appropriate names. Most build scripts should use
builder-full.inc.sh; some helper scripts can use builder-basic.inc.sh.
Documented in resources/build/README.md.
Renames:
* resources/build/builder.inc.sh to resources/build/builder-full.inc.sh
* resources/build/build-utils.sh to resources/build/builder-basic.inc.sh
Other changes:
* Moves Android-specific functions out of builder-basic.inc.sh and into
android/build.sh.
* Renames functions in builder-basic.inc.sh
More functions may be moved from builder-basic.inc.sh into utils.inc.sh
or other scripts in the future.
Fixes: #14065
Build-bot: build all
Test-bot: skip
Adds documentation for various functions, removes unused functions (a
couple of simple, very lightly used functions were unDRYed; these could
go back the other way across all shell scripts if necessary). TODO items
noted for follow-up refactoring.
Fixes: #14275
Relates-to: #14269
Build-bot: build all
Test-bot: skip
Previously the builder scripts defined a readonly `VERSION` environment
variable for the Keyman version. That caused problems when another
(external) script tried to define a `VERSION` variable. We encountered
this problem when trying to move the TC build steps of a configuration
into a single script (#13399) when we tried to source `~/.nvm/nvm.sh`.
This change uses a Keyman specific prefix for the version variables and
renames `VERSION` → `KEYMAN_VERSION` etc. Unfortunately these variables
are used in a lot of places, so this turned out to be a bit of a yak
shave.
Test-bot: skip
Note: there is a bit of potential confusion about the difference between
/resources/builder.inc.sh (the full implementation for builder scripts),
and /resources/build/builder.inc.sh (the source script that builder
scripts should always use).
This allows us to make assumptions that will always be true for builder
scripts that may not be true for other scripts, such as setting base
folder.
Fixes#11324.
* Always `cd "$THIS_SCRIPT_PATH"`
* Remove unnecessary `cd` from all build.sh
* Remove unnecessary `set -eu` from all build.sh (and `# set -x`)
* Replace old build-utils.sh incantation in a few build.sh scripts
Fixes#11220.
Replaces the sed script for matching emoji in ios prepRelease with a
tiny node project that uses emoji-regex package, to reduce future
maintenance.
Also strips lingering emoji and :xxx: emoji shortcut strings from
HISTORY.md. Note that :xxx: are not stripped programatically at this
time.
Fixes#10721.
This arose partly because of the change from release- to release@ in at
17.0.30-alpha (triggered by sentry req). Will require a corresponding
patch to CI script to pass the --from and --to parameters.
* Fixes the path for the increment-version module
* Uses exit code 50 instead of 1 to indicate 'success, but no change',
so that low-value error exit codes as used by node.js don't
indavertently continue the build
* Addresses some Typescript-specific breaking changes in destructured
types.
* Removes npm version calls and package*.json updates -- these are no
longer required.
* Adds resources/build/version as a workspace to root package.json.
* Runs `npm ci` on the repo root for resources/build/version.
* Adds additional logging.
Fixes#6391.
The issue here was quite nuanced: where we merged a PR, and then merged
the changes into an earlier PR, but left it open, the earlier PR could
end up getting the release information comment from keyman-server
instead of the later, merged PR. This was because the commit would
appear in both PRs, so GitHub would report on both of them.
This is not a 100% fix -- but it should stop the PR tagging on unmerged
pull requests. The one situation where this may leave a gap is where we
have multiple PRs merged on the same day, where a later PR's changes are
merged into an earlier PR; in this case I think it is possible that one
of the PRs will not get the informative version comment.
But given the current fix should avoid the more common situation, I am
comfortable to leave it as is for now. This is only informative data...
and a more complete fix is a distraction.
Starts to resolve the chaos of putting version numbers into every
package.json file.
Adds a new keyman-version package which mimics the
version information in build-utils.sh. This will eventually replace
resources/web-environment.
Fixes the build for the lexical model compiler tools to remove refs to
package.json version and use keyman-version instead.
More fixes coming in a follow-up.
Fixes#6320.
Part 1 of moving from lerna to a simpler, maintained monorepo solution,
using TypeScript Projects and NPM Workspaces.
There is more work to be done here. At this point, KeymanWeb builds and
runs without errors, but the built file is substantially different,
mostly in include order.
Using TypeScript Projects, we move away from the need to run build
scripts in various locations for almost all the Typescript modules.
TODO: Embedded versions and tests have not been verified.
TODO: developer/server is not yet verified.
TODO: developer/js (needs a rename!) is not yet verified.
TODO: Currently, the predictive-text folder needs refactoring to move
the construction of the worker wrapper out of the Predictive Text build
and into the final assembly of keymanweb.js (as it should be valid to
run it as a separate .js anyway).
TODO: Most of the `<reference>` paths need to be re-verified. Ideally
there should be no references outside the current module for any given
.ts.
TODO: The embedded vs browser vs node (headless) builds should be tidied
up for consistency so that it's obvious what depends on what. This is
currently messiest in the predictive-text folder, where the output names
diverge from the filenames and the various files are mixed in the same
folder (as evidenced by the exclusions listed in each tsconfig.json).
TODO: `npm install` should be removed from most build scripts and
instead `npm ci` (#6196) should be run only once from the top-level
folder for any given build. I've had eliminated side-effects from the
`install` action for npm, which makes it easier to reason about state.
TODO: verify_npm_setup and related functions can probably be eliminated.
TODO: most of the build scripts should be largely eliminated for web.
TODO: several ts projects use inconsistent output folders.
TODO: it may be possible to generate a .d.ts for models/types so that
we can use a consistent reference for those as well.
TODO: build.sh, tsconfig.json should always be in the module's top-level
folder, not in a subfolder such as src (e.g. see input-processor/src,
keyboard-processor/src, web/source).
TODO: resources/web-environment should be in common/web.
TODO: other js node_modules imports should be wrapped like es6-shim.
TODO: fix up the publish code for npm modules
TODO: eliminate version numbers from package.json if possible?
Whew, that's most of the stuff I noticed!
1. Adds parsing of commit comments to extract PR title + number.
2. Uses `--first-parent` to exclude merges that are not to the
base branch. This means that if you merge a PR into another, that
child PR will not be listed in the history. From my perspective, this
is fine -- we prefer chained PRs to be merged bottom-up rather than
top-down, and this allows for PRs to be collapsed into one history
entry if we really want that anyway.
3. Cleans up the git log range to work correctly regardless of your
current branch.
4. Reads git log information from origin/base rather than base.
Note: this assumes that you have recently `git fetch origin` (or
`git pull origin`); that's up to you.
Relates to #5172.
Adds a report-history.sh script to help report on what changes will be
in the upcoming build. Uses the same strategy as we currently use to
grab pull request numbers for HISTORY.md.
Also fixes duplicate entries that sometimes appear in HISTORY.md due to
merges from master into PR branches.
Adds a report-history mode for version.js so we can examine the list of
changes that will be in the next release build for a release branch
(master, beta, stable-x.y).
How to run:
pushd resources/build/version
npm install
npm run build:ts
popd
node resources/build/version/lib/index.js report-history -t "$GITHUB_TOKEN" -b "$base"
* Note: Always run from repo root.
* Where $GITHUB_TOKEN is a personal access token
* Where $base, if specified is the branch to validate (default: master)
Allows us to manually run a version increment, for example if there is
a problem with a release build such as running out of space on the
download server.
Fixes#4484.
tsysinfox64 was being signed, but too late in the build
process as it is embedded into tsysinfo.exe. There was
code to sign it earlier, but that was being skipped due
to a missing build flag. This has been a problem for
some time.
