spiegel_podman/test/buildah-bud/make-new-buildah-diffs
Ed Santiago f56f5851b3 buildah-bud tests: simplify
Experience this week has shown that managing .diff files
is too difficult for humans, and too fragile. Opportunities
for errors abound. So, let's try to minimize the diffs.

We can't eliminate the diffs to helpers.bash: those are
true code changes that are absolutely required for running
tests using podman instead of buildah. We need to carry
those ourselves: they are not appropriate for the buildah
repo itself.

What we can do is simplify the patching of bud.bats. That
is fragile, because bud.bats changes often, and context-
sensitive git patch files can easily get confused.

Recognizing that the changes to bud.bats fall under two types:

  - tests that are skipped
  - tests in which podman error messages differ from buildah's

...we now have a new script, apply-podman-deltas, which
is (I hope) much user-friendlier. It understands two directives:

  errmsg - alter the expected error message
  skip   - skip a test

Both operate based on a bats test name. The test name must
match exactly. These directives use 'sed' to update bud.bats.
If any directive fails, the script will keep going (so you
get as many errors as possible in a run), then exits failure.

Instructions (README.md) now explain the process for dealing
with all expected test failures.

(Sneak checkin: add '--filter=NAME' option to test runner,
allowing for targeted and much shorter test runs).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-04-08 14:43:49 -06:00

63 lines
2 KiB
Bash

#!/bin/bash
#
# This script is intended to help developers get buildah-tests-under-podman
# working again in case of failure.
#
ME=$(basename $0)
die() {
echo "$ME: $*" >&2
exit 1
}
# Confirm that we're in a test-buildah* subdir of podman
whereami=$(basename $(pwd))
if [[ ! $whereami =~ test-buildah-v ]]; then
die "Please run me while cd'ed to a test-buildah-vN.M directory"
fi
# FIXME: check that git repo is buildah
git remote -v | grep -q [BUILDAHREPO] \
|| die "This does not look like a buildah repo (git remote -v)"
# We could do the commit automatically, but it's prudent to require human
# involvement.
modified=$(git status --untracked=no --porcelain)
if [[ -n "$modified" ]]; then
echo $modified
die "Please commit your changes: git commit --amend --all"
fi
# Remove any 00??-*.patch files
rm -f 0001-*.patch
# Check count of commits, barf if need to squash
n_commits=$(git log --pretty=format:%h [BASETAG]..HEAD | wc -l)
if [[ $n_commits -gt 1 ]]; then
die "Please squash your commits"
fi
# Scope check: make sure the only files changed is helpers.bash
changes=$(git diff --name-status [BASETAG]..HEAD | egrep -v '\stests/helpers.bash')
if [[ -n "$changes" ]]; then
echo $changes
die "Found modified files other than 'tests/helpers.bash'"
fi
###############################################################################
# All right - things look good. Generate the patch, and copy it into place.
git format-patch [BASETAG]
# Once again, make sure there's exactly one and only one commit
shopt -s nullglob
patch2=$(echo 0002-*.patch)
if [[ -n "$patch2" ]]; then
die "Internal error: I thought I checked for squashed commits, but still see $patch2"
fi
# All looks good. Now write that patch into its proper place in the
# podman repo. The sed and tac mess strips trailing whitespace and
# empty lines; we need to do this to pass github CI checks.
sed -e 's/ \+$//' <0001-*.patch |\
tac | sed -e '/./,$!d' | tac >| ../test/buildah-bud/buildah-tests.diff