d5a4a6e195
than consider it an expected failure. With this change, if $expected_region_highlight and $region_highlight coincidentally have the same number of elements, the test won't be considered to fail. This is useful in conjunction with the next commit, q.v.. At this time, no tests set $expected_mismatch explicitly. However, the commit after next (this commit's grandchild) will add a test that will set $expected_mismatch implicitly, using the functionality in the next commit (this commit's child). |
||
|---|---|---|
| .. | ||
| generate.zsh | ||
| README.md | ||
| tap-colorizer.zsh | ||
| tap-filter | ||
| test-highlighting.zsh | ||
| test-perfs.zsh | ||
zsh-syntax-highlighting / tests
Utility scripts for testing zsh-syntax-highlighting highlighters.
The tests harness expects the highlighter directory to contain a test-data
directory with test data files.
See the main highlighter for examples.
Each test should define the string $BUFFER that is to be highlighted and the
array parameter $expected_region_highlight.
The value of that parameter is a list of strings of the form "$i $j $style".
or "$i $j $style $todo".
Each string specifies the highlighting that $BUFFER[$i,$j] should have;
that is, $i and $j specify a range, 1-indexed, inclusive of both endpoints.
$style is a key of $ZSH_HIGHLIGHT_STYLES.
If $todo exists, the test point is marked as TODO (the failure of that test
point will not fail the test), and $todo is used as the explanation.
If a test sets $skip_test to a non-empty string, the test will be skipped
with the provided string as the reason.
If a test sets unsorted=1 the order of highlights in $expected_region_highlight
need not match the order in $region_highlight.
Normally, tests fail if $expected_region_highlight and $region_highlight
have different numbers of elements. Tests may set $expected_mismatch to an
explanation string (like $todo) to avoid this and skip the cardinality check.
Note: $region_highlight uses the same "$i $j $style" syntax but
interprets the indexes differently.
Note: Tests are run with setopt NOUNSET WARN_CREATE_GLOBAL, so any
variables the test creates must be declared local.
Isolation: Each test is run in a separate subshell, so any variables,
aliases, functions, etc., it defines will be visible to the tested code (that
computes $region_highlight), but will not affect subsequent tests. The
current working directory of tests is set to a newly-created empty directory,
which is automatically cleaned up after the test exits. For example:
setopt PATH_DIRS
mkdir -p foo/bar
touch foo/bar/testing-issue-228
chmod +x foo/bar/testing-issue-228
path+=( "$PWD"/foo )
BUFFER='bar/testing-issue-228'
expected_region_highlight=(
"1 21 command" # bar/testing-issue-228
)
Writing new tests
An experimental tool is available to generate test files:
zsh -f tests/generate.zsh 'ls -x' acme newfile
This generates a highlighters/acme/test-data/newfile.zsh test file based on
the current highlighting of the given $BUFFER (in this case, ls -x).
This tool is experimental. Its interface may change. In particular it may
grow ways to set $PREBUFFER to inject free-form code into the generated file.
Highlighting test
test-highlighting.zsh tests the correctness of
the highlighting. Usage:
zsh test-highlighting.zsh <HIGHLIGHTER NAME>
All tests may be run with
make test
which will run all highlighting tests and report results in TAP format.
By default, the results of all tests will be printed; to show only "interesting"
results (tests that failed but were expected to succeed, or vice-versa), run
make quiet-test (or make test QUIET=y).
Performance test
test-perfs.zsh measures the time spent doing the
highlighting. Usage:
zsh test-perfs.zsh <HIGHLIGHTER NAME>
All tests may be run with
make perf