Repository navigation
kbuild: read kselftest suite results from the kernel's build report - #3264
Open
sashalevin wants to merge 1 commit into
Open
sashalevin wants to merge 1 commit into
sashalevin wants to merge 1 commit into
Conversation
The per-suite kselftest build results are derived from the files in the
kselftest tarball: a suite passes when it installed anything and fails
otherwise. That gets two cases wrong:
- Suites for other architectures, such as sparc64, powerpc or sgx on
arm64, build and install nothing by design and are reported as failed
on every build.
- A suite whose compilation failed still passes when it installed some
of its files, typically its scripts.
Since v7.4 the kernel records the exit status of each suite's build and
installation in the file named by KSFT_BUILD_REPORT, and advertises the
format of that file in KSFT_BUILD_REPORT_VERSION, see commit
39261b797f40 ("selftests: optionally record per-suite build and install
results") in the kernel.
Pass KSFT_BUILD_REPORT to the kselftest build and save the format
version next to the list of expected suites. When the kernel wrote a
version 1 report, a suite fails if any of its steps failed or it has no
record, passes if all of its steps succeeded and it installed files, and
is skipped if all of its steps succeeded but it installed nothing.
Records for subdirectories and installation dependencies, such as
net/lib, count for the suite they belong to.
Kernels without the report, and builds that wrote no usable record,
keep the presence-based results. The aggregate kselftest result and the
kernel build result do not change.
TuxMake runs the installation with make -i, which hides failures of
that step. Build failures are still recorded by the make all that runs
before it.
The report is named build_kselftest_report.tsv so that it is attached to
the kselftest node along with the build logs.
Signed-off-by: Sasha Levin <sashal@kernel.org>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The per-suite kselftest build results are derived from the files in the kselftest tarball: a suite passes when it installed anything and fails otherwise. That gets two cases wrong:
Since v7.4 the kernel records the exit status of each suite's build and installation in the file named by KSFT_BUILD_REPORT, and advertises the format of that file in KSFT_BUILD_REPORT_VERSION, see commit 39261b797f40 ("selftests: optionally record per-suite build and install results") in the kernel.
Pass KSFT_BUILD_REPORT to the kselftest build and save the format version next to the list of expected suites. When the kernel wrote a version 1 report, a suite fails if any of its steps failed or it has no record, passes if all of its steps succeeded and it installed files, and is skipped if all of its steps succeeded but it installed nothing. Records for subdirectories and installation dependencies, such as net/lib, count for the suite they belong to.
Kernels without the report, and builds that wrote no usable record, keep the presence-based results. The aggregate kselftest result and the kernel build result do not change.
TuxMake runs the installation with make -i, which hides failures of that step. Build failures are still recorded by the make all that runs before it.
The report is named build_kselftest_report.tsv so that it is attached to the kselftest node along with the build logs.