github-actions: kernelCI: Add support for running LTP tests ( Linux Test Project ) by shreeya-patel98 · Pull Request #1108 · ctrliq/kernel-src-tree

shreeya-patel98 · 2026-04-10T11:35:12Z

This PR adds LTP (Linux Test Project) support to the kernel CI workflow.

What's new :-

A new test-ltp job that runs LTP in parallel with kselftests using a separate qcow2 image
A compare-ltp job that does a named per-test diff against baseline (unlike kselftests which uses a ±3 threshold)
LTP results are informational only — regressions are shown in the PR body but don't block PR creation
Added skip_ltp input to skip LTP when not needed (e.g. CBR)
PR body now shows unified tables for both kselftests and LTP results

Notes :-

LTP suites covered: syscalls, syscalls-ipc, sched, math, nptl, crypto, cve
.github/ltp/ltp-expected.txt is included for tracking known/expected LTP failures. It's empty for now but can be populated later as we identify tests that consistently fail due to missing kernel features or known bugs, so they don't show up as regressions every run.
Two separate qcow2 images are built: one for kselftests, one for LTP

Testing :-

LTS 9.2 (I forget to push any new commits hence no PR was generated but testing works as expected. )
LTS 9.4
LTS 9.6
LTS 8.6
CLK 6.12
CLK 6.18
CBR 7.6
RLC 9
RLC 8
RLC 10

…kflow - Add test-ltp job: runs LTP in parallel with kselftests on separate runner - Add compare-ltp job: named per-test diff against baseline (not ±3 threshold) - Two separate QCOW2 artifacts: kernel-qcow2-kselftests-ARCH and kernel-qcow2-ltp-ARCH - Mount build_kselftests.sh and build_ltp.sh into builder container - LTP regressions are informational only - do not block PR creation - Add skip_ltp input to optionally skip LTP stage - compare-ltp outputs passed/failed counts, regressions, fixes per arch - Fetch create-pr-body-multiarch.sh from shreeya_kernelci_ltp instead of main - Workflow passes structured kselftest/LTP args to PR body script - Kselftests table skipped entirely when skip_kselftests=true - LTP table skipped entirely when skip_ltp=true Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Shreeya Patel <spatel@ciq.com>

… results Replace old Test Comparison section with consistent tables per test suite: - Both tables show: Architecture | Passed | Failed | Compared Against | Status - Kselftests table has tick (✅) and View kselftest logs link - LTP table has tick (✅) and View LTP logs link - LTP regression/fix details appear as bullet lists below LTP table - Tables skipped entirely when respective test suite is skipped - Removed old --comparison/--passed/--failed args, added structured args per suite Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Shreeya Patel <spatel@ciq.com>

…port Add skip_ltp input to the trigger workflow alongside the existing skip_kabi and skip_kselftests inputs. Validate, pass through env, and save to pr_metadata/skip_ltp.txt so the main workflow can read it via the pre-setup job's artifact-based metadata. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Shreeya Patel <spatel@ciq.com>

Copilot

Pull request overview

Adds Linux Test Project (LTP) coverage to the multi-arch kernel CI workflow, running it alongside kselftests and reporting per-test diffs against a baseline in the generated PR body.

Changes:

Introduces test-ltp and compare-ltp jobs and adds a skip_ltp workflow input/metadata propagation.
Splits qcow2 artifacts into separate images for kselftests vs LTP and updates downstream consumers.
Updates PR body generation to include unified tables for kselftests and LTP plus optional LTP regression/fix details.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
.github/workflows/kernel-build-and-test-multiarch.yml	Adds LTP execution + baseline comparison jobs, new qcow2 artifacts, and PR body enrichment wiring.
.github/workflows/kernel-build-and-test-multiarch-trigger.yml	Adds `skip_ltp` input validation and persists it into PR metadata artifacts.
.github/scripts/create-pr-body-multiarch.sh	Extends PR body formatting to include both kselftests and LTP tables/details.
.github/ltp/ltp-expected.txt	Introduces an expected-results file intended to filter/track known LTP outcomes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-10T11:41:52Z

.github/ltp/ltp-expected.txt

+#
+# Tests listed as FAIL here are known broken and will NOT block CI.
+# Tests listed as CONF are expected to be skipped due to kernel config.
+# Any test not listed that regresses from PASS->FAIL will block PR creation.


The documentation here says an unlisted PASS→FAIL regression will “block PR creation”, but the workflow treats LTP as informational only (jobs are continue-on-error: true and create-pr doesn’t gate on LTP results). Please update this comment to match the current CI behavior (e.g., note that it affects reporting/diffing only).

Suggested change

# Any test not listed that regresses from PASS->FAIL will block PR creation.

# Any test not listed that regresses from PASS->FAIL affects LTP reporting/diffing only;

# it does not block CI or PR creation.

Copilot · 2026-04-10T11:41:53Z

.github/workflows/kernel-build-and-test-multiarch.yml

+    - name: Checkout kernel-container-build (test branch)
+      id: generate_token
+      uses: actions/create-github-app-token@v1
+      with:
+        app-id: ${{ secrets.APP_ID }}
+        private-key: ${{ secrets.APP_PRIVATE_KEY }}
+        repositories: |
+          kernel-container-build


This step name is misleading: it generates a GitHub App token (id generate_token), but the name says “Checkout …”. Renaming it to something like “Generate GitHub App token” would avoid confusion when reading logs and debugging.

Copilot · 2026-04-10T11:41:53Z

.github/scripts/create-pr-body-multiarch.sh

 while [[ $# -gt 0 ]]; do
  case "$1" in
    --arch)
-      [[ $# -lt 2 ]] && { echo "Error: --arch requires a value" >&2; exit 1; }
      CURRENT_ARCH="$2"
-      # Only add to ARCHS array if not already present
      if [[ ! " ${ARCHS[@]:-} " =~ " ${CURRENT_ARCH} " ]]; then
        ARCHS+=("$CURRENT_ARCH")
      fi
      shift 2
      ;;
    --build-time)
-      [[ $# -lt 2 ]] && { echo "Error: --build-time requires a value" >&2; exit 1; }
-      [[ -z "$CURRENT_ARCH" ]] && { echo "Error: --arch must be specified before --build-time" >&2; exit 1; }
-      ARCH_DATA["${CURRENT_ARCH}_build_time"]="$2"
-      shift 2
-      ;;
+      ARCH_DATA["${CURRENT_ARCH}_build_time"]="$2"; shift 2 ;;
    --total-time)
-      [[ $# -lt 2 ]] && { echo "Error: --total-time requires a value" >&2; exit 1; }
-      [[ -z "$CURRENT_ARCH" ]] && { echo "Error: --arch must be specified before --total-time" >&2; exit 1; }
-      ARCH_DATA["${CURRENT_ARCH}_total_time"]="$2"
-      shift 2
-      ;;
-    --passed)
-      [[ $# -lt 2 ]] && { echo "Error: --passed requires a value" >&2; exit 1; }
-      [[ -z "$CURRENT_ARCH" ]] && { echo "Error: --arch must be specified before --passed" >&2; exit 1; }
-      ARCH_DATA["${CURRENT_ARCH}_passed"]="$2"
-      shift 2
-      ;;
-    --failed)
-      [[ $# -lt 2 ]] && { echo "Error: --failed requires a value" >&2; exit 1; }
-      [[ -z "$CURRENT_ARCH" ]] && { echo "Error: --arch must be specified before --failed" >&2; exit 1; }
-      ARCH_DATA["${CURRENT_ARCH}_failed"]="$2"
-      shift 2
-      ;;
+      ARCH_DATA["${CURRENT_ARCH}_total_time"]="$2"; shift 2 ;;


The argument parser no longer validates that options have a value (e.g., --arch at end) or that --arch was provided before arch-scoped flags like --build-time/--kselftest-passed. With set -u, this can fail with less actionable “unbound variable” errors or populate ARCH_DATA under an empty arch key. Consider restoring explicit checks (and a clear usage message) for missing values and for ordering/required context (CURRENT_ARCH set) before writing per-arch fields.

roxanan1996

LGTM except for the download artifact to compare ltp results.

roxanan1996 · 2026-04-10T12:12:57Z

.github/workflows/kernel-build-and-test-multiarch.yml

+
+        while read -r run; do
+          RUN_ID=$(echo "$run" | jq -r '.databaseId')
+          HEAD_BRANCH=$(echo "$run" | jq -r '.headBranch')


This is obsolete. See commit 560eb51
TLDR: HEAD_REF cannot be taken from the job metadata, so HEAD_REF is put in the artifact for every job.
Let me know if that makes sense.

Note: Since this is doing the same things in the kselftests job, I think it would be nice to create some common scripts/ github composite action in the future.

PlaidCat · 2026-04-10T17:54:30Z

.github/workflows/kernel-build-and-test-multiarch.yml

+
+        CURR_JSON=$(ls output-current/ltp-results.json 2>/dev/null | head -1)
+        PREV_JSON=$(ls output-previous/ltp-results.json 2>/dev/null | head -1)
+        EXPECTED_FILE=".github/ltp-expected.txt"


is this supposed to match the expected file above at .github/ltp/ltp-expected?
or is this at the a per branch .github/ltp-expected?

shreeya-patel98 and others added 3 commits April 10, 2026 11:11

Copilot AI review requested due to automatic review settings April 10, 2026 11:35

shreeya-patel98 requested review from PlaidCat, bmastbergen and kerneltoast as code owners April 10, 2026 11:35

Copilot started reviewing on behalf of shreeya-patel98 April 10, 2026 11:35 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

shreeya-patel98 requested a review from a team April 10, 2026 11:54

roxanan1996 reviewed Apr 10, 2026

View reviewed changes

PlaidCat reviewed Apr 10, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

github-actions: kernelCI: Add support for running LTP tests ( Linux Test Project )#1108

github-actions: kernelCI: Add support for running LTP tests ( Linux Test Project )#1108
shreeya-patel98 wants to merge 3 commits intomainfrom
shreeya_kernelci_ltp_rebased

shreeya-patel98 commented Apr 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

roxanan1996 left a comment

Uh oh!

roxanan1996 Apr 10, 2026

Uh oh!

PlaidCat Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

	# Any test not listed that regresses from PASS->FAIL will block PR creation.
	# Any test not listed that regresses from PASS->FAIL affects LTP reporting/diffing only;
	# it does not block CI or PR creation.

Conversation

shreeya-patel98 commented Apr 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

roxanan1996 left a comment

Choose a reason for hiding this comment

Uh oh!

roxanan1996 Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

PlaidCat Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants