Skip to content

feat(test): add ut-summary.json generation for test results and coverage - #317

Merged
lzwind merged 1 commit into
linuxdeepin:masterfrom
add-uos:feat/add-ut-summary-json
Aug 13, 2026
Merged

feat(test): add ut-summary.json generation for test results and coverage#317
lzwind merged 1 commit into
linuxdeepin:masterfrom
add-uos:feat/add-ut-summary-json

Conversation

@add-uos

@add-uos add-uos commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add tests/gen-ut-summary.py: standalone script to parse gtest XML reports and lcov coverage data, generating a structured ut-summary.json
  • Update tests/test-prj-running.sh: export env vars, set +e for test running, call gen-ut-summary.py, propagate test exit code

Output format

Running test-prj-running.sh now generates build-ut/ut-summary.json:

{
  "test_cases": {
    "total": 0,
    "passed": 0,
    "failed": 0
  },
  "line_coverage": {
    "total": 0,
    "passed": 0,
    "failed": 0,
    "coverage": "0.00%"
  },
  "function_coverage": {
    "total": 0,
    "passed": 0,
    "failed": 0,
    "coverage": "0.00%"
  }
}

Test plan

  • gen-ut-summary.py correctly parses gtest XML and lcov --summary output
  • Environment variables (projectdir, builddir, reportdir) properly exported
  • Test exit code propagated correctly

Summary by Sourcery

Generate a unified unit-test summary JSON from gtest XML reports and lcov coverage data and integrate it into the existing test/coverage workflow.

New Features:

  • Add gen-ut-summary.py to produce structured ut-summary.json combining test case counts and coverage statistics.
  • Extend test-prj-running.sh to invoke gen-ut-summary.py and emit ut-summary.json into the build-ut report directory.

Enhancements:

  • Export projectdir, builddir, and reportdir from test-prj-running.sh for downstream tooling and simplify path resolution.
  • Preserve and propagate the unit test exit code after coverage and summary generation.
  • Bump application version in linglong.yaml from 6.5.58.1 to 6.5.59.1.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @add-uos, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@github-actions

Copy link
Copy Markdown
  • 检测到敏感词export变动
详情
    {
    "export": {
        "tests/test-prj-running.sh": {
            "b": [
                "export builddir=build",
                "export reportdir=build-ut",
                "export scriptdir=\"$(cd \"$(dirname \"$0\")\" && pwd)\"",
                "export projectdir=\"$(cd \"${scriptdir}/..\" && pwd)\""
            ]
        }
    }
}

@github-actions

Copy link
Copy Markdown

TAG Bot

TAG: 6.5.59
EXISTED: no
DISTRIBUTION: unstable

@sourcery-ai

sourcery-ai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR introduces a Python helper to aggregate gtest XML and lcov coverage into a single ut-summary.json, and wires it into the existing test-prj-running.sh flow while adjusting environment handling and preserving the test exit status, plus a minor project version bump.

Sequence diagram for updated test-prj-running.sh and gen-ut-summary.py flow

sequenceDiagram
    actor Developer
    participant test_prj_running_sh as tests/test-prj-running.sh
    participant gtest as gtest
    participant lcov as lcov
    participant gen_ut_summary_py as tests/gen-ut-summary.py
    participant ut_summary_json as build-ut/ut-summary.json

    Developer->>test_prj_running_sh: tests/test-prj-running.sh
    test_prj_running_sh->>test_prj_running_sh: export_projectdir_builddir_reportdir
    test_prj_running_sh->>test_prj_running_sh: set_+e
    test_prj_running_sh->>gtest: run_gtest_tests
    gtest-->>test_prj_running_sh: gtest_XML_reports
    test_prj_running_sh->>lcov: lcov --summary
    lcov-->>test_prj_running_sh: lcov_summary_output
    test_prj_running_sh->>gen_ut_summary_py: tests/gen-ut-summary.py
    gen_ut_summary_py->>gen_ut_summary_py: parse_gtest_XML_and_lcov_summary
    gen_ut_summary_py->>ut_summary_json: generate_ut-summary.json
    gen_ut_summary_py-->>test_prj_running_sh: summary_generation_complete
    test_prj_running_sh-->>Developer: propagate_test_exit_code
Loading

Flow diagram for ut-summary.json generation from gtest and lcov

flowchart LR
    A[test_prj_running_sh] --> B[gtest XML reports]
    A --> C[lcov --summary output]
    B --> D[gen_ut_summary_py]
    C --> D[gen_ut_summary_py]
    D --> E[build-ut/ut-summary.json]
Loading

File-Level Changes

Change Details Files
Add a standalone Python script that generates ut-summary.json from gtest XML reports and lcov coverage data.
  • Implement XML parsing to compute total, passed, and failed test cases from gtest/JUnit-style reports.
  • Run lcov --summary via subprocess and regex-parse line and function coverage statistics.
  • Assemble a structured JSON object with test case and coverage summaries and write it to the report directory.
  • Use environment variables to locate project, build, and report directories, with optional overrides for XML and coverage inputs.
tests/gen-ut-summary.py
Update the test harness script to export common directories, adjust path resolution, integrate summary generation, and propagate the test exit code.
  • Export builddir, reportdir, scriptdir, and projectdir so they are available to downstream scripts.
  • Change CMake and report paths to derive from projectdir instead of locally computed project_root/script_dir.
  • Wrap test binary invocations with set +e/set -e to capture the test exit code while still running coverage-related commands.
  • Invoke the new gen-ut-summary.py script and exit with the captured test status instead of always exiting 0.
  • Use scriptdir consistently when calling exclude_unreachable.py.
tests/test-prj-running.sh
Bump the application version to reflect the new test/coverage summary feature.
  • Update the application version field from 6.5.58.1 to 6.5.59.1 in the packaging metadata.
linglong.yaml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

- Add gen-ut-summary.py: standalone script to parse gtest XML and lcov
  coverage data, output structured JSON (test_cases/line_coverage/function_coverage)
- Update test-prj-running.sh: export env vars, set +e for test running,
  call gen-ut-summary.py, propagate test exit code
@add-uos
add-uos force-pushed the feat/add-ut-summary-json branch from 9593524 to 42cdd2f Compare August 13, 2026 09:23
@github-actions

Copy link
Copy Markdown
  • 检测到敏感词export变动
详情
    {
    "export": {
        "tests/test-prj-running.sh": {
            "b": [
                "export builddir=build",
                "export reportdir=build-ut",
                "export scriptdir=\"$(cd \"$(dirname \"$0\")\" && pwd)\"",
                "export projectdir=\"$(cd \"${scriptdir}/..\" && pwd)\""
            ]
        }
    }
}

@deepin-ci-robot

Copy link
Copy Markdown

deepin pr auto review

★ 总体评分:75分

■ 【总体评价】

代码实现了单元测试摘要生成与测试脚本集成,但存在脚本退出码覆盖的逻辑瑕疵
逻辑基本正确但因生成摘要脚本失败会导致退出码丢失扣25分

■ 【详细分析】

  • 1.语法逻辑(存在错误)✕

test-prj-running.sh 中,python3 "${scriptdir}/gen-ut-summary.py"set -e 环境下执行。如果该 Python 脚本因任何原因(如 XML 格式错误、lcov 缺失等)抛出异常退出,Shell 脚本会立即终止,导致最后的 exit $test_exit_code 无法执行。
潜在问题:测试失败时若摘要生成失败,CI 系统将收到 Python 脚本的错误码而非测试失败码,可能导致测试结果误判。
建议:在执行 Python 脚本时添加 || true 或显式捕获退出码,确保无论摘要生成是否成功,最终都以测试退出码退出。

  • 2.代码质量(良好)✓

代码结构清晰,gen-ut-summary.py 包含完善的 docstring 和注释,解释了环境变量和输出格式。Shell 脚本中变量重命名统一了命名规范,使用 export 传递环境变量是合理的做法。
潜在问题:无
建议:无需修改

  • 3.代码性能(无性能问题)✓

Python 脚本使用 globElementTree 解析 XML,使用正则解析 lcov 输出,复杂度在可接受范围内。Shell 脚本流程未引入冗余操作。
潜在问题:无
建议:无需修改

  • 4.代码安全(存在0个安全漏洞)✓

漏洞对比统计:新增漏洞 0 个,减少漏洞 0 个,持平 0 个
代码仅处理本地测试报告和覆盖率文件,未涉及网络输入或敏感权限操作,无安全风险。

  • 建议:无需修改

■ 【改进建议代码示例】

diff --git a/tests/test-prj-running.sh b/tests/test-prj-running.sh
--- a/tests/test-prj-running.sh
+++ b/tests/test-prj-running.sh
@@ -78,6 +78,6 @@
 
 # 生成摘要 JSON
 echo "==> Generating summary JSON: ${report_path}/ut-summary.json"
-python3 "${scriptdir}/gen-ut-summary.py"
+python3 "${scriptdir}/gen-ut-summary.py" || true
 
 exit $test_exit_code

@deepin-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: add-uos, lzwind

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@lzwind
lzwind merged commit 906601e into linuxdeepin:master Aug 13, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants