[None][infra] AutoDeploy: Add trtllm runner for standalone llm-c by bmarimuthu-nv · Pull Request #15630 · NVIDIA/TensorRT-LLM

bmarimuthu-nv · 2026-06-25T15:36:55Z

Summary by CodeRabbit

New Features
- Standalone packages now bundle Tier-1 e2e harness sources from the auto-deploy examples.
- Runner files are included under a dedicated runners/trtllm location, including an updated entrypoint name and run instruction.
Bug Fixes
- Ensures the copied runner Python imports resolve correctly inside the standalone package.
- Preserves model registry content as-is during packaging.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Signed-off-by: Balamurugan Marimuthu <246387390+bmarimuthu-nv@users.noreply.github.com>

bmarimuthu-nv · 2026-06-25T15:41:56Z

@CodeRabbit summary

coderabbitai · 2026-06-25T15:42:01Z

✅ Action performed

Summary regeneration triggered.

coderabbitai · 2026-06-25T15:43:07Z

📝 Walkthrough

Walkthrough

The standalone package generator now copies the examples/auto_deploy runner harness into runners/, renames the main entrypoint, rewrites copied Python imports to llmc, and updates the generated e2e invocation text.

Changes

Standalone package runner harness

Layer / File(s)	Summary
Runner source setup `examples/auto_deploy/llmc/create_standalone_package.py`	Adds source path constants, keeps `runners/` in the managed output paths, and defines `_copy_runners()` to copy the example harness files and `model_registry/` into the packaged output.
Packaging step and usage text `examples/auto_deploy/llmc/create_standalone_package.py`	Copies the runners during packaging, rewrites copied Python imports from `tensorrt_llm._torch.auto_deploy` to `llmc`, logs completion, and updates the final e2e command text.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is only the template with no actual issue, solution, or test coverage details.	Add a short Description explaining the problem and fix, plus concrete Test Coverage entries for the new runner path.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the main change: adding a standalone trtllm runner for llm-c.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

Signed-off-by: Balamurugan Marimuthu <246387390+bmarimuthu-nv@users.noreply.github.com>

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/auto_deploy/llmc/create_standalone_package.py`:
- Around line 505-514: The _copy_runners() helper is silently skipping missing
required runner sources, which can produce an incomplete standalone package.
Update the logic that iterates over EXAMPLE_FILES and EXAMPLE_DIRS to fail fast
by raising an error as soon as an expected file or directory under
AD_EXAMPLES_SRC is absent, instead of continuing and returning a partial count.
Keep the existing copy behavior for present sources, but make the missing-source
case explicit in _copy_runners().

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7c455f11-03dc-478a-941c-d1d22946d530

📥 Commits

Reviewing files that changed from the base of the PR and between 85fc146 and 257c25d.

📒 Files selected for processing (1)

examples/auto_deploy/llmc/create_standalone_package.py

coderabbitai · 2026-06-25T16:09:31Z

+    for src_name, dst_name in EXAMPLE_FILES.items():
+        src = os.path.join(AD_EXAMPLES_SRC, src_name)
+        if os.path.isfile(src):
+            os.makedirs(runners_dst, exist_ok=True)
+            shutil.copy2(src, os.path.join(runners_dst, dst_name))
+            count += 1
+    for dname in EXAMPLE_DIRS:
+        src = os.path.join(AD_EXAMPLES_SRC, dname)
+        if os.path.isdir(src):
+            count += _copy_tree(src, os.path.join(runners_dst, dname))


🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Fail fast when expected runner sources are missing (Line 507, Line 513).

_copy_runners() currently skips missing required sources and continues, which can silently generate a broken standalone package. Raise immediately when required runner files/directories are absent.

Proposed fix

def _copy_runners(output_dir: str) -> int: @@ for src_name, dst_name in EXAMPLE_FILES.items(): src = os.path.join(AD_EXAMPLES_SRC, src_name) - if os.path.isfile(src): - os.makedirs(runners_dst, exist_ok=True) - shutil.copy2(src, os.path.join(runners_dst, dst_name)) - count += 1 + if not os.path.isfile(src): + raise FileNotFoundError(f"Missing required runner file: {src}") + os.makedirs(runners_dst, exist_ok=True) + shutil.copy2(src, os.path.join(runners_dst, dst_name)) + count += 1 for dname in EXAMPLE_DIRS: src = os.path.join(AD_EXAMPLES_SRC, dname) - if os.path.isdir(src): - count += _copy_tree(src, os.path.join(runners_dst, dname)) + if not os.path.isdir(src): + raise FileNotFoundError(f"Missing required runner directory: {src}") + count += _copy_tree(src, os.path.join(runners_dst, dname)) return count

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

for src_name, dst_name in EXAMPLE_FILES.items():

src = os.path.join(AD_EXAMPLES_SRC, src_name)

if os.path.isfile(src):

os.makedirs(runners_dst, exist_ok=True)

shutil.copy2(src, os.path.join(runners_dst, dst_name))

count += 1

for dname in EXAMPLE_DIRS:

src = os.path.join(AD_EXAMPLES_SRC, dname)

if os.path.isdir(src):

count += _copy_tree(src, os.path.join(runners_dst, dname))

for src_name, dst_name in EXAMPLE_FILES.items():

src = os.path.join(AD_EXAMPLES_SRC, src_name)

if not os.path.isfile(src):

raise FileNotFoundError(f"Missing required runner file: {src}")

os.makedirs(runners_dst, exist_ok=True)

shutil.copy2(src, os.path.join(runners_dst, dst_name))

count += 1

for dname in EXAMPLE_DIRS:

src = os.path.join(AD_EXAMPLES_SRC, dname)

if not os.path.isdir(src):

raise FileNotFoundError(f"Missing required runner directory: {src}")

count += _copy_tree(src, os.path.join(runners_dst, dname))

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/auto_deploy/llmc/create_standalone_package.py` around lines 505 - 514, The _copy_runners() helper is silently skipping missing required runner sources, which can produce an incomplete standalone package. Update the logic that iterates over EXAMPLE_FILES and EXAMPLE_DIRS to fail fast by raising an error as soon as an expected file or directory under AD_EXAMPLES_SRC is absent, instead of continuing and returning a partial count. Keep the existing copy behavior for present sources, but make the missing-source case explicit in _copy_runners().

bmarimuthu-nv · 2026-06-25T19:08:54Z

/bot run

tensorrt-cicd · 2026-06-25T19:15:06Z

PR_Github #55868 [ run ] triggered by Bot. Commit: 257c25d Link to invocation

github-actions Bot assigned bmarimuthu-nv Jun 25, 2026

bmarimuthu-nv force-pushed the bala/llmc-e2e branch from 00dde4e to 750b291 Compare June 25, 2026 15:39

llm-c add trtllm runner

fd91b34

Signed-off-by: Balamurugan Marimuthu <246387390+bmarimuthu-nv@users.noreply.github.com>

bmarimuthu-nv force-pushed the bala/llmc-e2e branch from 750b291 to fd91b34 Compare June 25, 2026 15:40

bmarimuthu-nv added 2 commits June 25, 2026 08:59

fix runner path

58c61f5

Signed-off-by: Balamurugan Marimuthu <246387390+bmarimuthu-nv@users.noreply.github.com>

lint

257c25d

Signed-off-by: Balamurugan Marimuthu <246387390+bmarimuthu-nv@users.noreply.github.com>

bmarimuthu-nv marked this pull request as ready for review June 25, 2026 16:06

bmarimuthu-nv requested a review from a team as a code owner June 25, 2026 16:06

bmarimuthu-nv requested a review from nvchenghaoz June 25, 2026 16:06

coderabbitai Bot reviewed Jun 25, 2026

View reviewed changes

suyoggupta approved these changes Jun 25, 2026

View reviewed changes

bmarimuthu-nv enabled auto-merge (squash) June 25, 2026 19:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[None][infra] AutoDeploy: Add trtllm runner for standalone llm-c#15630

[None][infra] AutoDeploy: Add trtllm runner for standalone llm-c#15630
bmarimuthu-nv wants to merge 3 commits into
NVIDIA:mainfrom
nv-auto-deploy:bala/llmc-e2e

bmarimuthu-nv commented Jun 25, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

bmarimuthu-nv commented Jun 25, 2026

Uh oh!

coderabbitai Bot commented Jun 25, 2026

Uh oh!

coderabbitai Bot commented Jun 25, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

coderabbitai Bot Jun 25, 2026

Uh oh!

bmarimuthu-nv commented Jun 25, 2026

Uh oh!

tensorrt-cicd commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

bmarimuthu-nv commented Jun 25, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

bmarimuthu-nv commented Jun 25, 2026

Uh oh!

coderabbitai Bot commented Jun 25, 2026

Uh oh!

coderabbitai Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

bmarimuthu-nv commented Jun 25, 2026

Uh oh!

tensorrt-cicd commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bmarimuthu-nv commented Jun 25, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 25, 2026 •

edited

Loading