test(models): add smoke test that every DH/URDF/ETS model constructs#547
Merged
Conversation
Existing tests are hand-written per model, so a model exported in __all__ can go completely untested -- Valkyrie, Fetch, KinovaGen3, FetchCamera, and LBR all had zero coverage this way, three of them silently broken with nothing to catch it (see tech-debt.md). Iterates each category's own __all__ directly (rather than scanning module __dict__/inspect.getmembers, which would also pick up re-exported base classes like Robot/URDFRobot) and asserts every listed class constructs with no arguments. Known-currently-broken models are tracked in an explicit EXPECTED_FAILURES set, which fails loudly in *both* directions: a newly-broken model fails the test, and a since-fixed "expected failure" that starts passing also fails the test, forcing its entry to be cleaned up rather than silently going stale. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Valkyrie, Fetch, and KinovaGen3 are now fixed on main; FetchCamera is removed entirely (so its entry would be a harmless no-op, but delete it for hygiene). Only LBR remains genuinely broken.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #547 +/- ##
=====================================
Coverage 0.00% 0.00%
=====================================
Files 136 136
Lines 13438 13438
=====================================
Misses 13438 13438 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/test_models.pyis entirely hand-written, one method per model — so a model exported in__all__can go completely untested.Valkyrie,Fetch,KinovaGen3,FetchCamera, andLBRall had zero coverage this way; three of them were silently broken with nothing to catch it (see tech-debt.md, and fix(models): patch broken upstream RD files for Valkyrie and Fetch #543/fix(models): remove FetchCamera, unshippable and unused in RVC3 #545/fix(models): make KinovaGen3 load via robot_descriptions, add XACRO_ARGS support #546).TestModelSmoke.test_all_models_constructthat iterates each ofDH/URDF/ETS's own__all__(notinspect.getmembers/module__dict__scanning, which would also pick up re-exported base classes likeRobot/URDFRobot) and asserts every listed class constructs with no arguments.EXPECTED_FAILURESset. This fails loudly in both directions: a newly-broken model fails the test, and a since-fixed "expected failure" that starts passing also fails the test — forcing whoever fixes it to clean up the entry rather than letting it go stale.EXPECTED_FAILUREScurrently lists 5 entries:LBR(needs an rtb-data release, tracked in tech-debt.md) plusValkyrie/Fetch/KinovaGen3/FetchCamera, all already fixed on unmerged branches (fix(models): patch broken upstream RD files for Valkyrie and Fetch #543, fix(models): remove FetchCamera, unshippable and unused in RVC3 #545, fix(models): make KinovaGen3 load via robot_descriptions, add XACRO_ARGS support #546) — each entry documents which PR resolves it, for easy cleanup once merged.Test plan
main: 29/29 pass intest_models.py, only the 5 documented models hitEXPECTED_FAILURESEXPECTED_FAILURESand confirmed the test fails, listing exactly the 5 known-broken models with their real exceptions — nothing else🤖 Generated with Claude Code