Skip to content

Log Analysis: BAT, IMU and ESC analysis modules - #1955

Open
OmkarSarkar204 wants to merge 4 commits into
ArduPilot:masterfrom
OmkarSarkar204:log-analysis-base
Open

Log Analysis: BAT, IMU and ESC analysis modules#1955
OmkarSarkar204 wants to merge 4 commits into
ArduPilot:masterfrom
OmkarSarkar204:log-analysis-base

Conversation

@OmkarSarkar204

@OmkarSarkar204 OmkarSarkar204 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Description

Log Analysis Checks for BAT, IMU temp and ESC calibration

Checklist

  • Run pre-commit checks locally
  • Verified by a human programmer
  • All commits are signed off (use git commit --signoff)
  • Code follows our coding standards
  • Documentation updated if needed
  • No breaking changes or properly documented

Testing

Describe how you tested these changes:

  • Unit tests pass
  • Integration tests pass
  • Manual testing performed
  • Tested on flight controller hardware

@coveralls

coveralls commented Aug 17, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 32549536820

Coverage decreased (-0.6%) to 90.2%

Details

  • Coverage decreased (-0.6%) from the base build.
  • Patch coverage: No coverable lines changed in this PR.
  • 283 coverage regressions across 11 files.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

283 previously-covered lines in 11 files lost coverage.

Top 10 Files by Coverage Loss Lines Losing Coverage Coverage
log_analysis/data_model_quality_esc.py 157 18.37%
log_analysis/data_model_quality_vibe.py 47 26.47%
log_analysis/data_model_quality_battery.py 20 84.14%
log_analysis/backend_log_extraction.py 15 84.65%
log_analysis/data_model_quality_base.py 14 83.87%
log_analysis/data_model_quality_imu.py 12 88.18%
backend_filesystem.py 6 92.62%
log_analysis/data_model_log_analysis.py 3 94.94%
log_analysis/utils.py 3 91.11%
backend_filesystem_configuration_steps.py 3 96.47%

Coverage Stats

Coverage Status
Relevant Lines: 17928
Covered Lines: 16171
Line Coverage: 90.2%
Relevant Branches: 5360
Covered Branches: 4491
Branch Coverage: 83.79%
Branches in Coverage %: No
Coverage Strength: 2.69 hits per line

💛 - Coveralls

@OmkarSarkar204
OmkarSarkar204 force-pushed the log-analysis-base branch 2 times, most recently from 0838bfc to a06deec Compare August 18, 2026 06:52

@amilcarlucas amilcarlucas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The battery stuff is looking good, you are detecting many misconfiguration issues already. Keep it up!

Comment thread ardupilot_methodic_configurator/log_analysis/backend_data_sources.py Outdated
@OmkarSarkar204
OmkarSarkar204 force-pushed the log-analysis-base branch 4 times, most recently from f4bd02a to 12f655c Compare August 19, 2026 05:05
@OmkarSarkar204
OmkarSarkar204 marked this pull request as ready for review August 19, 2026 05:06
@OmkarSarkar204
OmkarSarkar204 requested review from amilcarlucas and a lite review from Copilot August 19, 2026 05:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Adds new battery and IMU log analysis checks (battery health/parameter analysis and IMU temperature calibration analysis) while refactoring the log-quality model base class to support configuration-step derived/forced parameter evaluation.

Changes:

  • Introduces BatteryLogAnalysis and ImuLogAnalysis plus comprehensive new pytest suites for both.
  • Refactors BaseLogQualityAnalysisModel into BaseLogModel, adding helpers to evaluate derived/forced parameters per configuration step.
  • Updates config-step processing to compute forced+derived parameters in a single call path.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/test_data_model_log_analysis.py Updates tests to use BaseLogModel instead of the old base class.
tests/test_data_model_imu_analysis.py Adds end-to-end style tests for IMU quality + temperature calibration analysis.
tests/test_data_model_configuration_step.py Updates mocks/assertions for the new combined forced+derived computation API.
tests/test_data_model_battery_analysis.py Adds end-to-end style tests for battery quality + analysis models.
ardupilot_methodic_configurator/log_analysis/data_model_quality_vibe.py Migrates quality model to BaseLogModel.
ardupilot_methodic_configurator/log_analysis/data_model_quality_pm.py Migrates quality model to BaseLogModel.
ardupilot_methodic_configurator/log_analysis/data_model_quality_mode.py Migrates quality model to BaseLogModel.
ardupilot_methodic_configurator/log_analysis/data_model_quality_imu.py Adds IMU temperature calibration analysis and migrates quality model base.
ardupilot_methodic_configurator/log_analysis/data_model_quality_gnss.py Migrates quality model to BaseLogModel.
ardupilot_methodic_configurator/log_analysis/data_model_quality_fft.py Migrates quality model to BaseLogModel.
ardupilot_methodic_configurator/log_analysis/data_model_quality_esc.py Migrates quality model to BaseLogModel.
ardupilot_methodic_configurator/log_analysis/data_model_quality_err.py Migrates quality model to BaseLogModel.
ardupilot_methodic_configurator/log_analysis/data_model_quality_battery.py Adds battery analysis model + refactors battery quality checks.
ardupilot_methodic_configurator/log_analysis/data_model_quality_base.py Introduces BaseLogModel + helpers for expected/derived parameter evaluation.
ardupilot_methodic_configurator/log_analysis/data_model_quality_arm.py Migrates quality model to BaseLogModel.
ardupilot_methodic_configurator/log_analysis/data_model_log_quality.py Extends QualityIssue and LogQualityResult with param/suggestion + related step.
ardupilot_methodic_configurator/log_analysis/data_model_log_analysis_result.py Adds structured result types for analysis outcomes.
ardupilot_methodic_configurator/log_analysis/data_model_log_analysis.py Updates typing/imports to align with BaseLogModel.
ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py Adjusts multiplier resolution for pymavlink fixed-point fields.
ardupilot_methodic_configurator/data_model_parameter_editor.py Passes nested Components dict into log analysis inputs.
ardupilot_methodic_configurator/data_model_configuration_step.py Switches to compute_forced_and_derived_parameters and simplifies error handling.
ardupilot_methodic_configurator/backend_filesystem_configuration_steps.py Adds compute_forced_and_derived_parameters() helper API.
ardupilot_methodic_configurator/backend_filesystem.py Uses new combined forced+derived computation helper.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread ardupilot_methodic_configurator/log_analysis/data_model_quality_base.py Outdated
Comment thread ardupilot_methodic_configurator/log_analysis/data_model_quality_imu.py Outdated
Comment thread tests/test_data_model_battery_analysis.py
Comment thread tests/test_data_model_battery_analysis.py
@OmkarSarkar204 OmkarSarkar204 changed the title feat: log analysis battery and IMU temp analysis and changing BaseLog… Log Analysis: BAT and IMU analysis modules Aug 19, 2026
@OmkarSarkar204
OmkarSarkar204 force-pushed the log-analysis-base branch 8 times, most recently from 08668a6 to df2d1b3 Compare August 21, 2026 04:52
amilcarlucas
amilcarlucas previously approved these changes Aug 21, 2026
@amilcarlucas

Copy link
Copy Markdown
Collaborator

High — custom quality_models callers now crash
data_model_log_analysis.py (line 139) still documents/types quality_models as a list of model classes, but the loop now expects (quality_model, analysis_model) tuples at line 170. Passing the previously supported [SomeQualityModel] causes tuple-unpacking failure. Mypy also reports this mismatch.
Fix by either preserving class-list compatibility or changing the API/type/documentation consistently.

Medium — IMU calibration analysis falsely rejects valid data without apm_doc
data_model_quality_imu.py (line 198) treats every enabled value as unexpected when apm_doc is None, because tcal_enabled_codes() returns an empty set. Since apm_doc is explicitly optional, a valid INS_TCALn_ENABLE=1 is reported as invalid and its calibration range is not analyzed.
Accept known ArduPilot defaults when metadata is unavailable, or skip only metadata-dependent validation.

@OmkarSarkar204

OmkarSarkar204 commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

High — custom quality_models callers now crash data_model_log_analysis.py (line 139) still documents/types quality_models as a list of model classes, but the loop now expects (quality_model, analysis_model) tuples at line 170. Passing the previously supported [SomeQualityModel] causes tuple-unpacking failure. Mypy also reports this mismatch. Fix by either preserving class-list compatibility or changing the API/type/documentation consistently.

Medium — IMU calibration analysis falsely rejects valid data without apm_doc data_model_quality_imu.py (line 198) treats every enabled value as unexpected when apm_doc is None, because tcal_enabled_codes() returns an empty set. Since apm_doc is explicitly optional, a valid INS_TCALn_ENABLE=1 is reported as invalid and its calibration range is not analyzed. Accept known ArduPilot defaults when metadata is unavailable, or skip only metadata-dependent validation.

Thank You, so much for this, fixed and rebased

…QualityAnalysisModel to BaseLogModel to avoid naming confusion

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

fix: bat analysis with some inconsitency

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

fix: match the format of configuration steps to compare paramters and compute derived and forced parameters and add the imu analysis

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

test: fix test

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

test: added tests for BAT and IMU analysis

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

test: fix tests

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

test: fix tests

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

test: fix tests

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

test: fix tests

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

test: fix tests

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>

test: fix tests

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
@OmkarSarkar204
OmkarSarkar204 force-pushed the log-analysis-base branch 2 times, most recently from b58e73a to 0b6cfcd Compare August 21, 2026 15:37
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
@OmkarSarkar204 OmkarSarkar204 changed the title Log Analysis: BAT and IMU analysis modules Log Analysis: BAT, IMU and ESC analysis modules Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants