Skip to content

Log Analysis Introduced with backend_log_extraction.py#1748

Open
OmkarSarkar204 wants to merge 13 commits into
ArduPilot:masterfrom
OmkarSarkar204:log_analysis_feats
Open

Log Analysis Introduced with backend_log_extraction.py#1748
OmkarSarkar204 wants to merge 13 commits into
ArduPilot:masterfrom
OmkarSarkar204:log_analysis_feats

Conversation

@OmkarSarkar204

@OmkarSarkar204 OmkarSarkar204 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Description

Pr Introducing the Log_Analysis module for Ardupilot log, which will be used by and with AMC

Checklist

  • Run pre-commit checks locally
  • Verified by a human programmer
  • All commits are signed off (use git commit --signoff)
  • Code follows our coding standards
  • [] Documentation updated if needed
  • No breaking changes or properly documented

Testing

Describe how you tested these changes:

  • Unit tests pass( for existing files not real files)
  • Integration tests pass
  • Manual testing performed
  • Tested on flight controller hardware

Copilot AI review requested due to automatic review settings June 19, 2026 06:18
@OmkarSarkar204 OmkarSarkar204 marked this pull request as draft June 19, 2026 06:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Introduces a new log_analysis module to parse ArduPilot .bin logs, extracting parameter values/defaults, firmware version info, message counts, and frame type, and reuses that logic from extract_param_defaults.py.

Changes:

  • Added log_analysis/backend_log_extraction.py with log parsing + extraction helpers (params, firmware version, frame type).
  • Added package initializer log_analysis/__init__.py.
  • Refactored extract_param_defaults.py to reuse shared validation/version-parsing helpers from the new module.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 10 comments.

File Description
ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py New backend log extraction implementation (open/close log, parse messages, extract params/firmware/frame type).
ardupilot_methodic_configurator/log_analysis/init.py Initializes the new log_analysis package with a module docstring.
ardupilot_methodic_configurator/extract_param_defaults.py Replaces in-file regex/version parsing with imports from backend_log_extraction.

Comment thread ardupilot_methodic_configurator/extract_param_defaults.py Outdated
Comment thread ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py Outdated
Comment thread ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py Outdated
Comment thread ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py Outdated
Comment thread ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py Outdated
Comment thread ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py Outdated
Comment thread ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py Outdated
Comment thread ardupilot_methodic_configurator/log_analysis/__init__.py Outdated
Comment thread ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py Outdated
@OmkarSarkar204 OmkarSarkar204 force-pushed the log_analysis_feats branch 5 times, most recently from f4b3beb to b1e5229 Compare June 19, 2026 07:44
@OmkarSarkar204

Copy link
Copy Markdown
Contributor Author

The codespell was failing

codespell................................................................Failed
- hook id: codespell
- exit code: 65

ardupilot_methodic_configurator/data_model_configuration_step.py:351: SER ==> SET

So i just supressed it for now

Comment thread ardupilot_methodic_configurator/log_analysis/backend_log_extraction.py Outdated
@coveralls

coveralls commented Jun 19, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 28046795869

Coverage decreased (-0.7%) to 93.678%

Details

  • Coverage decreased (-0.7%) from the base build.
  • Patch coverage: No coverable lines changed in this PR.
  • 6 coverage regressions across 1 file.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

6 previously-covered lines in 1 file lost coverage.

File Lines Losing Coverage Coverage
extract_param_defaults.py 6 95.88%

Coverage Stats

Coverage Status
Relevant Lines: 13651
Covered Lines: 12788
Line Coverage: 93.68%
Relevant Branches: 2096
Covered Branches: 2014
Branch Coverage: 96.09%
Branches in Coverage %: No
Coverage Strength: 2.8 hits per line

💛 - Coveralls

@OmkarSarkar204 OmkarSarkar204 force-pushed the log_analysis_feats branch 2 times, most recently from bdf514b to 4239dca Compare June 19, 2026 14:39
@OmkarSarkar204 OmkarSarkar204 changed the title Log Analysis Introduced with ardupilot_log_extraction.py Log Analysis Introduced with backend_log_extraction.py Jun 20, 2026
@OmkarSarkar204 OmkarSarkar204 force-pushed the log_analysis_feats branch 4 times, most recently from f5a17b7 to 9c9ba7c Compare June 20, 2026 05:32
@amilcarlucas

Copy link
Copy Markdown
Collaborator

Thanks for this. I reviewed the code and did some improvements.

@OmkarSarkar204

OmkarSarkar204 commented Jun 20, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for this. I reviewed the code and did some improvements.

I was thinking of adding the test once ive finished the basic signals, but Thank You , Im working on the PM, IMU , VIBE and ERR messages and will try to do it by today midnight. :)

@OmkarSarkar204

Copy link
Copy Markdown
Contributor Author

Extracted till now
PARM ,VER, MSG (version fallback), BAT, PM, IMU, VIBE, GPS, ERR, ARM, MODE

@OmkarSarkar204 OmkarSarkar204 force-pushed the log_analysis_feats branch 3 times, most recently from e32105f to 871c7f6 Compare June 22, 2026 07:37

@amilcarlucas amilcarlucas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. By plugins I meant: ARCHITECTURE_battery_monitor.md and ARCHITECTURE_motor_test.md

Comment thread ardupilot_methodic_configurator/log_analysis/analysis_plugins.json Outdated
@OmkarSarkar204 OmkarSarkar204 force-pushed the log_analysis_feats branch 2 times, most recently from 47ae7f1 to c70df12 Compare June 28, 2026 17:13
@amilcarlucas

Copy link
Copy Markdown
Collaborator

This looks good, we can merge it in.

OmkarSarkar204 and others added 9 commits July 1, 2026 19:47
… param_default to remove code duplication.

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
@OmkarSarkar204

OmkarSarkar204 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

This looks good, we can merge it in.

Thank You! I updated the pr and did some minor commnets changes :)

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Comment thread REUSE.toml
@OmkarSarkar204

Copy link
Copy Markdown
Contributor Author

Thank you for the tests i corrected the commit and fixed the ci failures should be passing now

@amilcarlucas amilcarlucas requested a review from Copilot July 2, 2026 10:41
@amilcarlucas

Copy link
Copy Markdown
Collaborator

Thanks. I requested a github copilot code review. It that has no findings, we can merge this.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 5 comments.

Comment thread ardupilot_methodic_configurator/log_analysis/__init__.py Outdated
Comment thread tests/integration_backend_log_extraction.py
Comment thread tests/test_extract_param_defaults.py
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Amilcar Lucas <amilcar.lucas@iav.de>
@OmkarSarkar204

OmkarSarkar204 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

@amilcarlucas , IS this ok, please let me know if any other changes required?

Signed-off-by: Omkar Sarkar <omkarsarkar24@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants