Skip to content

refactor: untrack generated python proto, regenerate in build#5077

Open
Ma77Ball wants to merge 10 commits into
apache:mainfrom
Ma77Ball:feat/changeProtobufGenerate
Open

refactor: untrack generated python proto, regenerate in build#5077
Ma77Ball wants to merge 10 commits into
apache:mainfrom
Ma77Ball:feat/changeProtobufGenerate

Conversation

@Ma77Ball
Copy link
Copy Markdown
Contributor

@Ma77Ball Ma77Ball commented May 15, 2026

What changes were proposed in this PR?

Stop version-tracking the betterproto-generated Python files under amber/src/main/python/proto/. Add a genPythonProto sbt task that runs bin/python-proto-gen.sh as a dependency of WorkflowExecutionService/Compile/compile, with a graceful skip when protoc is missing. Wire protoc 3.19.4 (matching PB.protocVersion) and betterproto[compiler]==2.0.0b7 into the three image build stages (worker,master, web-app) and the three CI jobs that touch Python proto (python, python-state-materialization-mac, amber-integration). The 14th tracked file (proto/org/apache/texera/web/__init__.py) had no corresponding .proto source and is gone after regen; grep confirms no callers.

Any related issues, documentation, or discussions?

Closes: #4102

How was this PR tested?

locally: sbt clean WorkflowExecutionService/compile regenerates the tree and pytest -m "not integration" passes; skip path verified with protoc absent. All three Docker images build end-to-end. CI covers the workflow YAML on
push.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude Opus 4.7 in compliance with ASF

@github-actions github-actions Bot added feature engine dependencies Pull requests that update a dependency file python ci changes related to CI docs Changes related to documentations dev labels May 15, 2026
@Ma77Ball
Copy link
Copy Markdown
Contributor Author

/request-review @Yicong-Huang

@github-actions github-actions Bot requested a review from Yicong-Huang May 15, 2026 11:18
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 15, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.51%. Comparing base (bf2f92c) to head (ec349cb).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5077      +/-   ##
============================================
+ Coverage     43.35%   43.51%   +0.16%     
+ Complexity     2212     2210       -2     
============================================
  Files          1049     1051       +2     
  Lines         40560    40685     +125     
  Branches       4322     4322              
============================================
+ Hits          17583    17703     +120     
- Misses        21886    21887       +1     
- Partials       1091     1095       +4     
Flag Coverage Δ
access-control-service 39.53% <ø> (ø)
agent-service 33.76% <ø> (ø)
amber 43.78% <ø> (-0.03%) ⬇️
computing-unit-managing-service 0.00% <ø> (ø)
config-service 0.00% <ø> (ø)
file-service 32.18% <ø> (ø)
frontend 34.61% <ø> (ø)
python 90.79% <ø> (+0.28%) ⬆️
workflow-compiling-service 56.81% <ø> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Yicong-Huang
Copy link
Copy Markdown
Contributor

Yicong-Huang commented May 15, 2026

Thanks, that's a good direction for change! Let inline comments.

Please make pr description concise. The "how you test" section is basically saying "ci" can verify. I don't get too much new info reading it.

Comment thread amber/build.sbt Outdated
Comment thread bin/computing-unit-master.dockerfile Outdated
@Yicong-Huang Yicong-Huang changed the title refactor: untrack generated python proto, regenerate in build [Hackathon] refactor: untrack generated python proto, regenerate in build May 15, 2026
@Ma77Ball Ma77Ball changed the title [Hackathon] refactor: untrack generated python proto, regenerate in build refactor: untrack generated python proto, regenerate in build May 15, 2026
@Ma77Ball Ma77Ball requested a review from Yicong-Huang May 22, 2026 08:11
@Ma77Ball
Copy link
Copy Markdown
Contributor Author

Ma77Ball commented May 22, 2026

@Yicong-Huang, I addressed the comments. Please review when available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci changes related to CI dependencies Pull requests that update a dependency file dev docs Changes related to documentations engine feature python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Untrack generated python protobuf files

3 participants