translations: add "make prune-translations" to remove stale translations#22392
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: developowl The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Can one of the admins verify this patch? |
b45f92c to
98439bc
Compare
medyagh
left a comment
There was a problem hiding this comment.
@developowl thank you for this PR
plz fix the lint and gomft
Please run 'make fmt' and commit the changes
There was a problem hiding this comment.
Pull request overview
This PR introduces a new prune-translations command to remove stale translation keys from language JSON files, implementing the Single Responsibility Principle by separating cleanup logic from the extraction process.
Key Changes:
- Removed automatic cleanup logic from the
extractcommand that previously deleted empty, unused translation keys - Created new
prune-translationscommand that removes all keys from translation JSON files that are not present instrings.txt - Added Makefile targets for
prune-translations(standalone) andupdate-translations(combined extract + prune workflow)
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.
| File | Description |
|---|---|
| pkg/minikube/extract/extract.go | Removed the automatic cleanup logic that deleted empty, unused translations during extraction |
| cmd/prune-translations/prune-translations.go | New command that reads valid keys from strings.txt and removes stale keys from all translation JSON files |
| Makefile | Added prune-translations target to run the pruning tool and update-translations target to execute both extract and prune operations |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
medyagh
left a comment
There was a problem hiding this comment.
@developowl I think this would be cool to be added to our
"make generate-docs"
so when this job runs
minikube/.github/workflows/docs.yml
Line 25 in 3d9b452
it also includes translations
| // This tool removes stale translations (keys not in strings.txt) from language JSON files. | ||
| // Usage: go run cmd/prune-translations/prune-translations.go (from minikube root) | ||
|
|
||
| package main |
There was a problem hiding this comment.
how about add a unit test for this , and you can make dummy test data and expect it to to do the right thing, and make sure we dont ever delete real translations by mistake
There was a problem hiding this comment.
@medyagh
I wrote a unit test. I added dummy data to each unit test function. Is this correct? Please review my unit test. :)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Read valid keys from strings.txt | ||
| data, err := os.ReadFile("translations/strings.txt") | ||
| if err != nil { | ||
| panic("Run from minikube root directory. strings.txt not found.") |
There was a problem hiding this comment.
The error_message "Run from minikube root directory. strings.txt not found." is unclear about the actual problem. If the file exists but has read permissions issues, this message would be misleading. Consider using a message that indicates the actual error or providing the original error details.
| panic("Run from minikube root directory. strings.txt not found.") | |
| panic(fmt.Errorf("failed to read translations/strings.txt: %w", err)) |
- Extract PruneTranslations() function with configurable directory path - Add PruneResult struct to return detailed results - Extract pruneFile() helper function - Improve error handling with wrapped errors This refactoring enables unit testing without touching real translation files.
Add comprehensive unit tests using temporary directories to ensure we never accidentally delete real translations: - TestPruneTranslations_RemovesStaleKeys: verify stale keys are removed - TestPruneTranslations_PreservesValidKeys: verify valid keys are kept - TestPruneTranslations_MultipleFiles: test multiple file handling - TestPruneTranslations_EmptyTranslationFile: edge case handling - TestPruneTranslations_MissingStringsFile: error handling - TestPruneTranslations_InvalidStringsJSON: error handling - TestPruneTranslations_DoesNotTouchRealTranslations: safety check
developowl
left a comment
There was a problem hiding this comment.
I wrote a unit test. I added dummy data to each unit test function. Is this correct? Please review my unit test. :)
|
@medyagh Test cases:
All tests use Run tests: go test -v ./cmd/prune-translations/... |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| func main() { | ||
| results, err := PruneTranslations("translations") | ||
| if err != nil { | ||
| panic(err) |
There was a problem hiding this comment.
Using panic for error handling in main() is not idiomatic Go. Consider using log.Fatalf or os.Exit instead to provide better error messages and stack traces. For example: log.Fatalf(\"Failed to prune translations: %v\", err)
There was a problem hiding this comment.
@copilot open a new pull request to apply changes based on this feedback
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
@developowl: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
thank you @developowl |
fixes #21468
Problem(Issue)
Each language.json contained duplicate and orphan keys (keys that were deleted from strings.txt but still persisted in the JSON files).
Proposed Solution
strings.txtwith each language JSON file to identify duplicates and orphan keys.strings.txtto individual translation files.string: translationpairs are preserved during the cleanup process.cmd/directory.Makefiletarget for easy execution.Implementation Details
Analyzing the current system architecture.
extract.go: extract logic filewriteStringsToFiles: strings to file function🚨 Root Cause
!ok: The key is missing from e.translations (it is no longer used in the current source code).len == 0: The translation value is an empty string.Data Retention Strategy
To ensure that we don't lose any existing translations, I've implemented a strict pruning logic. As shown in the table below, even if a string is no longer detected in the source code (!ok), it is preserved if it already has a translated value. Only unused strings with empty values are removed.
Solution 1) Easy version (I don't choose.)
Solution 2) create
prune-translations&update-translationsalso, removed the deletion logic from the extractAdhering to the Single Responsibility Principle (SRP):
extract: Handles only string extraction from the source code.prune-translations: Handles only the cleanup (pruning) of obsolete or orphan keys.update-translations: A wrapper command that performs the entire workflow (extraction and pruning) at once.prune-translations Job
key countsof each JSON file with strings.txt after the cleanup.Add
prune-translations,update-translationstarget into MakefileTest!
Test Case 1)
extract->prune-translationsTest Case 2) Only
update-translationsAfter Test
ko.jsonTo maintainer.
@medyagh
My apologies for the delay due to personal reasons. I have remained committed to this issue and have finally completed the task. While I considered Solution 1) as a simpler alternative, I felt it assigned too many responsibilities to the extract command alone. I would appreciate it if you could positively review the
prune-translationsandupdate-translationscommands I have implemented. Thank you for your patience.🥝