fix(yolo): correct data.yaml paths for all YOLO formats (#240)#506
Open
davidnichols-ops wants to merge 1 commit into
Open
fix(yolo): correct data.yaml paths for all YOLO formats (#240)#506davidnichols-ops wants to merge 1 commit into
davidnichols-ops wants to merge 1 commit into
Conversation
Author
|
Thanks for the review consideration! Quick summary of what this PR does and why it's a clean merge: Scope: Single method ( The fix addresses 4 bugs in one method:
Verification:
Happy to address any feedback! |
The __reformat_yaml method had four bugs that caused broken training
paths in downloaded YOLO datasets:
1. yolov8 path fix was gated on ultralytics==8.0.196 -- for any other
version (or when ultralytics was not installed) paths remained as
../train/images, ../valid/images, ../test/images, which are wrong
because the split dirs are siblings of data.yaml, not in the parent.
2. yolov9 was listed in the amend formats but had no callback case --
the YAML was read and rewritten with zero changes.
3. lstrip("..") was used to strip ../ prefixes, but str.lstrip strips
any *combination* of the given characters, not a prefix. This
silently mangled paths like ../../train/images.
4. The test key was always written even when the dataset had no test
split, producing a path to a non-existent directory.
Fix:
- Remove the ultralytics version gate entirely; paths are now always
corrected for yolov8 and yolov9.
- Add yolov9 to the relative-path branch.
- Replace lstrip with a proper _strip_rel_prefix helper that iteratively
removes ../ and ./ prefixes.
- Use self.splits["test"] > 0 to decide whether to emit the test key;
remove it when the split is absent.
- Use os.path.join instead of string concatenation for absolute paths.
Tests: 13 new tests in TestReformatYaml covering yolov8/yolov9 relative
paths, legacy absolute paths, test-split presence/absence, ultralytics
independence, double-parent-prefix edge case, and preservation of other
YAML fields. All 804 existing tests still pass.
Closes roboflow#240
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
d7c429f to
8bbf8fa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #240 — Incorrect Data Path in YOLOv8 Dataset Configuration (5 comments, confirmed regression from v1.1.40 → v1.2.16, reported by multiple users).
When downloading a YOLO dataset via
version.download("yolov8"), the generateddata.yamlcontained incorrect relative paths:The
../prefix is wrong — thetrain/,valid/,test/directories are siblings ofdata.yaml, not in the parent directory. This broke YOLO training out of the box.Root cause — four bugs in
__reformat_yamlyolov8 path fix was gated on
ultralytics==8.0.196— for any other version (or when ultralytics wasn't installed at all) paths remained broken. This was the primary cause of the regression.yolov9 had no path fix at all — it was listed in the amend formats list but had no callback case, so the YAML was read and rewritten with zero changes.
lstrip("..")was misused —str.lstripstrips any combination of the given characters, not a prefix. This silently mangled paths like../../train/imagesand.hidden/train/images.testkey was always written even when the dataset had no test split, producing a path to a non-existent directory.Fix
train/images,valid/images,test/images).lstripwith_strip_rel_prefix— a proper helper that iteratively removes../and./prefixes without mangling legitimate path characters.self.splits["test"] > 0to decide whether to emit thetestkey; it is removed when the split is absent.os.path.joininstead of string concatenation for absolute paths in legacy formats.Behavior matrix
yolov8../train/images(broken unless ultralytics==8.0.196)train/imagesyolov9../train/images(always broken)train/imagesyolov5pytorch<location>train/images(lstrip bug)<location>/train/imagesyolov7pytorch<location>train/images(lstrip bug)<location>/train/imagesmt-yolov6<location>train/images(lstrip bug)<location>/train/imagesTest plan
TestReformatYamlcovering:../prefix)testkey when no test split existstestkey when no test split exists../../double-parent prefix is correctly strippedcoco) are not modifiednc,names,roboflow) are preservedruff format --checkpassesruff checkpassesmypy roboflowpassesGenerated with Devin