fix: wrap comma-containing field in quotes in v5.4 Field_Level CSV (Fixes #775, #744)#792
Open
rtmalikian wants to merge 1 commit into
Open
fix: wrap comma-containing field in quotes in v5.4 Field_Level CSV (Fixes #775, #744)#792rtmalikian wants to merge 1 commit into
rtmalikian wants to merge 1 commit into
Conversation
The userGuidance field for cdm_source.source_documentation_reference
contains a comma ('...source data, e.g. a data dictionary.') that was
not wrapped in quotes, causing CSV parsers to split it into 14 fields
instead of 13.
Adds double quotes around the field value to properly escape the comma.
Fixes OHDSI#775
Fixes OHDSI#744
Signed-off-by: Raphael Malikian <rtmalikian@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #775
Fixes #744
Problem
The
OMOP_CDMv5.4_Field_Level.csvfile has a CSV formatting error at thecdm_source.source_documentation_referencerow. TheuserGuidancefield value contains an unquoted comma:Standard CSV parsers (Python
csv, csvlint.io, GitHub's web UI) interpret the comma as a field delimiter, splitting this row into 14 fields instead of the expected 13. GitHub even displays a warning banner when viewing the file.The same issue does not exist in v5.3 or v6.0 — only v5.4 is affected.
Solution
Wrap the
userGuidancefield in double quotes to properly escape the embedded comma:Verification
csv.reader()now parses all 551 data rows with exactly 13 fields each (previously row 355 had 14)git diff --cachedconfirms CRLF line endings preserved (1 line changed, not a multi-line rewrite)inst/csv/OMOP_CDMv5.4_Field_Level.csvmodified — no other files affectedChangelog
Files Changed
inst/csv/OMOP_CDMv5.4_Field_Level.csv— Added quotes aroundsource_documentation_referenceuserGuidance field (line 399)Verification
csv.reader()parses all rows correctly (13 fields each)About the Author: Raphael Malikian — Clinical AI Solutions Architect. I specialise in building and fixing AI/ML systems for healthcare, including vector databases, RAG pipelines, and clinical NLP. If you need help with your project or think I can add value to your organisation, feel free to reach out — I'd love to connect.
📧 rtmalikian@gmail.com
🔗 GitHub: https://github.com/rtmalikian
🔗 LinkedIn: http://www.linkedin.com/in/raphael-t-malikian-mbbs-bsc-hons-71075436a
Disclosure: This code was developed with assistance from mimo-v2.5-pro (Xiaomi) via Hermes Agent (Nous Research). All changes were reviewed, tested against the actual codebase, and verified for correctness.