Skip to content

[0.5.0] Coordinate precision error, with actionable messages and consistent error messages - #80

Merged
sujata-m merged 1 commit into
mainfrom
feature-coordinate-precision-errors
Aug 5, 2026
Merged

[0.5.0] Coordinate precision error, with actionable messages and consistent error messages#80
sujata-m merged 1 commit into
mainfrom
feature-coordinate-precision-errors

Conversation

@sujata-m

@sujata-m sujata-m commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Promotes over-precise coordinates from a non-blocking warning to a validation error, makes every error report a real filename, and adds remediation guidance pointing users at the OSW data wizard. Version bumped to 0.5.0 for the breaking changes below.

Coordinate precision is a validation error

Coordinates carrying more than coordinate_precision decimal places (default 7) now fail validation instead of producing a warning. Each offending feature is reported individually with its filename and feature index, and validation stops before schema checks when any are found.

Feature 12 in 'opensidewalks.edges.geojson' contains coordinates with more than 7 decimal places. Reduce them to at most 7 decimal places; you can use the OSW data wizard tool to clean this up.

Precision is judged on written decimal places, never on how close a value is to a shorter one — 48.9999999999, 49.0000000001, and 49.00000000 are all rejected even though each is 49.0 or a hair from it. Anything with 7 or fewer decimals (49, 49.0, 49.0000000) passes. Exponent notation is measured after normalization.

Errors always name a real file

issues[].filename was inconsistent: some errors reported the internal dataset key (edges, zones), some reported an All placeholder, and only some reported the actual filename. Every error now carries the GeoJSON file the problem was found in, and error text that named a dataset key names the file too:

Check Before After
_u_id/_v_id/_w_id coordinate mismatch edges / zones opensidewalks.edges.geojson
Duplicate _ids nodes nodes.geojson
Vertex limit, collapsed geometry, missing column dataset key filename
Invalid geometries, unmatched references All file holding the bad values

Remediation guidance

Both precision errors and ext:* null/NaN errors now tell users how to fix the data and point at the OSW data wizard.

allow_zero_length_lines now defaults to True

Collapsed LineString geometries in edges, lines, and external line data are accepted by default. Set allow_zero_length_lines=False to reject them. Zero-area polygons and zones remain invalid regardless.

Verified, no code change needed

Two behaviors were confirmed already correct and are now pinned by tests:

  • Node references have no tolerance. _u_id, _v_id, and _w_id must match their node coordinates exactly. Deviations of 1e-7, 5e-8, 1e-9, and 1e-12 are all rejected, on either axis.
  • _u_id may equal _v_id. No such constraint exists in the validator or the schemas. A self-loop with real length always passes; one collapsed to a point is governed solely by allow_zero_length_lines and is reported as a geometry problem, never as a reference problem.

Breaking changes

  • ValidationResult.warnings and the COORDINATE_PRECISION_WARNING constant are removed. The precision message is now part of errors/issues.
  • Datasets with more than 7 decimal places of coordinate precision that previously passed will now fail. This is common in real-world data.
  • Consumers matching on filename == 'edges' or filename == 'All' must be updated to match filenames.
  • allow_zero_length_lines default flipped from False to True.

Testing

 > coverage run --source=src/python_osw_validation -m unittest discover -v tests/unit_tests

----------------------------------------------------------------------
Ran 231 tests in 39.524s

OK
> coverage report
Name                                                      Stmts   Miss  Cover
-----------------------------------------------------------------------------
src/python_osw_validation/__init__.py                       570     43    92%
src/python_osw_validation/config.py                          20      0   100%
src/python_osw_validation/extracted_data_validator.py        99      1    99%
src/python_osw_validation/geometry_mapping_validator.py     160     26    84%
src/python_osw_validation/helpers.py                        202     22    89%
src/python_osw_validation/zipfile_handler.py                 48      1    98%
-----------------------------------------------------------------------------
TOTAL                                                      1099     93    92%

…istent error messages

## Summary

Promotes over-precise coordinates from a non-blocking warning to a validation error, makes every error report a real filename, and adds remediation guidance pointing users at the OSW data wizard. Version bumped to `0.5.0` for the breaking changes below.

### Coordinate precision is a validation error

Coordinates carrying more than `coordinate_precision` decimal places (default 7) now fail validation instead of producing a warning. Each offending feature is reported individually with its filename and feature index, and validation stops before schema checks when any are found.

```
Feature 12 in 'opensidewalks.edges.geojson' contains coordinates with more than 7 decimal places. Reduce them to at most 7 decimal places; you can use the OSW data wizard tool to clean this up.
```

Precision is judged on **written decimal places**, never on how close a value is to a shorter one — `48.9999999999`, `49.0000000001`, and `49.00000000` are all rejected even though each is 49.0 or a hair from it. Anything with 7 or fewer decimals (`49`, `49.0`, `49.0000000`) passes. Exponent notation is measured after normalization.

### Errors always name a real file

`issues[].filename` was inconsistent: some errors reported the internal dataset key (`edges`, `zones`), some reported an `All` placeholder, and only some reported the actual filename. Every error now carries the GeoJSON file the problem was found in, and error text that named a dataset key names the file too:

| Check | Before | After |
|---|---|---|
| `_u_id`/`_v_id`/`_w_id` coordinate mismatch | `edges` / `zones` | `opensidewalks.edges.geojson` |
| Duplicate `_id`s | `nodes` | `nodes.geojson` |
| Vertex limit, collapsed geometry, missing column | dataset key | filename |
| Invalid geometries, unmatched references | `All` | file holding the bad values |

### Remediation guidance

Both precision errors and `ext:*` null/NaN errors now tell users how to fix the data and point at the OSW data wizard. The two message templates live in one place (`COORDINATE_PRECISION_REMEDY`, `NULLISH_VALUE_REMEDY`) so the wording can change in a single edit.

### `allow_zero_length_lines` now defaults to `True`

Collapsed `LineString` geometries in edges, lines, and external line data are accepted by default. Set `allow_zero_length_lines=False` to reject them. Zero-area polygons and zones remain invalid regardless.

## Verified, no code change needed

Two behaviors were confirmed already correct and are now pinned by tests:

- **Node references have no tolerance.** `_u_id`, `_v_id`, and `_w_id` must match their node coordinates exactly. Deviations of `1e-7`, `5e-8`, `1e-9`, and `1e-12` are all rejected, on either axis.
- **`_u_id` may equal `_v_id`.** No such constraint exists in the validator or the schemas. A self-loop with real length always passes; one collapsed to a point is governed solely by `allow_zero_length_lines` and is reported as a geometry problem, never as a reference problem.

## Breaking changes

- `ValidationResult.warnings` and the `COORDINATE_PRECISION_WARNING` constant are removed. The precision message is now part of `errors`/`issues`.
- Datasets with more than 7 decimal places of coordinate precision that previously passed will now fail. This is common in real-world data.
- Consumers matching on `filename == 'edges'` or `filename == 'All'` must be updated to match filenames.
- `allow_zero_length_lines` default flipped from `False` to `True`.

## Testing
```
 > coverage run --source=src/python_osw_validation -m unittest discover -v tests/unit_tests

----------------------------------------------------------------------
Ran 231 tests in 39.524s

OK
```

```
> coverage report
Name                                                      Stmts   Miss  Cover
-----------------------------------------------------------------------------
src/python_osw_validation/__init__.py                       570     43    92%
src/python_osw_validation/config.py                          20      0   100%
src/python_osw_validation/extracted_data_validator.py        99      1    99%
src/python_osw_validation/geometry_mapping_validator.py     160     26    84%
src/python_osw_validation/helpers.py                        202     22    89%
src/python_osw_validation/zipfile_handler.py                 48      1    98%
-----------------------------------------------------------------------------
TOTAL                                                      1099     93    92%

```
@sujata-m
sujata-m requested review from MashB and susrisha August 5, 2026 11:28
@sujata-m
sujata-m merged commit 167fcab into main Aug 5, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants