fix(sqlite): correct sub-second decoding of pre-epoch REAL datetimes#4340
Open
zelinewang wants to merge 1 commit into
Open
fix(sqlite): correct sub-second decoding of pre-epoch REAL datetimes#4340zelinewang wants to merge 1 commit into
zelinewang wants to merge 1 commit into
Conversation
Datetimes stored as a REAL Julian day number (e.g. via SQLite's julianday()) were decoded by splitting the UNIX timestamp with trunc()/fract().abs(). timestamp_opt() always adds the nanoseconds forward in time, so for values before 1970 this pushed the result up to ~2 seconds off and could move it onto the wrong side of the epoch (e.g. 1969-12-31 23:59:59.5 decoded as 1970-01-01 00:00:00.5). Round seconds toward negative infinity with floor() and take the sub-second remainder relative to that, so nanos is always a valid forward offset. Post-epoch values are unaffected. Adds unit tests for the pre- and post-epoch paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SQLite can store a datetime as a REAL holding a Julian day number, for example
the result of
julianday(...). When such a value is read back into achronotype,
decode_datetime_from_floatconverts the Julian day to a UNIX timestampand then splits it into whole seconds and nanoseconds:
timestamp_optalways addsnanosforward in time. For timestamps beforethe epoch the fractional part is negative, so
trunc()(which rounds towardzero) together with
fract().abs()moves the result in the wrong direction.The decoded instant ends up as much as ~2 seconds off, and for values in the
last second before the epoch it lands on the wrong side of it entirely. For
instance
1969-12-31 23:59:59.5decodes as1970-01-01 00:00:00.5.The fix rounds the seconds toward negative infinity with
floor()and takesthe sub-second remainder relative to that floor, so
nanosis always a validnon-negative forward offset:
Values at or after the epoch are unchanged, since there
floor()equalstrunc()and the fraction is already non-negative. The integer decode path(
decode_datetime_from_int) was already exact for whole seconds; this bringsthe REAL path in line with it.
Added unit tests in
sqlx-sqlite/src/types/chrono.rscovering a pre-epochvalue with a sub-second component (which fails before this change and passes
after) and a post-epoch value (guarding against a regression). I also
confirmed the end-to-end path with an in-memory
SELECT julianday('1969-12-31 23:59:59.500')read asDateTime<Utc>: beforethe change it returns
1970-01-01 00:00:00.500, after it returns1969-12-31 23:59:59.500(to within f64 round-off).Does your PR solve an issue?
No linked issue. I came across this while looking at how the REAL/Julian-day
storage class is decoded; the existing datetime tests only cover the TEXT
formats, so this path was untested.
Is this a breaking change?
No. This is a correctness fix. It does change the decoded value for pre-1970
datetimes stored as REAL Julian day numbers, but only from an incorrect value
to the correct one; the public API is unchanged and post-epoch values are
unaffected.