Describe the enhancement requested
When serializing a DateTimeOffset into an Arrow type, the TimestampType feels like a natural choice. However there is no time unit which fully represents both the precision and range of a C# DateTimeOffset. A DateTimeOffset tick is 100 nanoseconds long and the full range of representable values is between years 0001 and 9999 according to the documentation.
There are currently 2 options for TimeUnit when converting a C# DateTimeOffset to TimestampType
- TimeUnit.Microseconds : Lose precision but be able to fully convert every valid DateTimeOffset to Timestamp
- TimeUnit.Nanoseconds : Keep precision, but the range of representable calendar years is reduced to 1677 - 2262
Of course, users could serialize DateTimeOffset into a long in the Arrow Table and remember the type when deserializing. However, it would be nice to preserve the type information. Having a new TimeUnit equal to a tick would solve this tension.
I appreciate that adding a new time unit would be a fundamental change to arrow across more than just the .NET ecosystem, please let me know if this has been considered before or if the discussion needs to be moved elsewhere.
Describe the enhancement requested
When serializing a DateTimeOffset into an Arrow type, the TimestampType feels like a natural choice. However there is no time unit which fully represents both the precision and range of a C# DateTimeOffset. A DateTimeOffset tick is 100 nanoseconds long and the full range of representable values is between years 0001 and 9999 according to the documentation.
There are currently 2 options for TimeUnit when converting a C# DateTimeOffset to TimestampType
Of course, users could serialize DateTimeOffset into a long in the Arrow Table and remember the type when deserializing. However, it would be nice to preserve the type information. Having a new TimeUnit equal to a tick would solve this tension.
I appreciate that adding a new time unit would be a fundamental change to arrow across more than just the .NET ecosystem, please let me know if this has been considered before or if the discussion needs to be moved elsewhere.