[SPARK-58300][PROTOBUF] Add from_protobuf option to keep Timestamp/Duration as struct

[bhollis-dbx]

What changes were proposed in this pull request?

from_protobuf always maps google.protobuf.Timestamp -> TimestampType and google.protobuf.Duration -> DayTimeIntervalType. This is Spark's well-known-type conversion and there is currently no way to turn it off.

This adds a from_protobuf reader option convert.timestamp.duration.to.native (default true, preserving today's behavior). When set to false, both well-known types fall through to the generic message path and deserialize as their raw proto shape struct<seconds: bigint, nanos: int>.

The two well-known-type special-cases in SchemaConverters.structFieldFor are the only places these types are selected. Guarding both with the flag makes Timestamp/Duration fall through to the existing generic case MESSAGE => -> StructType path. The deserializer needs no change: it dispatches on the catalyst type, so the existing (MESSAGE, StructType) case already produces the struct. The option mirrors the existing unwrap.primitive.wrapper.types precedent.

Scope note: the to_protobuf serializer is intentionally not touched -- this is a read-path option only.

Why are the changes needed?

A consumer whose downstream schema was defined against the {seconds, nanos} struct representation (e.g. a table populated by a different decode path) and must stay stable, or that needs the exact seconds/nanos rather than a microsecond-truncated interval/timestamp, currently has no way to keep these well-known types in their raw proto shape.

Does this PR introduce any user-facing change?

Yes, additively. A new from_protobuf reader option convert.timestamp.duration.to.native is introduced. Its default is true, which preserves the existing behavior (google.protobuf.Timestamp -> TimestampType, google.protobuf.Duration -> DayTimeIntervalType). Setting it to false makes those well-known types deserialize as struct<seconds: bigint, nanos: int> instead. Default behavior is unchanged, so this is purely opt-in.

How was this patch tested?

Added ProtobufFunctionsSuite test "convert.timestamp.duration.to.native=false keeps Timestamp/Duration as struct", which round-trips a Timestamp and a Duration field through to_protobuf/from_protobuf with the option set to false and asserts:

• the deserialized field type is struct<seconds: bigint, nanos: int> (not TimestampType/DayTimeIntervalType), and
• the values round-trip correctly (the Duration 4.5s -> seconds=4, nanos=500000000; the Timestamp seconds are derived from the input so the assertion is time-zone independent).

Default behavior (true) is unchanged, so existing well-known-type tests continue to exercise the native-type path. Ran the new test and the existing WKT round-trip tests locally via sbt; all pass.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)