[Lua] Add Fory Serialization Support for Lua

[chaokunyang]

Feature Request

Add Apache Fory xlang serialization support for Lua.

Is your feature request related to a problem? Please describe

No response

Describe the solution you'd like

Design Plan

Implement a Lua xlang runtime that is wire-compatible with the xlang specification and implementation guide.

Scope and Deliverables

• Xlang encoder/decoder with little-endian binary format.
• Type registry for numeric and named user types.
• Struct/enum/union serialization support.
• Schema-consistent mode and compatible mode (meta share / TypeDef).
• Cross-language compatibility tests against existing implementations.
• Metatable restoration during deserialization for Lua struct-like objects.

Protocol Compatibility Requirements

• Header bitmap flags: null, xlang, oob.
• Reference flags: NULL(-3), REF(-2), NOT_NULL(-1), REF_VALUE(0).
• Internal type IDs (0~56) and separate user_type_id varuint32 for registered user types.
• Type meta rules for NAMED_*, COMPATIBLE_STRUCT, TYPED_UNION, and shared TypeDef streaming.
• Meta string encoding + dedup.
• Value encoding for varints, strings, collections/maps, struct field order, and union payload.

Lua-Specific Design Points

• Baseline runtime: Lua 5.4 (Lua 5.3 compatible where possible).
• Optional fast path: LuaJIT/FFI acceleration, pure Lua remains the compatibility baseline.
• Numeric policy: explicit int64/uint64 handling via wrappers where Lua number precision is insufficient.
• Table disambiguation: declared type descriptors for list/map/struct resolution.
• Reference tracking: identity tracking for reference-like values (table).

Metatable Restoration Strategy

• Register struct-like descriptors with namespace/type_name (or numeric type ID), create_instance(ctx), and metatable metadata.
• Decode flow for struct-like values:
  1. Resolve descriptor from type meta.
  2. Allocate instance via create_instance.
  3. Set metatable immediately.
  4. Reserve/fill reference slot around field decode for cycle-safe graphs.
  5. Populate fields using schema-consistent or compatible-mode mapping.
• For unknown named types in compatible mode, fallback to plain table while preserving decoded fields.

Implementation Phases

• Phase 0: bootstrap API and error taxonomy.
• Phase 1: buffer, endian IO, varints, header.
• Phase 2: reference resolver + type ID/meta core.
• Phase 3: primitive/string/time/binary serializers.
• Phase 4: list/set/map/array serializers.
• Phase 5: meta string + TypeDef.
• Phase 6: enum/struct/union.
• Phase 7: skip logic + malformed-input hardening.
• Phase 8: performance optimization and benchmarks.

Testing Plan

• Lua unit tests for buffer/varint/string/ref/type-meta.
• Comprehensive serialization tests for lua.
• Cross-language tests:
  • Java <-> Lua.
  • See java/fory-core/src/test/java/org/apache/fory/xlang/XlangTestBase.java.

Milestone Exit Criteria

• M1: buffers/primitives/strings/header/varints.
• M2: collections/maps/reference tracking complete.
• M3: struct/enum/union complete (schema-consistent mode).
• M4: compatible mode + TypeDef meta-share complete.
• M5: cross-language CI green with baseline perf report.

Describe alternatives you've considered

No response

Additional context

Reference implementation plan document in repository:

• https://fory.apache.org/docs/next/specification/xlang_serialization_spec
• https://fory.apache.org/docs/next/specification/xlang_implementation_guide

[chaokunyang]

This issue is reserved for gsoc, do not take over it currently

[ulekarsiddhant0-boop]

May, I start working on it @chaokunyang . I decided to contribute in apache fory project for GSoC 2026. Hope you assign this issue to me.
Thank you!

[chaokunyang]

Please follow https://community.apache.org/gsoc/ for this issue