Skip to content

Metadata decoder keeps only the first symbol-mapping interval — silently drops continuous-contract roll history #34

Description

@tbeason

Line refs are against the registered v0.1.3 tarball.

Problem

The metadata decoder reads only the first interval of each symbol mapping and silently skips the rest (src/decode.jl:343, comment: "For now, just read the first interval (simplified)"):

# For now, just read the first interval (simplified)
if intervals_count > 0
    ...
    push!(mappings, (raw_symbol, mapped_symbol, Int64(start_date_raw), Int64(end_date_raw)))
    # Skip remaining intervals for now
    for _ in 2:intervals_count
        pos += 4 + 4 + symbol_cstr_len
    end
end

Impact

For continuous futures queries (stype_in=continuous, e.g. ES.n.0 over years), the metadata's symbol mapping is the roll history — one interval per underlying contract. Keeping only the first interval means metadata.mappings silently reports the continuous symbol as mapping to a single contract for the whole range. Anyone joining records back to underlying contracts via metadata gets wrong answers with no warning. Same applies to any equity that changed instrument IDs over a long window.

The mappings::Vector{Tuple{String,String,Int64,Int64}} type also flattens raw symbol -> intervals into one tuple per symbol, so fixing this is a (small) breaking change to the field shape — either Vector of per-symbol interval lists, or one tuple per (symbol, interval) pair.

Workaround until fixed: use the symbology.resolve endpoint for roll maps instead of DBN metadata.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions