-
Notifications
You must be signed in to change notification settings - Fork 17
Better TOML libs -> tomlkit for style, tomli(-w) for speed #496
Copy link
Copy link
Open
Labels
accountingprolly positioning: the accounting of "what/when (is) owned"prolly positioning: the accounting of "what/when (is) owned"clearingauction and mm tech: EMS, OMS, algo-tradingauction and mm tech: EMS, OMS, algo-tradingconfigledgertrade, accounts and other user focal event history tracking, management and storagetrade, accounts and other user focal event history tracking, management and storageperfefficiency and latency optimizationefficiency and latency optimization
Metadata
Metadata
Assignees
Labels
accountingprolly positioning: the accounting of "what/when (is) owned"prolly positioning: the accounting of "what/when (is) owned"clearingauction and mm tech: EMS, OMS, algo-tradingauction and mm tech: EMS, OMS, algo-tradingconfigledgertrade, accounts and other user focal event history tracking, management and storagetrade, accounts and other user focal event history tracking, management and storageperfefficiency and latency optimizationefficiency and latency optimization
Since we already had to roll our own based on writing a particular
styled
.clearstable to thepps.tomlfile, this is a task tochoose the best (combo of) lib(s).
Here's the verbatim bullet-task breakout from #345:
after much pain and anguish getting
the
tomlencoder to output things sanely, i think it's just worthwriting or switching to a super fast custom encoder that writes
adhoc-ly in the format we want for positions entries in
pps.toml,the reader-writer separation is already what
tomlidoes (the fastest pure py reader soon to land in the stdlib) andthen we can just drop
tomloutright and worry about getting a general encoder later.support better indentation
per broker-account for better readability,
indented tables per broker-account would
be super nice if possible
tomlkitbelow which supports this -> https://github.com/pikers/tomlkit/blob/writing_docs_tweaks/tomlkit/items.py#L1644Array.multiline()Support
DecimalandMutableMappingcodecs, add a multiline-Arrayindent control python-poetry/tomlkit#294Turns out there's been a bunch of project updates in the space 🙏🏼,
tomlkit: from thepoetry/pendulumauth:https://github.com/sdispater/tomlkit and is based on a
rustlibDecimalinput support: SupportDecimalserialization? python-poetry/tomlkit#288bidictinput support: Supportbidictinputs? python-poetry/tomlkit#289msgspec.tomlnow exists :party: which means we can natively renderour msgs and structs easily!
tomli(-w)underneath so using that probably makes the mostsense for our speed solution.
a list of other alt libs that we prolly won't use but figured might as
well put them here just in case:
pytomlppc++ wrapped parser: https://bobfang1992.github.io/pytomlpp/pytomlpp.htmltoml-w, stdlib'stomlsibling writer lib: https://github.com/hukkin/tomli-w#does-tomli-w-support-writing-documents-with-comments-or-custom-whitespaceDecimalsupport which i don't think is intomlkit: https://github.com/hukkin/tomli#construct-decimaldecimals-from-toml-floats