Reduce LLM token costs 40-60% for structured data

(github.com)

2 points | by prashantdudami 6 hours ago ago

1 comments

  • prashantdudami 6 hours ago ago

    Author here. I've been building enterprise LLM systems and noticed we were burning tokens on repeated JSON attribute names.

    TOON separates schema from data—you declare field names once, then send pipe-delimited values. LLMs (GPT-4, Claude, Gemini) parse it correctly.

    Install: pip install toon-token-optimizer

    The format is simple: @schema:name,age,city John|30|NYC Jane|25|LA

    Handles nested objects (dot notation), arrays, escaping, etc.

    Full writeup on the methodology: https://www.linkedin.com/posts/prashantdudami_llmarchitectur...

    Happy to answer questions!