Key Takeaways

  • JSON is great for web apps but wasteful for large language models.
  • TOON (Token-Oriented Object Notation) is a compact JSON alternative.
  • TOON data format can cut LLM token use by 30–60% in many cases.
  • It stays human-readable while giving models cleaner, AI-ready data.
  • You can keep JSON in your APIs and use TOON only for LLM calls.

JSON has been our friend for a long time. It runs inside APIs, browsers, and many apps. However, in the LLM age, JSON is starting to feel heavy.

Every brace, quote, and key name becomes a token. Tokens cost money. They also make prompts longer and slower. A new idea, called TOON, tries to fix this by giving us a smarter, AI-ready data format.

What TOON Is and Why It Exists

TOON stands for Token-Oriented Object Notation. It is a compact way to write the same data you put in JSON, but with fewer extra symbols and less noise.

Think of TOON as:

  • The same data model as JSON.
  • A cleaner skin that is easier for LLMs to read.
  • A token efficient encoding that saves money and time.

You still work with normal objects, arrays, and nested fields. However, you type less punctuation. The model sees less clutter. This makes TOON a strong JSON alternative for AI work.

TOON is designed as an AI data format for large language models. It tries to answer a simple question:

“How can we send structured data for LLMs without wasting tokens on syntax?”

Why JSON Struggles in the AI Era

JSON is still perfect for many things. Browsers love it. Backends love it. However, large language models see JSON in a different way.

Here is what hurts JSON inside prompts:

  • Verbose syntax
    Curly braces, quotes, commas, and colons appear again and again.

  • Repetition of keys
    In big arrays, every row repeats the same field names. This is fine for machines, but wasteful for LLM token usage.

  • Token cost
    LLMs pay for every symbol. Extra punctuation and repeated keys turn into real money.

  • Parsing overhead for models
    JSON looks neat to us. However, to a model, it can be a busy forest of symbols. This can make reasoning harder.

This is why people now look for a more compact AI-ready data format. TOON steps into that space.

How TOON Looks: Simple Examples

Let’s compare JSON vs TOON with a few tiny samples. The shapes here match public TOON data format examples and docs.

1. Simple object

JSON:

{ "name": "Alice", "age": 30, "city": "Bengaluru" }

TOON:

name: Alice
age: 30
city: Bengaluru

Same data. Fewer tokens. Easier to scan.

2. Array of values

JSON:

{ "colors": ["red", "green", "blue"] }

TOON:

colors[3]: red,green,blue

The [3] tells us there are three items. The list holds the values.

3. Array of objects

JSON:

{
  "users": [
    { "id": 1, "name": "Alice" },
    { "id": 2, "name": "Bob" }
  ]
}

TOON:

users[2]{id,name}:
  1,Alice
  2,Bob

Here, users[2]{id,name} defines the schema one time. The next lines hold the rows. This is where TOON really shines for LLM prompt optimization, because you do not repeat field names.

4. Nested objects

JSON:

{
  "user": {
    "id": 1,
    "name": "Alice",
    "profile": { "age": 30, "city": "Bengaluru" }
  }
}

TOON:

user:
  id: 1
  name: Alice
  profile:
    age: 30
    city: Bengaluru

Here, indentation shows nesting. It feels a bit like YAML, but it still maps to the JSON data model and supports compact data serialization.

Why TOON Matters for LLM Workloads

TOON is not just “shorter JSON.” It is shaped around how LLMs work.

Most modern models:

  • Read inputs as streams of tokens.
  • Do best when structure is clear and regular.
  • Struggle more when prompts are noisy or huge.

TOON helps in several ways:

  1. Fewer tokens
    For large, uniform arrays of objects, TOON often saves 30–60% tokens compared to pretty-printed JSON. This can cut your LLM bill in half for some jobs.

  2. Cleaner structure for the model
    Because you declare keys once and then stream rows, the pattern becomes obvious. Models see a nice table instead of a long forest of braces.

  3. Human-friendly layout
    TOON keeps things readable. You can open a TOON block and understand it at a glance, which helps with debugging AI API design and tests.

  4. Guardrails for structure
    Array sizes and field lists are explicit. This gives the model more hints and also makes it easier to validate responses.

In short, TOON gives you a purpose-built AI data format while staying close to JSON.

When TOON Beats JSON (and When It Does Not)

TOON is strong, but it is not magic. It works best in some clear cases.

Great use cases for TOON

  • Big tables of similar objects.
    Example: users, orders, products, logs, events.

  • Analytics-style inputs.
    Rows and columns that look like CSV, but where you still want field names.

  • Agent memory and state.
    Data that flows through many LLM calls, where every saved token matters.

  • Internal LLM pipes.
    Inside your system, between tools and agents, where you control both ends.

Cases where JSON may still be fine

  • Deeply nested, irregular data.
    When shapes vary a lot, JSON’s explicit structure can be easier to read.

  • External APIs and public contracts.
    JSON is still the main web standard, so you usually keep it at the edges.

  • Small payloads.
    For tiny objects, token savings may not justify a new format.

Many teams will use both: JSON for the outside world, TOON for the inside LLM calls. That balance shows up often in real examples.

How TOON Works Under the Hood

You do not need to know every rule, but a few ideas help:

  • TOON keeps the JSON data model. Anything you can encode as JSON, you can encode as TOON.
  • It mixes ideas from CSV (rows) and YAML (indentation) to stay compact.
  • Arrays of objects use a schema header like users[3]{id,name,email} and then list rows.
  • Nested objects use indentation instead of many braces.
  • Optional key folding lets you turn nested keys into dotted paths, like meta.items.total.

Libraries can convert JSON to TOON and back. You do not have to hand-write this format in most apps.

Getting Started with TOON in Your Stack

You can try TOON in small, safe steps. Here is a simple path.

1. Keep JSON as your main API format

Do not rip out JSON. Instead:

  • Use JSON for public REST and GraphQL APIs.
  • Use JSON for database documents and logs, at least at first.

This keeps your tools, SDKs, and partners happy.

2. Convert JSON to TOON just for LLM calls

When you send big structured data into a model:

  1. Build or fetch the data as a normal JSON object.
  2. Use a TOON library to encode it.
  3. Drop the TOON block into your prompt.

This lets you enjoy TOON’s gains without breaking contracts.

3. Use existing libraries and tools

There are already open-source SDKs and tools for this token-oriented object notation in languages like:

  • Python
  • TypeScript / JavaScript
  • Go
  • Rust
  • .NET (early work)

Most libraries expose simple encode() and decode() functions:

  • encode(json_value) -> toon_text
  • decode(toon_text) -> json_value

Some also give helper functions to estimate token savings.

4. Measure token savings for your real data

Do not guess. Instead:

  • Pick a few real payloads.
  • Encode them as JSON and TOON.
  • Run a token counter for your target model.

You may find that some data structures save little, while flat arrays save a lot. Use that to decide where TOON matters.

5. Add tests and validation

Because TOON is still new, add checks:

  • Unit tests that ensure decode(encode(x)) returns the same JSON.
  • Schema checks around your TOON blocks.
  • Simple linters for indentation and array lengths.

This keeps your AI API design safe as you add this new format.

Practical Tips and Best Practices

Here are some short tips to use this AI-ready data structure well.

  • Start with internal tools, not public APIs.
  • Use TOON for data-heavy prompts, not for every little object.
  • Keep comments and natural language outside the TOON block.
  • Label your TOON sections in prompts so future devs know what they see.
  • Document your standard layouts so other teammates can reuse them.

Also, remember that TOON is part of a bigger story. It is one piece in a modern LLM data format toolkit, along with RAG, vector stores, and smart prompt design.

Did You Know? Box

Did You Know?

  • Some public tests show TOON cutting prompt size by over 40% on large user tables.
  • TOON was built as “schema-aware JSON for LLM prompts,” not as a full web standard.
  • You can mix TOON with normal text in the same prompt, as long as you mark the boundaries.
  • Several guides now call TOON a “drop-in JSON alternative” for AI-heavy workloads.

Conclusion

JSON is not going away. It still powers the web. However, the AI era brings new needs. Big models care about tokens, structure, and clarity.

TOON steps in as a compact AI data format. It keeps the JSON data model but trims the extra punctuation. It makes arrays of objects feel like tables. It helps LLMs see your data with less noise.

You do not have to choose TOON vs JSON for everything. You can keep JSON on the outside and use TOON on the inside. Over time, you may find more spots where TOON data format saves money, improves answers, and keeps prompts simple.

If you work with LLM apps, it is worth testing TOON now. It might quietly become your favorite tool for compact data serialization in prompts.

FAQs

What is TOON in simple words?

TOON is a compact way to write the same data you put in JSON. It removes extra symbols and repeats fewer key names. This makes it cheaper and clearer for large language models.

Is TOON meant to replace JSON everywhere?

No. TOON is built mainly for AI and LLM workloads. JSON still fits best for most web APIs and many systems. In practice, teams often use JSON for APIs and TOON only when talking to models.

How much token saving can TOON give?

For large, uniform data sets, many tests show 30–60% fewer tokens compared to formatted JSON. Real savings depend on your exact data and model tokenizer.

Does TOON keep all the same data as JSON?

Yes. TOON is a lossless encoding of the JSON data model. Anything you can express in JSON can be expressed in TOON, then decoded back to the same structure.

How do I start using TOON in my LLM app?

Keep your data as JSON inside your code. Before calling the model, pass that JSON through a TOON library to encode it. Use the TOON text inside your prompt. When needed, you can decode TOON back into JSON on the way out.

References