Markdown vs JSON for LLM Context: Which Format Saves Tokens

You have the same information in front of you and two ways to hand it to a model: as JSON, because that is how it lives in your database, or as Markdown, because that is what your document converter spits out. The choice feels cosmetic. It is not. The format you pick changes how many tokens you pay for and how reliably the model can reason over the content. Here is what actually differs, backed by what the benchmarks show, and a rule that tells you which to reach for.

Why the format changes your token bill

Tokenizers do not care that a bracket is 'just punctuation'. Every character you send is counted and billed, and JSON is a syntax built for machines to parse unambiguously, not for compactness. Each field carries a quoted key, a colon, quoted or bracketed values, commas between entries, and braces around every object. On nested data that scaffolding repeats on every record.

Markdown carries structure too, but with far lighter markers. A heading is one to three hash characters. A list item is a dash. A table row is a handful of pipes. Those markers map directly to document structure while adding almost no character overhead, so more of your token budget goes to actual content instead of syntax.

Markdown vs JSON: the token difference

Published comparisons consistently put Markdown ahead of JSON on token count for the same information. The savings depend heavily on the shape of the data: for flat prose the gap is modest, but for deeply nested records with many repeated keys it widens. Reported figures commonly land in the 15 to 30 percent range, and some benchmarks on nested data measured Markdown using roughly a third fewer tokens than the equivalent JSON, with formats like XML costing far more still.

There is a second, less obvious reason Markdown tends to win: it is the format models saw most during training. GitHub READMEs, documentation sites, and forum answers are overwhelmingly Markdown, so a model reading a hash for a heading or a pipe table is working in its native register. Several reasoning benchmarks report higher accuracy when the same content is presented as Markdown rather than wrapped in JSON.

So the win is double: fewer tokens for the same meaning, and content in the shape the model handles most fluently. On a document you send hundreds of times a day, that difference compounds into real cost.

When JSON is still the right call

None of this means Markdown is always better. JSON exists because ambiguity is expensive, and there are jobs where its explicitness is exactly what you want. If you are extracting structured output, calling tools, or exchanging data with another system, JSON with a schema is the correct choice, not a wasteful one.

Reach for JSON when field boundaries must be unambiguous: records with required fields, enums, or types that a downstream program will validate. A user profile, an API response, a config object, a row of transactions with strict keys. Here the braces and quotes are not noise, they are the contract, and flattening them into prose would lose the very structure you need.

The failure mode is using JSON for the wrong content type. Wrapping an article, a report, or a policy document in a JSON string does not make it more structured. It just adds escaped quotes and newlines, inflates the token count, and hands the model prose dressed up as data.

A simple rule: content versus data

The clean way to decide is to ask whether you are sending content or data. Content is written to be read: articles, manuals, contracts, meeting notes, anything with paragraphs, headings, and tables. Send that as Markdown. Data is written to be parsed: records, keys, values, things a program consumes. Send that as JSON.

• Prose, documents, reports, transcripts, anything with a reading order: Markdown.
• Records, API responses, configs, tool-call arguments and results: JSON.
• A document that also has tables: Markdown, with the tables as Markdown tables so rows and columns survive.
• A structured answer you want the model to return: ask for JSON, even if the input was Markdown.

Most real pipelines use both, and that is correct. A common pattern is Markdown for the grounding context the model reasons over, and JSON for the structured result you ask it to return. Instructions and documents go in as Markdown; the extraction comes back as validated JSON. You are not picking a side, you are matching each format to the job it is good at.

For RAG specifically, the content side almost always wants Markdown. Headings give you natural chunk boundaries, tables keep their rows and columns, and each chunk carries more meaning per token, which sharpens both retrieval and the final answer.

Getting clean Markdown out of real documents

The catch is that most of your content does not start as clean Markdown. It starts as a PDF, a Word file, a slide deck, or a spreadsheet, and naive extraction turns those into exactly the kind of noisy text that erases Markdown's advantage: broken words, flattened tables, repeated page headers, and dates that arrive as serial numbers. Convert badly and you get the token cost of prose with none of the structure.

Good conversion is what makes the content-as-Markdown rule pay off. You want real headings, tables that stay tables, reading order preserved, and page furniture stripped, so the output is compact and the structure survives into the model's context.

That is the job PackForAI does: it turns PDF, DOCX, PPTX, XLSX, CSV, and JSON into clean, compact Markdown built for LLM context, and shows the token count so you can see the difference on your own file. If you have decided your documents belong in Markdown, it handles the part between the raw file and clean input. You can try it on one document at packforai.com.

Pick by content type, not habit: Markdown for what the model reads, JSON for what a program parses. The wrong format quietly costs you tokens and accuracy.