Let's talk about Structured Prompting

JSON, XML, and a new primitive for AI agents you've been missing out on

The Recent Buzz on X

Over the past week, X has been ablaze with discussions on structured prompting techniques for large language models. It started with a surge in posts hyping "JSON prompting" as a revolutionary skill for 2025, promising to transform LLMs like ChatGPT, Claude, and Gemini into reliable, hallucination-free agents. Threads shared copy-paste templates for tasks like content generation and data extraction, emphasizing how JSON's key-value structure provides clarity and consistency. Influential voices like Theo predicted it would dominate conversations, and viral posts from creators like Sehaj Singh amplified the trend, garnering thousands of likes and reposts.

But the discourse quickly evolved into a debate, with critics pointing out JSON's limitations—like token inefficiency and noise from whitespace—and advocating for alternatives. XML prompting emerged as a strong contender, with users claiming it's superior for creating clear boundaries in prompts. Posts from Matt Shumer urged ditching JSON entirely, while Jo Kristian Bergum called XML the "cheat code" used by LLM companies themselves. This shift highlights a growing consensus that structured formats aren't just hype; they're essential for scaling AI reliability, though the community remains divided on the best approach.

JSON vs. XML Prompting: Why XML Wins

OpenAI’s Noah MacCallum set the record straight:

• I’ve actually done experiments on this and markdown or xml is better

• “Models are trained on json” → yes they’re also trained on a massive amount of plain text, markdown, etc

• JSON isn’t token efficient and creates tons of noise/attention load with whitespace, escaping, and keeping track of closing characters

• JSON puts the model in a “I’m reading/outputting code” part of the distribution, not always what you want

My theory is that XML prompting excels because it explicitly delimits the start and end of sections with semantic tags, creating what we might call "semantic attentional boundaries." These tags—<task> for objectives, <context> for background—act as clear fences in the model's attention mechanism, allowing it to compartmentalize information more effectively than JSON's nested braces. Anthropic's documentation on Claude even recommends XML for this reason, noting improved parsing and reduced errors.

Introducing LLML: The Compositional Primitive for AI Contexts

LLML is a tool I built to make writing XML easy, LLML transforms nested data structures into optimized markup. It's available in Python, TypeScript, Rust, and Go, under MIT license. You compose data structures and LLML handles converting it to XML (or JSON, if you really want to).

Here’s a quick TypeScript example:

import { llml } from "@zenbase/llml";

const researchPrompt = llml({
  role: "Expert Researcher",
  query: "Impact of quantum computing on cryptography",
  sources: [
    "Academic papers from arXiv",
    "Recent news from TechCrunch",
    "Expert opinions from IEEE"
  ],
  steps: [
    "Retrieve and summarize key documents",
    "Identify breakthroughs and risks",
    "Synthesize recommendations"
  ],
  output_format: "Structured report with sections: Summary, Analysis, Future Implications",
  guardrails: [
    "Cite sources accurately",
    "Avoid unsubstantiated claims"
  ]
});

// Outputs VibeXML:
// <role>Expert Researcher</role>
// <query>Impact of quantum computing on cryptography</query>
// <sources>
//   <sources-1>Academic papers from arXiv</sources-1>
//   <sources-2>Recent news from TechCrunch</sources-2>
//   <sources-3>Expert opinions from IEEE</sources-3>
// </sources>
// ... (and so on)

This setup makes the agent prom modular: swap sources or steps without reformatting everything.

And a Python one:

from zenbase_llml import llml

agent_prompt = llml({
    "role": "Agentic RAG Analyzer",
    "initial_query": "Optimize supply chain logistics with AI",
    "retrieval": {
        "database": "Vector store of industry reports",
        "top_k": 5
    },
    "reasoning_loop": [
        "Evaluate retrieved docs for relevance",
        "Refine query if needed",
        "Generate final synthesis"
    ],
    "tools": ["search_api", "summarizer"],
    "guardrails": [
        "Limit iterations to 3",
        "Ensure cost under $0.05"
    ]
})

# Outputs VibeXML:
# <role>Agentic RAG Analyzer</role>
# <initial_query>Optimize supply chain logistics with AI</initial_query>
# <retrieval>
#   <database>Vector store of industry reports</database>
#   <top_k>5</top_k>
# </retrieval>
# ... (etc.)

LLML isn't just formatting—it's a declarative approach that makes prompt composition a breeze. Check the GitHub repo for more: github.com/zenbase-ai/llml. If you're building AI systems, this is a primitive you've been missing.

Pro tip: Format your complex tool call results as XML dynamically with LLML 😉