Structured Output from LLMs: JSON, Tables, and Reliable Formatting

You asked the AI to "list the top 5 features and their status." It gave you a beautifully written paragraph. Not a list. Not a table. Not JSON. Just a wall of text that your code can't parse and your stakeholders won't read.

Getting reliable, structured output from LLMs — JSON, tables, YAML, XML, or any consistent format — is one of the most common challenges in AI development. The model can produce structured output, but only if you ask correctly.

This guide covers the techniques, API settings, and prompt patterns that make LLM output predictable and parseable.

Why Structure Matters

For humans reading AI output, structure is a convenience. For applications consuming AI output, structure is a requirement.

Unstructured output:

The analysis shows that revenue grew by 12% in Q4, driven primarily by
the enterprise segment which saw 23% growth. However, the SMB segment
declined by 4%, which is concerning given that it represents 40% of our
customer base...

A human can read this. But try extracting revenue_growth, enterprise_growth, and smb_growth programmatically — you'd need regex, NLP, or another LLM call just to parse it.

Structured output:

{
  "period": "Q4",
  "revenue_growth": 12,
  "segments": [
    {"name": "Enterprise", "growth": 23, "concern": false},
    {"name": "SMB", "growth": -4, "concern": true, "note": "40% of customer base"}
  ]
}

Same information. But now your application can parse it, your database can store it, and your dashboard can display it — no human in the loop.

Method 1: API-Level JSON Mode

Most providers now offer native JSON mode that guarantees valid JSON output.

OpenAI

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Extract product information as JSON."},
        {"role": "user", "content": "The iPhone 16 Pro costs $999 and has 256GB storage."}
    ],
    response_format={"type": "json_object"}
)
# Guaranteed valid JSON output
data = json.loads(response.choices[0].message.content)

Important: When using response_format: json_object, you MUST mention "JSON" in your prompt. Otherwise the API returns an error. For more code-level patterns, check out our prompt templates for developers.

OpenAI Structured Outputs (Strictest)

For guaranteed schema compliance:

from pydantic import BaseModel

class Product(BaseModel):
    name: str
    price: float
    storage_gb: int
    category: str

response = client.beta.chat.completions.parse(
    model="gpt-4o",
    messages=[...],
    response_format=Product
)
product = response.choices[0].message.parsed  # Typed Product object

This guarantees every field exists, has the correct type, and matches your schema exactly. No parsing errors possible.

Anthropic (Claude)

Claude doesn't have a dedicated JSON mode toggle, but reliably produces JSON when instructed. Use prefilling to force JSON output:

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    messages=[
        {"role": "user", "content": "Extract product info as JSON: iPhone 16 Pro, $999, 256GB"},
        {"role": "assistant", "content": "{"}  # Prefill forces JSON start
    ]
)
# Response continues from "{" — guaranteed to start as JSON

Google Gemini

import google.generativeai as genai

model = genai.GenerativeModel(
    'gemini-2.0-flash',
    generation_config={"response_mime_type": "application/json"}
)
response = model.generate_content("Extract product info: iPhone 16 Pro, $999, 256GB")

Method 2: Prompt-Level Structure

When you can't use API-level JSON mode (or need formats other than JSON), prompt design gets the job done.

The Schema-In-Prompt Pattern

Show the exact schema you want:

Extract the following information and return it as JSON matching
this exact schema:

{
  "company_name": "string",
  "revenue": "number (in millions USD)",
  "growth_rate": "number (percentage)",
  "employees": "integer",
  "headquarters": "string",
  "founded_year": "integer"
}

If any field is not mentioned in the text, use null.
Do NOT add fields not in the schema.

Text:
[paste text to extract from]

Why it works: The model treats the schema as a template. Showing the exact key names, types, and structure is more reliable than describing the format in words.

The Example Output Pattern

Show a complete example of the desired output:

Analyze the following customer feedback and categorize it.

Example input: "The app crashes every time I try to export a PDF.
I've been waiting 3 weeks for a fix."

Example output:
{
  "category": "bug_report",
  "severity": "high",
  "feature": "pdf_export",
  "sentiment": "frustrated",
  "action_needed": "engineering_escalation",
  "wait_time_mentioned": "3 weeks"
}

Now analyze this feedback:
[actual feedback to analyze]

Few-shot examples are the most reliable way to get consistent format across many inputs.

Delimiter Patterns

For non-JSON structured output, use clear delimiters:

Analyze this code and provide your review in the following format:

===SUMMARY===
[One paragraph overview]

===ISSUES===
- [CRITICAL] [description]
- [WARNING] [description]
- [INFO] [description]

===SUGGESTED_FIX===
```code
[fixed code]

===CONFIDENCE=== [HIGH/MEDIUM/LOW]

Delimiters like `===SECTION===` are easy to parse with string splitting and unambiguous to the model.

## Method 3: XML Tags (Especially for Claude)

Claude responds particularly well to XML-structured prompts and outputs:

Analyze this customer support ticket and respond using these XML tags:

[billing/technical/feature_request/complaint] [low/medium/high/critical] [positive/neutral/negative/angry]

[One sentence summary]

[Draft response to customer] [Notes for the support team]

Ticket: [paste ticket]

**Why XML works well:**
- Unambiguous opening and closing tags
- Nesting is natural (`<analysis><category>...</category></analysis>`)
- Easy to parse programmatically (`xml.etree` in Python, DOMParser in JS)
- Claude's training data includes extensive XML, so it follows the structure reliably — see our [ChatGPT vs Claude vs Gemini comparison](/blog/chatgpt-vs-claude-vs-gemini) for more on each model's strengths

## Method 4: Markdown Tables

For human-readable structured output, markdown tables are ideal:

Compare these three database options for our use case.

Present the comparison as a markdown table with these columns:

Include rows for:

• Data model
• Scaling approach
• Query flexibility
• Cost at our scale (~1M records, 10K queries/day)
• Operational complexity
• Best for our use case? (Yes/No with reasoning)

Our use case: [describe requirements]

Tables work well because:
- Models are trained on millions of markdown tables
- The structure is rigid enough to be consistent
- Output renders nicely in any markdown viewer
- Easy to convert to CSV or spreadsheet format

## Method 5: YAML for Configuration

YAML is more readable than JSON for configuration and specification files:

Generate a Docker Compose configuration for the following stack:

• Web app: Node.js 20, port 3000, needs DATABASE_URL env var
• Database: PostgreSQL 16, persistent volume, port 5432
• Cache: Redis 7, port 6379
• Reverse proxy: Nginx, ports 80 and 443

Output as valid YAML (docker-compose.yml format). Include health checks for all services. Use named volumes, not bind mounts. Add comments explaining non-obvious configuration choices.

YAML is best for:
- Config files (Docker, Kubernetes, CI/CD)
- Human-readable structured data
- Nested data with comments

## Error Handling: When the Model Breaks Format

Even with perfect prompts, models occasionally break format. Here's how to handle it:

### Validation with Retry

```python
import json
from pydantic import BaseModel, ValidationError

class ExpectedOutput(BaseModel):
    name: str
    score: float
    tags: list[str]

def get_structured_output(prompt: str, max_retries: int = 3) -> ExpectedOutput:
    for attempt in range(max_retries):
        response = call_llm(prompt)
        try:
            data = json.loads(response)
            return ExpectedOutput(**data)
        except (json.JSONDecodeError, ValidationError) as e:
            if attempt < max_retries - 1:
                prompt = f"""Your previous response was not valid JSON
                or didn't match the expected schema.

                Error: {str(e)}

                Please try again. Return ONLY valid JSON matching this schema:
                {ExpectedOutput.model_json_schema()}

                Original request: {prompt}"""
            else:
                raise

Extraction Fallback

If the model wraps JSON in explanatory text, extract it:

import re

def extract_json(text: str) -> dict:
    """Extract JSON from a response that might contain surrounding text."""
    # Try parsing the whole response
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        pass

    # Try extracting from code blocks
    json_match = re.search(r'```(?:json)?\s*\n?(.*?)\n?```', text, re.DOTALL)
    if json_match:
        return json.loads(json_match.group(1))

    # Try finding JSON object/array boundaries
    for start_char, end_char in [('{', '}'), ('[', ']')]:
        start = text.find(start_char)
        end = text.rfind(end_char)
        if start != -1 and end != -1:
            return json.loads(text[start:end + 1])

    raise ValueError("No valid JSON found in response")

Schema Validation with Zod (TypeScript)

import { z } from 'zod';

const ProductSchema = z.object({
  name: z.string(),
  price: z.number().positive(),
  category: z.enum(['electronics', 'clothing', 'food', 'other']),
  in_stock: z.boolean(),
  tags: z.array(z.string()).max(5),
});

type Product = z.infer<typeof ProductSchema>;

function parseResponse(raw: string): Product {
  const data = JSON.parse(raw);
  return ProductSchema.parse(data); // Throws ZodError if invalid
}

5 Ready-to-Use Templates

1. API Response Formatting

Process this request and return a JSON response matching this API schema:

{
  "status": "success" | "error",
  "data": {
    [your data schema here]
  },
  "metadata": {
    "processing_time": "string",
    "model_confidence": "number (0-1)",
    "warnings": ["string"]
  }
}

Return ONLY the JSON object. No explanatory text before or after.

Request: [user request]

2. Data Extraction from Unstructured Text

Extract structured data from the following text.

Return a JSON array where each item has:
{
  "entity": "string (name of person, company, or product)",
  "entity_type": "person" | "company" | "product",
  "attributes": {
    [relevant attributes as key-value pairs]
  },
  "mentioned_in": "string (quote the relevant sentence)"
}

If an attribute is mentioned but ambiguous, set its value to null and add
a "notes" field explaining the ambiguity.

Text:
[paste unstructured text]

3. Report Card / Scorecard

Evaluate [subject] and present the results as a scorecard.

Format as a markdown table:

| Category | Score (1-10) | Assessment | Evidence |
|----------|:---:|-----------|----------|

Categories to evaluate:
1. [Category 1]
2. [Category 2]
3. [Category 3]
4. [Category 4]
5. [Category 5]

After the table, include:
- **Overall Score**: [weighted average]
- **Top Strength**: [one sentence]
- **Top Weakness**: [one sentence]
- **Recommendation**: [one sentence]

Subject: [what you're evaluating]
Context: [relevant background]

4. Comparison Table

Compare [items] across [dimensions].

Output as a markdown table. Use these symbols for quick scanning:
- ✅ Strong / Yes
- ⚠️ Partial / With caveats
- ❌ Weak / No / Missing
- 🔷 Not applicable

| Dimension | [Item A] | [Item B] | [Item C] |
|-----------|---------|---------|---------|

Dimensions:
[list what to compare]

After the table, add a "Bottom Line" section: one sentence recommendation
for each common use case.

5. Configuration File Generation

Generate a [config file type] for the following requirements:

Requirements:
[list requirements]

Output constraints:
- Valid [YAML/JSON/TOML/INI] syntax
- Include comments explaining non-obvious settings
- Use sensible defaults for any settings not specified
- Group related settings together
- Mark security-sensitive values as [CHANGE_ME]

Output ONLY the configuration file. No explanatory text.

Best Practices Summary

Rules of Thumb

1. Use API-level JSON mode when available — it's more reliable than prompt-level instructions. Pair this with a well-crafted system prompt for consistent behavior across requests
2. Always show the exact schema — don't describe it in words, show the structure
3. Include an example — one good example beats 100 words of instruction
4. Say "return ONLY the JSON" — prevents the model from adding explanatory text
5. Handle errors in code — even perfect prompts occasionally produce invalid output
6. Validate with Pydantic/Zod — catch schema violations before they reach your application
7. Use the right format for the job — JSON for machines, markdown tables for humans, YAML for configs

How Promplify Helps

When you optimize a prompt with Promplify, the engine analyzes whether your task would benefit from structured output and:

• Adds format specifications if your prompt is missing them
• Includes schema examples that make the model's output consistent
• Structures instructions so the model knows exactly what shape to produce
• Selects appropriate delimiters for multi-section outputs

Paste a prompt like "compare these three tools" and the optimized version will include table format, comparison dimensions, and a clear output structure.

Key Takeaways

• Unstructured AI output breaks applications — always specify format for programmatic use
• API-level JSON mode is the most reliable method (use it when available)
• Show the exact schema you want — a template beats a description
• XML tags work especially well with Claude
• Always validate AI output in code — no format guarantee is 100%
• Use Pydantic (Python) or Zod (TypeScript) for runtime validation
• Choose the right format: JSON for APIs, tables for humans, YAML for configs

---

Getting inconsistent output from your AI prompts? Try Promplify free — the optimizer adds format specifications and schema examples that make your AI output structured, consistent, and parseable.