Skip to main content

2.3 Structured Output

AI-Generated Content

AI-generated content may contain errors. Always verify against official sources.

2.3 Structured Output

Key Concepts: JSON mode · XML tags · Schema enforcement · Pydantic validation

Official Docs: OpenAI Structured Outputs · Anthropic Tool Use · Pydantic Docs

Why Structured Output?​

When your code needs to parse the model's response — extract a score, populate a database, call a downstream API — free-form text is fragile. Structured output guarantees a machine-readable format.

Method 1 — OpenAI Structured Outputs​

OpenAI's response_format with a Pydantic model guarantees the output matches your schema via constrained decoding.

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()

class SentimentResult(BaseModel):
    sentiment: str          # "positive" | "negative" | "neutral"
    confidence: float       # 0.0 – 1.0
    key_phrases: list[str]

response = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Analyse the sentiment of the review."},
        {"role": "user",   "content": "Absolutely loved it — fast shipping, great quality!"}
    ],
    response_format=SentimentResult,
)

result: SentimentResult = response.choices[0].message.parsed
print(result.sentiment)     # positive
print(result.confidence)    # 0.97

Method 2 — JSON Mode (Wider Compatibility)​

import json
from pydantic import BaseModel, ValidationError

class Review(BaseModel):
    sentiment: str
    score: int

raw = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": 'Respond ONLY with JSON: {"sentiment": str, "score": int 1-5}'},
        {"role": "user",   "content": "The product broke on day one."}
    ],
    response_format={"type": "json_object"},
).choices[0].message.content

try:
    review = Review.model_validate_json(raw)
    print(review)
except ValidationError as e:
    print("Validation error:", e)

Method 3 — Anthropic XML Tags​

import anthropic, re

client = anthropic.Anthropic()

prompt = """
Analyse the sentiment and respond using EXACTLY this format:
<sentiment>positive|negative|neutral</sentiment>
<confidence>0.0 to 1.0</confidence>

Review: 'Fantastic product, exceeded expectations!'
"""

text = client.messages.create(
    model="claude-3-5-haiku-20241022",
    max_tokens=128,
    messages=[{"role": "user", "content": prompt}],
).content[0].text

sentiment  = re.search(r"<sentiment>(.*?)</sentiment>",   text).group(1)
confidence = re.search(r"<confidence>(.*?)</confidence>", text).group(1)
print(sentiment, confidence)

Common Mistakes​

Common Mistakes
  1. Trusting JSON mode without validation — JSON mode guarantees valid JSON, not correct data. Always validate schema with Pydantic.
  2. Using Structured Outputs with a model that doesn’t support it — client.beta.chat.completions.parse requires gpt-4o-2024-08-06 or newer. Older models will error.
  3. Overly complex schemas — deeply nested optional fields confuse the model. Flatten your schema where possible.
  4. Not handling ValidationError — even JSON mode can produce structurally valid but semantically wrong values (e.g., confidence: 1.5). Always wrap in try/except.

Quick Quiz​

Test Your Understanding

Q1. What guarantee does OpenAI’s Structured Outputs (response_format=MyModel) provide?
A1. The output is guaranteed to match the Pydantic schema via constrained decoding — it will not produce structurally invalid JSON.

Q2. What is the minimum model version required for OpenAI Structured Outputs?
A2. gpt-4o-2024-08-06 or newer.

Q3. Anthropic Claude doesn’t have a native JSON mode. What is the recommended alternative?
A3. XML tags — ask the model to wrap values in named XML tags and extract them with regex or an XML parser.

Q4. Why should you always validate with Pydantic even when using JSON mode?
A4. JSON mode guarantees syntactically valid JSON, but not correct field values or types. Pydantic catches semantic errors (wrong type, out-of-range values, missing required fields).

Student Exercise​

Exercise 2.6 — Structured extraction pipeline
Build a pipeline that takes a raw job posting (plain text) and extracts: job_title (str), company (str), required_skills (list[str]), salary_range (str | None), remote (bool). Use OpenAI Structured Outputs with a Pydantic model. Test on 3 real job postings from a job board.

Further Reading​

Next → 2.4 Domain-Specific Prompting