Skip to main content

6.5 Claim Decomposition

AI-Generated Content

AI-generated content may contain errors. Always verify against official sources.

6.5 Claim Decomposition

Key Concepts: Breaking narratives into atomic claims · Typed claim taxonomy

Key Paper: FActScoring: Fine-Grained Atomic Evaluation (Min et al., 2023)

What is Claim Decomposition?​

Instead of validating an entire paragraph as a whole, break it into atomic claims — individual facts that can each be verified independently.

Original paragraph:
"The Transformer architecture was introduced in 2017 by Vaswani et al. at Google Brain.
It uses multi-head self-attention and achieved state-of-the-art results on translation."

Decomposed claims:
1. The Transformer architecture was introduced in 2017. [VERIFIABLE]
2. It was introduced by Vaswani et al. [VERIFIABLE]
3. The authors worked at Google Brain. [VERIFIABLE]
4. It uses multi-head self-attention. [VERIFIABLE]
5. It achieved state-of-the-art results on translation. [VERIFIABLE]

Step 1 — Decompose into Atomic Claims​

from openai import OpenAI
import json

client = OpenAI()

def decompose_into_claims(text: str) -> list[str]:
    """Break text into a list of atomic, verifiable claims."""
    resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{
            "role": "user",
            "content": f"""Break the following text into atomic factual claims.
Each claim should be a single, self-contained, verifiable statement.
Do NOT include opinions or subjective statements.
Return as JSON: {{"claims": [list of claim strings]}}

Text: {text}"""
        }],
        temperature=0,
        response_format={"type": "json_object"},
    )
    result = json.loads(resp.choices[0].message.content)
    return result.get("claims", [])

text = """GPT-3 was released by OpenAI in 2020. It has 175 billion parameters 
and was trained on 570GB of text data. It demonstrated few-shot learning."""

claims = decompose_into_claims(text)
for i, claim in enumerate(claims, 1):
    print(f"{i}. {claim}")

Step 2 — Verify Each Claim​

def verify_claim(claim: str, context: str = None) -> dict:
    """Verify a single atomic claim."""
    if context:
        prompt = f"""Given this context, is the claim supported, contradicted, or unverifiable?
Context: {context}
Claim: {claim}
JSON: {{"verdict": "supported|contradicted|unverifiable", "confidence": 0.0-1.0}}"""
    else:
        prompt = f"""Is this factual claim accurate based on your training knowledge?
Claim: {claim}
JSON: {{"verdict": "true|false|uncertain", "confidence": 0.0-1.0, "note": "<brief reason>"}}"""
    
    resp = client.chat.completions.create(
        model="gpt-4o",  # Use stronger model for verification
        messages=[{"role": "user", "content": prompt}],
        temperature=0,
        response_format={"type": "json_object"},
    )
    return json.loads(resp.choices[0].message.content)

def claim_level_validation(text: str, context: str = None) -> dict:
    """Full claim decomposition + verification pipeline."""
    claims = decompose_into_claims(text)
    results = []
    
    for claim in claims:
        verification = verify_claim(claim, context)
        verification["claim"] = claim
        results.append(verification)
    
    # Summary
    failed = [r for r in results if r["verdict"] in ("false", "contradicted")]
    uncertain = [r for r in results if r["verdict"] in ("uncertain", "unverifiable")]
    
    return {
        "total_claims": len(claims),
        "passed": len(claims) - len(failed) - len(uncertain),
        "failed": len(failed),
        "uncertain": len(uncertain),
        "overall_valid": len(failed) == 0,
        "details": results,
    }

result = claim_level_validation(
    "GPT-3 was released by OpenAI in 2020 with 175 billion parameters."
)
print(f"Claims: {result['total_claims']} | Failed: {result['failed']} | Valid: {result['overall_valid']}")

Common Mistakes​

Common Mistakes
  1. Verifying compound claims — "GPT-3 was released in 2020 with 175B parameters" is two claims. Decompose first, then verify each separately.
  2. Using weaker model to verify — the verifier must be as capable as (or more capable than) the generator. Use GPT-4o for verification.
  3. Not providing context — for RAG outputs, always pass the retrieved context to the verifier. This enables faithfulness checking rather than pure fact-checking.

Quick Quiz​

Test Your Understanding

Q1. What is an atomic claim?
A1. A single, self-contained, verifiable factual statement that cannot be further decomposed without losing meaning.

Q2. What is the key paper associated with atomic claim-level evaluation?
A2. FActScoring (Min et al., 2023) — which introduced fine-grained atomic fact verification for long-form text generation.

Q3. Why is claim decomposition more useful than whole-paragraph validation?
A3. It identifies exactly which specific facts are wrong, providing actionable feedback. Whole-paragraph validation only gives a pass/fail verdict.

Student Exercise​

Exercise 12.5 — Fact check a Wikipedia paragraph
Take a Wikipedia paragraph on a technical topic. Decompose it into atomic claims using the pipeline. Verify each claim against GPT-4o. Report the accuracy percentage and list any claims GPT-4o disputes.

Further Reading​

Next → 12.6 Building a Validation Pipeline