Citation Grounding Protocol

Build a Claim Map

The smallest unit in this experiment is a claim, not a paragraph. Before running Claude, mark which source passage supports each expected claim. Also mark claims that are tempting but absent. The answer key should include "not supported" outcomes because source-grounded systems must know when not to answer.

Use source documents that are public, stable, and short enough for manual review at first. The protocol can later scale to retrieval systems, but the first test should remove retrieval as a variable so citation behavior is visible.

Source Pack

Prepare three kinds of documents: the authoritative source, a near-duplicate with outdated or conflicting wording, and an irrelevant but topically similar distractor. Claude should cite the authoritative source for supported claims, handle conflict explicitly, and avoid citing irrelevant passages simply because they share keywords.

If the run uses Anthropic citations, record the returned cited passages. If it uses web search, record search settings and domains. Anthropic web search docs describe current web access with citations and domain controls; those settings are part of the experiment.

Rubric

Score each answer on four axes: claim correctness, passage relevance, citation completeness, and unsupported additions. A claim with no citation fails completeness. A correct claim with an unrelated citation fails relevance. A cited passage that merely mentions the same topic is not enough.

When a source says "unknown," "not available," or "not in this document," the answer should preserve that uncertainty. Do not reward fluent completion that fills gaps from prior knowledge.

• Pass: every material claim is supported by the cited passage.
• Partial: main answer is right, but one claim is uncited or weakly supported.
• Fail: cited passage does not support the claim, or unsupported facts are added.
• Abstain pass: answer says the source pack does not contain enough evidence.

Adversarial but Fair Cases

Use stale documentation, changelog edits, or policy updates as fair edge cases. Do not hide malicious instructions in the source pack unless the experiment is explicitly a prompt-injection test. Citation experiments should first isolate grounding behavior.

When a topic becomes model-release analysis or benchmark interpretation, link to Claude Reports. This page only defines whether a cited answer is grounded.

Publication Standard

A publishable result includes the source pack or hashes, prompt, model ID, date, raw answer, claim map, and scorer notes. If the source pack includes copyrighted or private material, publish only enough metadata to reproduce the structure with substitute documents.