Experiment 06

Vision Reading Protocol

A protocol for testing Claude image and document reading with controlled fixtures, known answers, and coordinate-aware scoring.

"send images to Claude"
Primary source: Vision

Task Types

Vision reading is not one task. Separate OCR, layout understanding, chart reading, visual comparison, coordinate localization, and document reasoning. A model may read printed text well and still struggle with tiny axis labels or ambiguous spatial instructions.

Start with synthetic fixtures whose answers you control. Then add real-world scans, screenshots, and photos. Keep the synthetic set because it tells you whether a failure is visual perception, image quality, or domain ambiguity.

Fixture Controls

Record file name, hash, dimensions, compression, crop, rotation, and whether the image was embedded directly or referenced through another surface. If you downsample or crop, store the transformed file as a separate fixture. Do not treat "same screenshot" as same input after editing.

For document pages, create known-answer keys for visible text, table cells, footnotes, and layout relationships. Add one trick case where a visually prominent label is not the requested answer.

  • OCR: exact text, including punctuation and line breaks if they matter.
  • Layout: relationship such as "caption below the left chart."
  • Chart: value read from axis and series.
  • Coordinate: bounding region or clicked target, scored with tolerance.
  • Refusal-to-infer: answer says the image does not show enough evidence.

Prompting

Ask for structured output when possible: fields for observed text, inferred answer, confidence, and evidence location. Separate observation from inference. For example, "Observed: the label appears to say Q3. Inference: Q3 revenue is the requested value." This separation makes hallucinated visual details easier to catch.

When testing coordinate workflows, cite Anthropic coordinate guidance from the vision docs and record the coordinate system. A top-left origin, normalized coordinates, and pixel coordinates are not interchangeable.

Scoring

Score OCR exactness, semantic answer correctness, evidence location, and unsupported visual details separately. If the answer is right but the evidence location is wrong, the result should not pass a workflow that requires visual traceability.

For public examples, use fixtures you are allowed to redistribute or publish only metadata plus a replacement fixture. Do not upload private documents simply to create a compelling demo.

Routing

If the page becomes a gallery of impressive visual demos, route it to Magic Claude. If it becomes a production document-workflow tutorial, route to Claude Helps. Curious Claude owns the reproducible test harness.

Experiment FAQ

Should I use screenshots or original PDFs?

Test the format your product will actually send. If both are plausible, keep them as separate variables and record dimensions and transformations.

What should I do with private documents?

Do not publish them. Use synthetic or redistributable fixtures publicly, and keep private fixture metadata internal.

Primary Sources