Reader testing¶

Reader tests check whether someone can find useful Bob documentation without knowing the site structure in advance. Test the live site and llms.txt as separate entry points, and record the route each reader tries before they reach a page.

Six-task protocol¶

Use five independent readers for each entry point. Give each reader only the live site URL or only the llms.txt URL. Do not give them the expected door, page name, or the other entry point. Present tasks one at a time, in the table's order, and do not explain Bob or point to a page. Allow up to three minutes per task; do not offer hints. Stop a task when the reader says they have the answer or asks for help.

Record the first door or llms.txt section the reader chooses, whether they reach the intended page, elapsed time, and any short observation. The site doors map to the published tabs as follows: Learn → Get started, Do → Guides, Understand → Concepts, Look up → Reference. For llms.txt, use the matching section title. A task succeeds when the reader reaches the intended page and can explain the requested goal or fact.

The pass bar is at least four of five readers choosing the expected door first for each task. A miss is evidence to improve labels or homepage copy. Keep page locations and the URL contract stable; reader tests do not authorize moving pages.

Results table¶

Keep one row per reader and task. Use a stable anonymous reader ID, and record separate rounds for site-only and llms.txt-only readers.