Highlights Reference Note Sync¶

bob ref creates Highlights-ready PDFs from Markdown and turns Highlights app PDF annotations into Obsidian reference notes in the Bob vault.

By default, create files a new PDF into lib/, writes its reference note and reading task, and commits and pushes those paths together. xlib/ is the fallback queue when intake is kept or cannot safely run; scan drains that queue and repairs notes or annotations. Marker write-back to a PDF is a separate opt-in (sync --write-pdf or scan --write-pdfs).

Code lives in this bob-cli repository. On the MacBook, use a checkout at ~/projects/bob-cli; do not install from ad hoc scripts outside that checkout.

Contents¶

• What it does
• MacBook validation notes
• Default paths
• Marker note grammar
• Required parent, type, and ref type
• Synced properties
• PDFs open in Highlights
• Scan, safety, and Git/ob behavior
• Generated body contract
• Image notes
• Continuation highlights (<)
• Underline emphasis
• Sections (# and @)
• Reading tasks in parent notes (v2)
• MacBook setup guide
• Scheduled scan
• Disable one PDF
• Backup and rollback
• Conflict resolution
• Expected failures

What it does¶

bob ref implements marker/frontmatter synchronization, Markdown/TextBundle sidecar parsing, generated note rendering, TextBundle image selection asset copying, image notes (a screenshot path pasted into a highlight comment), continuation highlights (a < note groups a highlight with the one before it), recursive library scan with xlib intake, prerequisite checks, output collision detection, dirty-target refusal, and atomic note writes.

sync <pdf> loads one PDF with native Rust code, treats the first standalone /Text annotation on page 1 as the marker note, parses the marker list, creates or updates ref/<ref_type>/<pdf-basename>.md for PDFs under lib/<ref_type>/, and rewrites only the managed Highlights body region from the sidecar when one is present. Top-level library PDFs and explicit out-of-library syncs keep the legacy ref/<pdf-basename>.md target. marker <pdf> inspects the same marker without writing.

scan --intake-only (-i) is the drain for the xlib/ queue: it files only the queued PDFs and late audio and leaves the annotation pass, and the pre-scan hook, to the full scan; see Intake-only scan.

scan runs the configured pre-scan hook on writing runs, then moves pending PDFs from the configured intake directory into the mirrored library path together with same-stem companion audio (.mp3, .m4a, .ogg, .opus), recursively finds PDFs under the configured library directory, and processes them in stable path order. New references file one ordinary reading task in their area, project, or inbox note while the reference note shows a managed embed; see Reading tasks in parent notes (v2). Notes that still carry an in-note ^ref tracker keep the legacy behavior described below. Pass -n, --no-hooks on scan, or before the subcommand as bob ref --no-hooks scan, to ignore the hook. Only scan and doctor honor that flag. A parent --no-hooks before create, marker, or sync is accepted and ignored. Per-PDF validation or write failures are reported without stopping unrelated PDFs; the final command status is still non-zero when any PDF fails. It refuses intake destinations that already exist and duplicate output paths such as two PDFs that would both write the same ref/<ref_type>/<basename>.md target.

doctor checks vault paths, library/ref/xlib directories, pending intake, the configured pre-scan hook, sidecar presence, marker readability, PDF write coordination, Git worktree status, the Markdown render tools (pandoc, xelatex, LaTeX packages), and optional ob availability. The pdf_write_coordination: row is on or off on macOS (a warning when off) and n/a (no OS file coordination) elsewhere. Companion audio sitting in xlib/ with no matching PDF in xlib/ or lib/ is a warning, not a failure. The warning-level xelatex and latex_packages rows report the Markdown render prerequisites: a missing xelatex warns that PDF creation is unavailable, and missing LaTeX packages warn with the tlmgr install line that provides them. It never writes files. Pass -n, --no-hooks on doctor, or before the subcommand as bob ref --no-hooks doctor, to skip the pre-scan hook check.

The xlib_pending: N row gains · oldest <age> (from file mtimes) while anything is queued, and doctor warns once the oldest queued PDF is over an hour old (xlib intake: N queued, oldest <age> — run bob ref scan -i …). xlib_pending: 0 is unchanged. The intake_last: row reads the per-host intake status record at bob_cli_state_dir()/ref/intake-status.json (schema_version, last_success {at, command, filed}, and last_failure {at, command, reason, message}) and prints ok <age> ago (<command>, <n> filed) and failure <age> ago (<reason>: <message>), newest first, or none recorded. bob ref create (and the producers that call it), bob ref scan -i, and every writing full bob ref scan update it; a queue reason counts as a failure, a dry run or an empty queue records nothing, and a failure to write the record only warns.

After the sync-prerequisite rows, doctor appends warning-level library health and coverage rows read through the read-only reference index. They never fail the command:

• library: ok (…) totals the indexed notes by reading state.
• library diagnostics rolls up row diagnostics by code with up to 3 example paths (marker_mirror_excluded and every REF_TASK_DIAGNOSTIC_CODES code are skipped here; they have their own rows below).
• ref tasks rolls up reading tasks (N live · M archived · K open v1, plus warn-level per-code counts in code order with up to 3 example paths; open_v1_tracker is count-only until migration).
• parents reports parent notes and aliases, or warns with the structural project_name_aliases problems.
• identity reports legacy notes superseded by a newer capture, or warns about identity keys shared by more than one PDF-backed note.
• annotations warns while notes still render a leaked marker mirror; the next bob ref scan removes those blocks without tombstones.
• coverage counts zorg-era status:: reading records in *.md files outside ref/ (hidden directories, _generated/, and *.assets/ excluded), minus records already mirrored into the library. A record is mirrored when some ref note carries the same source_path and either the same source_block with a matching source_id (notes without a source_id still match on block plus path), or the record's block in its source_blocks (one book note mirrors several folded chapters). The warn form names the top 5 files by count.

Available commands:

bob ref create <TARGET>... [-A|--author NAME] [-a|--audio PATH] [-b|--bob-dir PATH] [-C|--closed] [-d|--dry-run] [-f|--force] [-H|--html FILE] [-i|--include-id] [-k|--keep-intake] [-l|--lib-dir PATH] [-L|--listen] [-M|--plain] [-N|--name STEM] [-n|--no-audio] [-o|--output PDF] [-P|--parent NOTE] [-p|--published DATE] [-r|--ref-dir PATH] [-S|--section HEADING]... [-s|--status STATUS] [-T|--title TITLE] [-t|--ref-type DIR] [-x|--xlib-dir PATH]
bob ref doctor [-b|--bob-dir PATH] [-l|--lib-dir PATH] [-n|--no-hooks] [-r|--ref-dir PATH] [-x|--xlib-dir PATH]
bob ref marker <pdf> [-b|--bob-dir PATH] [-l|--lib-dir PATH] [-r|--ref-dir PATH] [-x|--xlib-dir PATH]
bob ref scan [-b|--bob-dir PATH] [-d|--dry-run] [-f|--format human|json] [-j|--jobs N] [-l|--lib-dir PATH] [-n|--no-hooks] [-r|--ref-dir PATH] [-v|--verbose] [-w|--write-pdfs] [-x|--xlib-dir PATH]
bob ref sync <pdf> [-b|--bob-dir PATH] [-d|--dry-run] [-l|--lib-dir PATH] [-p|--prefer marker|frontmatter] [-r|--ref-dir PATH] [-w|--write-pdf] [-x|--xlib-dir PATH]

create captures a web article URL into xlib/blogs/<slug>.pdf through the pinned web-clip adapter and stamps the same page-1 marker plus the web provenance fields (source_url, author, published, captured); the full contract lives in highlights-clip.md.

create accepts a vault note, a Markdown file, a local PDF, a PDF URL, an arXiv paper URL, or a web article URL and stages <xlib-dir>/<ref-type>/<stem>.pdf before inline intake files it to lib/ (by default ~/bob/xlib/notes/<note-stem>_<YYMMDD>.pdf for vault notes, ~/bob/xlib/chat/<basename>.pdf for other Markdown, and ~/bob/xlib/papers/<stem>.pdf for PDFs) unless -o, --output <PDF> selects the complete output path. The full target contract lives in highlights-create.md. --output keeps the supplied filename and requires a nonempty name with a case-insensitive .pdf extension. Relative output values are resolved from the current working directory; a leading ~ is expanded the same way as other Bob path options. --output cannot be combined with --ref-type or --name, because those only participate in default target derivation.

When --output is omitted, the default intake target walks <stem>, <stem>_2, … (see highlights-create.md): a name taken by a different reference is suffixed, and the next scan moves the final intake PDF to <lib-dir>/<ref-type>/<basename>.pdf. An exact --output path keeps today's strict behavior, refusing to create when that archived library PDF or sidecar already exists. An exact path inside the configured intake directory keeps the same intake workflow: the mirrored library destination is <lib-dir>/<intake-relative-path>, including any nested directories, and that occupied library PDF or Markdown/TextBundle sidecar is still refused even with --force. An exact path already inside the configured library treats the selected PDF as the library destination: an existing file requires --force, and there is no second mirrored-destination check. An exact path outside both managed directories does not invent or check an unrelated library destination.

For ordinary Markdown, it uses the frontmatter title, first H1, or file stem in that order, asks pandoc/xelatex for a hyperlinked three-level table of contents and PDF outline bookmarks, and embeds a page-1 /Text annotation containing:

- status: ready
- parent: sase
- title: <document title>

The marker parent is deliberately bare; generated Obsidian frontmatter turns it into a wikilink later. Vault notes infer an eligible area/project parent from the note itself or its parent: links; other targets require -P, --parent. -s, --status and -t, --ref-type override defaults. -d, --dry-run prints the plan with writes: none.

An explicit -P resolves before rendering, fetching, or writing through the shared area/project/inbox resolver; the marker stores the canonical route. It accepts sase, sase.md, and [[sase]] forms (whitespace trimmed, case-insensitive); an exact stem wins, then project_name_aliases (see docs/projects.md §"External names (project_name_aliases)"). The dry run reports the resolution, for example parent bob (project · bob.md · via alias bob-cli). An unknown name fails with near-miss and alias hints; a terminal project (project 'x' is done) and a non-area/project hub (obsidian_ref.md is not an area or project note) report themselves. When no parent can be inferred, a missing -P fails with a bob capture-targets hint. For --output paths, an existing target PDF requires -f, --force; a same-stem Markdown file beside it is always refused because Highlights would interpret it as a sidecar. Default targets instead suffix to <stem>_2, … when taken by a different reference. Dry-run and success output show a mirrored library_destination only for intake targets. Intake and direct library targets recommend bob ref scan; a PDF outside both managed directories states that recursive scan will not discover it and recommends bob ref sync <PDF>. BOB_PANDOC_COMMAND overrides the pandoc executable, and a render failure includes pandoc's diagnostic output plus a hint: line for missing LaTeX packages (or a missing xelatex). -i, --include-id opts into a stable marker ID derived from the Markdown filename. Bob canonicalizes the Markdown source, removes only the final .md extension from the filename, requires the resulting stem to be nonempty valid UTF-8, and embeds it as id:

- status: ready
- parent: sase
- title: <document title>
- id: xprompt_role_binding

The option is intended for research hook execution, for example:

bob ref create --include-id -P sase 202608/xprompt_role_binding/xprompt_role_binding.md

If the filename stem is invalid for marker output, the command fails before running pandoc or creating output directories. Without --include-id, manual create calls keep the previous marker shape and do not add id.

Listen cards in PDFs¶

Markdown authors can wrap a short audio summary in <div class="listen"> to render it as a light-blue callout in the Highlights PDF. A leading ♫ becomes a ▶ Play button when companion audio is bound and the target is in the Bob intake or library. Intake links point to the companion's final lib/ location. Relative links inside the card, such as the narration-script link, are omitted from the PDF because the source file is not available there. Without a companion, the card remains with its ♫ glyph.

create now binds audio from the sase-listen library, not only from a file that was already beside the PDF, and create --listen narrates the target fresh instead of discovering anything. Discovery order, first hit wins (skipped entirely under --listen, whose episode binds as <stem>.mp3 and reports (from --listen)):

1. --audio PATH (missing files and non-audio extensions are errors; --no-audio conflicts with --audio and disables discovery).
2. Frontmatter audio.episode_id resolved through <library>/<id>/manifest.json (audio.file must be a bare filename present in that episode directory; an unresolvable id warns and falls through).
3. Sibling <stem>_narration.md (stripping a trailing __final), hashed with SHA-256 and matched against source.sha256 or script.sha256 in <library>/*/manifest.json (newest created_at wins).

The library root is BOB_HIGHLIGHTS_AUDIO_LIBRARY, then highlights.audio_library in bob/config.yml, then $XDG_DATA_HOME/sase-listen/library, then ~/.local/share/sase-listen/library:

highlights:
  audio_library: "/data/sase-listen/library"
  # `bob ref create --listen` narrates the
  # target with `sase listen render ...`, which must write MP3 audio to {audio}.
  # The managed setup uses the SASE plugin (`sase listen`); a direct
  # `sase-listen render ...` setting migrates automatically.
  # Overridden by BOB_HIGHLIGHTS_LISTEN_COMMAND. See
  # docs/highlights-create.md for the full contract.
  listen_command: sase listen render {target} -e full -o {audio}

The copy lands beside the PDF target with the source extension lowercased (xlib/chat/<stem>.mp3), planned before any write: identical bytes are reused, different bytes refuse without --force, and an existing mirrored lib/<type>/<stem>.<ext> refuses because scan would refuse to move over it. The PDF is rendered to a temp file, the audio is copied atomically (temp file in the destination directory, then rename), and only then is the PDF stamped and installed; a failed install deletes an audio file this run created and never a reused file. Success and dry-run output show audio: <dest> (from <origin>) after the pdf: line, or audio: none; --include-id gains no required flag. Backfill an already-scanned PDF by copying its MP3 into xlib/<type>/<stem>.<ext> for scan to late-pair; do not rerun create --force on an annotated library PDF.

The URI template is selected from BOB_HIGHLIGHTS_AUDIO_LINK_TEMPLATE, then highlights.audio_link_template in bob/config.yml, then the default:

highlights:
  audio_link_template: "obsidian://open?vault={vault}&file={path}"

{vault} is the percent-encoded Bob directory name and {path} is the percent-encoded vault-relative companion path. Set the template to an empty string to disable the button. Only the generated PDF contains the Obsidian URI; the report Markdown stays portable.

Companion audio on scan¶

scan moves same-stem companion audio with its PDF from xlib/<rel> to lib/<rel>. Allowed extensions are .mp3, .m4a, .ogg, and .opus, checked in that order when choosing which file to record on the note.

When an audio file is waiting in xlib/ with no same-stem PDF still in intake, but lib/<rel>.pdf already exists, scan late-pairs it: the audio moves beside that library PDF and the existing reference note gains the player. Audio with no PDF in xlib/ or lib/ stays in intake for the next tick; bob ref doctor reports it as a warning. Destination collisions join the existing pre-write intake conflict report, so scan writes nothing.

The note records a bob-managed audio field from the companion's presence, never from a marker key:

audio: "[[lib/chat/research_swarm_listen_card.mp3]]"

New notes place the Obsidian player once, directly below the generated PDF task:

- [ ] #task #ref [[lib/chat/research_swarm_listen_card.pdf]] #hide ^ref

![[lib/chat/research_swarm_listen_card.mp3]]

## Highlights

Existing notes get the same embed on the first scan that observes a companion while the note still lacks audio. After that field exists, scan never touches the embed again, so a deleted player stays deleted. The managed Highlights region is unchanged. Dry-run reports the planned frontmatter and embed like other note changes.

Backfill an already-read report by copying the MP3 to ~/bob/xlib/<ref-type>/<stem>.mp3; the next scan attaches it. Do not rerun create --force to replace a library PDF that already has Highlights annotations. The vault already tracks *.mp3 (about 2 MB per brief edition).

Path configuration options are -b, --bob-dir <PATH>, -l, --lib-dir <PATH>, -r, --ref-dir <PATH>, and -x, --xlib-dir <PATH>. scan also accepts -j, --jobs <N> and -v, --verbose (the detailed per-PDF plan report; the default scan report is concise).

sync <pdf> --dry-run prints the resolved configuration and planned note/PDF actions without modifying either side. Without --dry-run, the command writes the reference note when frontmatter changes. It only writes the PDF marker when the selected projection needs marker write-back and --write-pdf is supplied. For recursive scans, --write-pdfs is the bulk opt-in for marker write-back; scan --dry-run --write-pdfs remains read-only and previews the same marker updates.

MacBook validation notes¶

Linux-side tests cover synthetic PDF markers, sidecar fixtures, conflict detection, generated highlight rendering, recursive scan preflights, and dirty target refusal. Validate the following against real Highlights-authored files on the MacBook before enabling PDF marker write-back:

• Real Highlights sidecar Markdown may vary from the fixture-backed parser contract. Keep the documented Highlights Note Format settings fixed while testing.
• PDF marker write-back is implemented with native PDF annotation writes, but real Highlights-authored files may expose annotation shapes not covered by Linux fixtures. Use --write-pdf or --write-pdfs only after backing up the target PDFs.
• Scheduled scan should stay dry-run or note-only until the guided acceptance test passes and highlights.coordinate_pdf_writes is on (see PDFs open in Highlights). Until then, bulk PDF marker writes need an explicit backup and a watched terminal run.
• Confirm a real underline renders as underlined bold inside its host quote in bob ref show; verify SGR 1;4 in a color terminal and literal ** in plain output. Enable ref-highlights in Obsidian and check the gold underline in both light and dark themes.

Default Paths¶

The Bob vault root is BOB_DIR, defaulting to ~/bob.

PDFs are discovered under the Highlights library directory:

BOB_HIGHLIGHTS_LIB_DIR=lib

Freshly created PDFs are written under the Highlights intake directory:

BOB_HIGHLIGHTS_XLIB_DIR=xlib

Reference notes are written under:

BOB_HIGHLIGHTS_REF_DIR=ref

Create-time intake is enabled by default. Use -k, --keep-intake to leave a new PDF in xlib/ for a later bob ref scan -i or local timer, or set:

highlights:
  ingest_on_create: false

BOB_HIGHLIGHTS_INGEST_ON_CREATE overrides that setting. 0, false, no, and off disable inline intake; 1, true, yes, and on enable it. Blank means unset. The environment value is case-insensitive.

Writing scans can run a pre-scan hook before xlib intake:

highlights:
  # Runs from the vault root before `bob ref scan` inspects the library.
  # Use it for custom work that must happen before a full scan.
  pre_scan_hook: my_custom_pre_scan

BOB_HIGHLIGHTS_PRE_SCAN_HOOK overrides the config file value. Set it to an empty value to disable a configured hook. The hook runs as sh -c <command> from BOB_DIR and inherits stdout/stderr, so scheduled logs include its output. Bob exports BOB_HIGHLIGHTS_IN_PRE_SCAN_HOOK=1 to the hook child. If it exits non-zero, scan reports the exit code and aborts before intake. A writing scan holds bob_sync.lock before the hook runs, so a hook that itself runs bob vault-sync is a silent no-op: vault-sync skips when that lock is already held. scan --dry-run reports the hook it would run without executing it. Pass -n, --no-hooks to scan or doctor, or place it on the parent command before that subcommand, to ignore the hook from every source. The same parent flag before create, marker, or sync is accepted and ignored. The legacy highlights.pre_scan_command key and BOB_HIGHLIGHTS_PRE_SCAN_COMMAND variable are hard errors.

Relative BOB_HIGHLIGHTS_LIB_DIR, BOB_HIGHLIGHTS_XLIB_DIR, and BOB_HIGHLIGHTS_REF_DIR values are resolved under BOB_DIR. Absolute paths and ~/... paths are used directly after tilde expansion. lib and xlib must be distinct, non-nested directories so intake cannot move PDFs inside the tree being scanned.

During scan, every pending intake PDF maps from xlib/<rel> to lib/<rel>. For example:

~/bob/xlib/chat/report.pdf
~/bob/lib/chat/report.pdf
~/bob/ref/chat/report.md

For ~/bob/lib/books/systems-performance.pdf, the default output note is:

~/bob/ref/books/systems-performance.md

The first path component below the library directory is the reference type. For the example above, generated frontmatter includes:

ref_type: books

Deeper paths preserve the full library-relative path in ref/ while keeping the first component as ref_type:

~/bob/lib/books/performance/systems-performance.pdf
~/bob/ref/books/performance/systems-performance.md

Top-level library PDFs remain supported for existing files and do not derive a ref_type:

~/bob/lib/systems-performance.pdf
~/bob/ref/systems-performance.md

Marker Note Grammar¶

The marker note is the first standalone /Text PDF note annotation on page 1. It is not identified by a sentinel token, and standalone notes on later pages are not marker candidates.

The marker note is an unordered list of key: value pairs:

- status: ready
- parent: obsidian
- title: Systems Performance
- id: systems-performance
- aliases: ["Systems Performance", "Brendan Gregg systems performance"]
- topics: [linux, performance]
- source_url: https://example.com/book
- source_note: zk/harbor_cranes

Allowed list markers:

- key: value
* key: value

status and parent are required marker keys. Marker parent must be a bare note target such as obsidian or Systems Performance; Obsidian wikilinks such as [[obsidian]], aliases, embeds, and block links are rejected in the PDF marker. Generated reference notes still render parent frontmatter as an Obsidian wikilink, for example parent: "[[obsidian]]".

source_note, when present, is a bare vault-relative note route without a .md suffix. It is provenance for a snapshot made from a vault note; marker sync stores it as a canonical frontmatter wikilink such as source_note: "[[zk/harbor_cranes]]". Marker input rejects wikilinks, aliases, embeds, and fragments for this field.

status must be a scalar string and must exactly match one of:

• ready
• next
• wip
• read
• abandoned
• legacy

Existing status: unread and status: done marker/frontmatter/base inputs are accepted only as deprecated aliases and are normalized to ready and read, respectively, during sync.

Values should be parsed as a YAML-compatible subset when possible:

• strings
• numbers
• booleans
• nulls
• inline lists

Invalid values fall back to strings when that is unambiguous. Duplicate marker keys, invalid marker list items, pipeline-owned marker keys, command-managed marker keys such as type or ref_type, and missing or empty status or parent produce clear errors. Unsupported status values are also rejected.

Required Parent, Type, and Ref Type¶

New Markdown notes under ~/bob require parent frontmatter. The PDF marker is authoritative for parent, so the marker must include it.

Generated reference notes also always include command-managed frontmatter:

type: "[[ref]]"

Nested library PDFs also include path-derived frontmatter:

ref_type: books

type and ref_type are replaced on each successful sync, excluded from marker/frontmatter round-trip hashes, and rejected if either appears in the PDF marker note. ref_type is omitted for top-level library PDFs and explicit out-of-library syncs because there is no library category to derive.

Synced Properties¶

The PDF marker note and the reference note frontmatter share a synced user-property projection.

Pipeline/provenance fields are excluded from marker sync:

source_pdf
source_pdf_sha256
highlights_sidecar
highlights_count
highlights_synced_at
highlights_marker_base
highlights_marker_replaced_hash
highlights_marker_hash
highlights_marker_fields
pipeline_version

The command-managed type, ref_type, and audio fields are also excluded from marker sync. type is always rendered as [[ref]]; ref_type is rendered only when the PDF path is under lib/<ref_type>/. audio is rendered as a quoted wikilink to the library companion ("[[lib/<type>/<stem>.mp3]]") when a same-stem audio file sits beside the PDF, and omitted otherwise. It never enters the marker projection, hash, or base.

Standard synced user fields are status, parent, title, id, research, aliases, topics, source_url, source_note, author, published, captured, and return_links. source_note is the canonical source vault route for note snapshots. id is ordinary user frontmatter for the basename-derived source identifier and does not require highlights_marker_fields. Existing research values remain legacy standard fields for repository-relative Markdown source paths; they round-trip without being renamed automatically to id. captured is the snapshot date (YYYY-MM-DD) of a web capture; like the other provenance fields it is omitted rather than written wrong. return_links is stamped true by bob ref create when a Markdown render actually pairs return links, so bob ref sync knows to keep their tag glyphs and return pills out of synced highlight text; it never needs highlights_marker_fields.

Every new reference note also records a note-local created field stamped from the writing invocation's clock (BOB_NOW overrides it for testing):

created: 2026-10-03T15:42:18-0400

The datetime format is YYYY-MM-DDTHH:mm:ss±ZZZZ with seconds and a numeric timezone offset. The field is set once at creation — including sidecar-free notes and notes created after xlib/ intake — and later syncs preserve the authored value verbatim. It never joins the synced projection, marker hash, base snapshot, or highlights_marker_fields (even a stale opt-in list naming it is ignored), and setting it never requires --write-pdf / --write-pdfs or modifies PDF bytes. It differs from captured (the YYYY-MM-DD web-capture snapshot date synced from the marker) and from highlights_synced_at (refreshed whenever sidecar highlights rerender). Notes created before this field existed gain none on sync; their historical creation times are not backfilled. A PDF marker containing created is rejected with a diagnostic explaining that it records reference-note creation time, before any note or PDF write.

Unknown marker keys should round-trip into frontmatter. New frontmatter keys should sync back to the marker only when they are standard supported fields or explicitly listed in highlights_marker_fields.

legacy_status is ordinary frontmatter used to preserve a migrated note's old status value. It is preserved when rendering notes, but it is not a standard marker-synced field and will not be written back to the PDF marker unless a marker explicitly opts into it as an unknown synced field.

Implemented conflict policy:

• If only the marker changed, update frontmatter when any status edit agrees with the visible lifecycle task.
• If only frontmatter changed, update the PDF marker note when PDF writes are enabled and any status edit agrees with the visible lifecycle task.
• Treat the generated PDF task line as the reference-reading status: [ ] means ready, [*] means next, [/] means wip, [x]/[X] means read, and [-] means abandoned. [?] means Blocked: it agrees with any open selected status without contributing, and reopens a terminal selected status to ready (to finish a blocked reference, check the ^ref task [x]).
• If that task status differs from the selected marker/frontmatter projection, apply it as a note-side status signal. If marker or frontmatter also changed status to a different value from the stored base, report a conflict instead of guessing.
• If marker and frontmatter changed different fields from the stored base, auto-merge them. PDF marker writes are still opt-in with targeted sync --write-pdf or bulk scan --write-pdfs.
• If both changed the same field differently, fail without modifying either side unless --prefer marker or --prefer frontmatter is supplied.

The last synced user-property projection is stored as highlights_marker_hash and as compact JSON in highlights_marker_base. After a marker write, the note also stores highlights_marker_replaced_hash: the parent-free projection hash of the marker that write overwrote. A later scan that finds the PDF marker equal to that replaced copy treats it as a stale editor save (see Stale saves). The field is dropped on a later note write that observes the PDF marker matching highlights_marker_hash. A missing field (legacy notes) disables the guard. The hash keeps old-note compatibility; the base snapshot lets the command prove safe field-level merges. --prefer frontmatter, any generated-task lifecycle change, and any auto-merge that includes frontmatter changes require --write-pdf for targeted sync, or --write-pdfs for recursive scan, whenever the PDF marker must be updated.

PDF marker writes are performed by saving a temporary PDF next to the original and renaming it over the target. Before first PDF writes, commit or otherwise back up the PDF library. See PDFs open in Highlights for how marker writes behave when a PDF is open.

PDFs open in Highlights¶

When highlights.coordinate_pdf_writes is on, bob ref scan --write-pdfs and bob ref sync --write-pdf wrap each PDF whose plan needs a marker write in one bounded macOS file-coordination write (NSFileCoordinator, .forMerging). Highlights saves unsaved annotations first, bob re-checks the marker in those fresh bytes, writes on top of them, and Highlights reloads the same tab in place. Create-time ingest uses the same executor.

The key defaults to off until a guided acceptance test on the Mac passes. On Linux the key is accepted and inert: there is no OS coordination, and the reading probe always reports not reading unless an internal override is set.

Holds¶

If anything is uncertain, bob holds that PDF: nothing is written for it, the scan exits 0, and the next scan retries. A hold is not a failure.

Concise scan output prints a yellow held line and a N held summary count. Verbose scan (-v) prints pdf_write_coordination: on and a reading_hold line when the key is on, then pdf_marker_action: updated (coordinated in 0.42 s) or held (<reason>; retrying next scan). bob ref sync uses the same phrases and still exits 0 on a hold.

-a, --attended¶

bob ref scan -a, --attended means a person asked for this scan right now: never hold marker writes because Highlights is in use. It changes nothing else. Bob Refs ⌘S and Bob Mac Capture's scan after ⌘S in Highlights pass it. bob ref sync and create-time ingest never hold for reading. The scheduled cron scan does not pass -a.

Budgets and overrides¶

Defaults are 10 s per PDF, a 30 s scan-wide cap, and a 30 s reading window. Once the cap is spent, remaining marker-write PDFs hold immediately. Internal overrides, for tests and the acceptance kit:

• BOB_REF_PDF_WRITE_BUDGET_SECONDS
• BOB_REF_PDF_SCAN_BUDGET_SECONDS
• BOB_REF_READING_WINDOW_SECONDS
• BOB_REF_READING_PROBE=reading|away
• BOB_REF_FRONTMOST_BUNDLE_ID and BOB_REF_HID_IDLE_SECONDS

Unknown probe results do not hold. Dry runs never coordinate and never probe.

Enabling¶

The key ships off and stays off until the guided acceptance test below passes on the Mac. To enable afterwards, set exactly this in the Mac config file:

highlights:
  coordinate_pdf_writes: true

Until then, an uncoordinated write that races Highlights ends in the "The file has been changed by another application" dialog (see Expected failures): prefer Save Anyway so your annotations are kept. The next -w scan restores the marker; do not edit status: by hand.

Stale saves¶

A PDF marker that matches the copy bob's latest marker write replaced is a stale editor save, not a status edit. The next -w scan keeps the note and the reading-task lane and restores the marker. Concise scan output adds the dim detail marker restored after a stale save. Verbose sync_reason (and ref sync) appends marker reverted to the copy bob replaced (stale editor save); restoring it. JSON notes[].marker_write.restored is true only for that restore.

One limit: deliberately setting the marker back to the exact copy bob replaced is undone. Change status through the reading task instead.

Acceptance kit¶

scripts/dev/highlights_coordination_acceptance is the guided runbook. It creates a throwaway vault and config under $TMPDIR (never ~/bob or the real config), asks you to pick a small library PDF and then a ~20 MB one to copy, and walks the research's nine steps: uncoordinated control, the main save-first case, frontmost and background tabs, a half-typed popover, a modal sheet, a detached SSH/cron run, large-PDF timing, sidecar import, and marker re-location after renumbering. Each step prints what to do in Highlights, runs bob ref sync --write-pdf or bob ref scan --write-pdfs with verbose detail, and records pass/fail in results.md. The closing verdict prints the enable line above when steps 2, 3, 5, and 6 pass; it keeps the key off and flags Plan B if annotations are lost or a dialog appears.

--self-test runs the non-GUI plumbing anywhere without Highlights (scratch vault, coordinated write, receipt, hold, -a, dry-run rule) and is wired into just check-scripts.

Mac notification¶

Every writing bob ref scan that holds the writer lock already drops the scan receipt. Bob Mac Capture watches the matching Darwin notification, including scans started after ⌘S in Highlights, and renders it: one quiet, threaded "Updated in Highlights" notification when Bob Refs is hidden, a footer clause when it is visible, and a "Highlights needs you" banner only for coordination_timeout holds. The kit's last step confirms the notification fires for a scratch scan. Keep Bob Mac Capture's Settings › References toggle on (the default) to receive it.

Scan, Safety, and Git/ob Behavior¶

Create-time intake holds bob_sync.lock before scan.lock, matching the reference writer lock order. It pulls first, rechecks the target against the fresh tree, moves one PDF and its same-stem companions, writes the note and reading task, commits only the paths for that reference, and pushes before releasing the locks. A PDF created directly in lib/ skips the pull so a pre-sync cannot commit that PDF without its note. When the lock, worktree, remote, destination, or write guard prevents intake, create keeps the PDF in xlib/ and reports a queue reason; the command still succeeds so a later scan can drain it.

scan --dry-run reports every discovered PDF, including PDFs that are still pending under xlib. If a pre-scan hook is configured, the dry run reports that it would run and the exact hook command, but does not execute it. The default report is concise. -v, --verbose prints the detailed per-PDF plan, including each planned xlib/<rel> -> lib/<rel> intake move. Dry runs do not move anything. Valid PDFs show their target reference note, sidecar path if present, selected sync source, and note/PDF marker action. Invalid PDFs show a plan_error. Scan output also reports write_pdfs: true|false so bulk marker-write runs are auditable. Dry runs do not create directories, move intake files, write notes, write PDFs, or run the pre-scan hook, even when combined with --write-pdfs.

Before planning notes, writing scans run the optional pre-scan hook and then preflight intake. Pass -n, --no-hooks to skip the hook. Intake moves each PDF under xlib to the mirrored lib path and moves same-stem Markdown sidecars, TextBundle directories, and companion audio with the PDF, so annotation text, image assets, and the listen-card player are not orphaned. Late-pair audio (intake audio whose PDF is already in lib/) moves on the same preflight. If any destination PDF, sidecar, or audio file already exists, the whole scan aborts before moving or writing anything. Intake runs before the dirty-target Git check because a newly rendered intake PDF is normally untracked, and the move never overwrites an existing path; a tracked synced PDF behaves as an ordinary Git rename.

After intake, the command rejects duplicate output paths, builds per-PDF plans, and checks Git status for existing vault files that successfully planned PDFs would modify. If a target ref note or PDF marker target is dirty, it fails before any note/PDF write, except for a tracked target ref note whose only body change is the exact generated ^ref checkbox toggle, optionally combined with frontmatter edits. There is no force mode for scan; commit, stash, or clean unrelated dirty files before rerunning.

Image assets copied from TextBundle sidecars are treated as note-side writes. They are planned during dry runs, checked by the same dirty-target preflight when the destination already exists, and written only during non-dry-run syncs. Scan output includes image totals when a planned PDF contains image selections.

Planning failures for one PDF do not stop valid plans from writing. Write-time failures such as a changed note/PDF or temporary save failure are reported as write_failure entries, and later valid PDFs continue. The scan summary reports plan_failures, write_failures, and scan_failures; any non-zero failure count makes the command exit non-zero even when other PDFs were processed.

Reference note writes and image asset copies are atomic temporary-file renames and are skipped when the rendered note or content-addressed asset is byte-identical to the existing file.

For PDFs under xlib, bob ref create runs vault-sync before intake and again after its scoped commit while holding bob_sync.lock. A PDF created directly under lib/ skips the pull so vault-sync cannot commit it without its note. Writing scans hold the same maintenance lock through intake and note writes, but do not sync themselves. bob nightly runs vault-sync before and after its maintenance commands. Every sync rewrites the source_pdf field, so a direct bob ref sync ~/bob/xlib/... dry run or one-off sync self-heals to the library-relative path on the next scan after intake.

Intake-only scan¶

bob ref scan -i (--intake-only) drains the local xlib/ queue. A create that kept intake (-k, the config switch, or the environment switch) or could not run it (lock busy, no pull, collision, and the other queue reasons) leaves its PDF there; scan -i files it later with the same session create uses. It is meant for a per-host timer: each host drains its own xlib/, and the full bob ref scan stays the annotation sync and repair pass.

bob ref scan -i            # file everything queued on this host
bob ref scan -i -d         # preview: what would be filed or stay queued
bob ref scan -i -f json    # the same drain as one JSON line

What it does, in order:

1. Validates the library layout.
2. Lists the queue without taking any lock: every xlib/ PDF (with its same-stem Markdown, TextBundle, and audio companions), then every xlib/ audio file whose library PDF already exists. Audio with no PDF anywhere is not drainable and is ignored. An empty queue prints intake: nothing queued, takes no lock, runs no sync, and exits 0, so an idle timer costs nothing.
3. With --dry-run, prints would ingest: xlib/… → lib/… → ref/… (or would attach audio:) per item and would queue: for items the local checks already refuse, then exits 0. It takes no lock and does not pull, so a collision that only the pulled tree reveals still appears at drain time.
4. Otherwise opens one session (bob_sync.lock, then scan.lock) and pulls once. Failing to take a lock, to find a Git worktree, to match BOB_DIR, or to pull is a hard error: exit 1, nothing moved.
5. Files each item in sorted order with its own commit, subject bob ref scan: ingest <type>/<stem> (or bob ref scan: attach audio <type>/<stem> for late audio). Planning or write refusals roll the move back. An item that cannot be filed prints queued: <xlib path> (<reason>: <detail>), stays in xlib/, and the drain continues. An item that stops after writing (incomplete:) ends the drain without a push, because the uncommitted writes are not safe to publish; background vault-sync publishes the commits already made.
6. Pushes once at the end (sync: pushed, or sync: push pending (…)), then prints intake: filed N · queued M.

The exit code is 0 when everything was filed and 1 while any item stays queued or stopped, so a stuck item fails its timer run visibly. A library PDF that has no note is never touched; only a full scan repairs that.

--intake-only ignores highlights.ingest_on_create and its environment switch, because running the drain is an explicit request. It skips the pre-scan hook, so JSON reports hook.status: "skipped". It conflicts with -w/--write-pdfs (usage error, exit 2): new references never write PDF bytes.

Typical order: bob ref create files the reference itself; bob ref scan -i drains whatever create had to queue; the full bob ref scan is the annotation sync (and repair) pass for PDFs that already live in lib/.

JSON output¶

bob ref scan -f json (equivalently --format json) prints the same scan as a versioned machine-readable report instead of the human display lines:

bob ref scan -w -f json

Stdout carries exactly one compact JSON line followed by a newline, and nothing else: no intake lines, no per-PDF lines, no summary line, and no ANSI escapes. The pre_scan_hook: run … status line goes to stderr, and the hook child's stdout is redirected to bob's stderr, so a chatty hook cannot corrupt the report. --format json with -v/--verbose is a usage error (exit 2). JSON mode works with --dry-run too and with --intake-only, which reports filed notes in notes, completed PDF moves in intake, and anything still queued in queued.

A successful or partially failed scan prints this envelope (field order is fixed: ok, schema_version, command, generated_at, mode, write_pdfs, hook, intake, summary, notes, failures, intake_only, queued, held):

A hard failure prints the error envelope instead (ok, schema_version, command, generated_at, mode, write_pdfs, intake, error, intake_only, queued), where intake lists any moves that already happened before the failure. queued is always [] there. error.hint is a string or null; error.paths is always an array of vault-relative paths. Exit codes: 0 means ok: true; 1 means per-PDF failures (the full envelope above, ok: false) or a hard failure (the error envelope below, with JSON on stdout either way); 2 is a clap usage error with no JSON on stdout.

Writer lock¶

A writing scan (not a dry run) takes two exclusive locks after layout validation and before the pre-scan hook runs, and holds both until the scan returns:

1. the shared vault-maintenance lock (bob_sync.lock, or BOB_VAULT_SYNC_LOCK_FILE when set);
2. the scan writer lock (<state>/bob-cli/ref/scan.lock).

The order is always bob_sync.lock then scan.lock. That serializes writing scans with vault-sync, nightly, and rename so vault-sync cannot push a library PDF without its ref note, and serializes two writing scans (for example Bob Refs ⌘S, Bob Mac Capture's scan after ⌘S in Highlights, and the 15-minute cron) so they do not race on intake moves. A dry run takes neither lock. Writing scans never run vault-sync themselves; vault-sync's next cycle after the scan releases the lock commits the PDF and note together.

The scan lock file is mode 0600 and the directory 0700. While another command holds the maintenance lock, a scan prints waiting for another vault maintenance run… once to stderr (in both human and JSON modes) and retries every 250 ms for up to 120 s, then fails with lock_busy. The message names the lock path. The hint says vault-sync, nightly, or rename held it; rerun after it finishes. While another scan holds the scan lock, a scan prints waiting for another bob ref scan to finish… once to stderr and retries the same way, then fails with scan_busy. Both waits are overridable with the internal BOB_REF_SCAN_LOCK_WAIT_SECONDS environment variable, for tests. scan --intake-only takes the same two locks in the same order, but with the shorter create-time waits (60 s and 30 s, overridable with BOB_REF_INTAKE_LOCK_WAIT_SECONDS) and holds them for the whole drain.

The pre-scan hook runs under bob_sync.lock. A hook that itself runs bob vault-sync is a silent no-op: vault-sync skips when that lock is already held. Bob exports BOB_HIGHLIGHTS_IN_PRE_SCAN_HOOK=1 so a hook can detect it and avoid shelling back into bob ref scan. A nested writing scan would wait on the held locks and fail with lock_busy. Human mode is otherwise byte-identical, except for the lock-wait lines on stderr and the lock_busy / scan_busy failures.

Scan receipt¶

Every writing bob ref scan that acquired the scan writer lock writes the exact envelope line JSON mode would print to <state>/bob-cli/ref/last-scan.json ($XDG_STATE_HOME, else ~/.local/state). That covers human and JSON modes, full and --intake-only drains, success, partial success, and hard failures after the lock (where the receipt holds the error envelope). In JSON mode the receipt bytes are identical to stdout. Dry runs, ref sync, and scans that never got the lock write no receipt. The receipt is written by temp file plus rename, mode 0600, in a 0700 directory. A receipt failure prints one bob ref: could not write scan receipt: … line to stderr and never changes stdout or the exit code.

On macOS, bob then posts the org.bobs.bob.ref.scan-finished Darwin notification so Bob Mac Capture re-reads the receipt off the main thread. Receipts are serialized by the scan writer lock, so the newest file always describes a scan that actually ran. That also gives unattended cron scans a readable record beyond the (usually empty) cron log.

Generated Body Contract¶

Highlights, image selections, comments on annotations, and standalone non-marker notes sync one way from the PDF/sidecar into the reference note.

For foo.pdf, sidecar discovery looks for foo.md first. If there is no Markdown sidecar, it recognizes foo.textbundle only when the bundle contains text.md or text.markdown; other TextBundle contents fail with an explicit unsupported-sidecar error. Image selections require the referenced asset file to exist relative to the sidecar text file's directory. This makes foo.textbundle/text.md resolve ![](assets/figure.png) to foo.textbundle/assets/figure.png. Plain Markdown sidecars can still reference local image files, but Highlights' plain Markdown export does not include image assets; re-export as TextBundle when area selections should sync. If no sidecar exists, sync still performs marker/frontmatter sync. For a new note it creates an empty managed region; for an existing note it preserves the existing managed region.

Known Highlights TextBundle Creation Bug¶

For PDFs that contain image or area (rectangle) annotations, the Highlights app sometimes fails to create the TextBundle sidecar in the first place, even with autosave enabled. This is a Highlights app limitation, not a bob ref behavior: Bob never creates or mutates sidecars, it only reads the PDF and sidecar content the app owns. Because Highlights can still update a TextBundle that already exists, the workaround is a one-time manual export per affected PDF. After the initial export, Highlights keeps the bundle updated, so this step is only needed once per PDF whose image annotations should sync.

Create or export the bundle beside the PDF, using the exact PDF basename, so that example.pdf is paired with example.textbundle/. For ~/bob/lib/<ref_type>/example.pdf, the expected layout is:

~/bob/lib/<ref_type>/example.pdf
~/bob/lib/<ref_type>/example.textbundle/text.md
~/bob/lib/<ref_type>/example.textbundle/assets/figure.png

The sidecar text must live at ~/bob/lib/<ref_type>/example.textbundle/text.md (or text.markdown), and every referenced image must live under ~/bob/lib/<ref_type>/example.textbundle/assets/.

After creating the bundle, run bob ref scan --dry-run and confirm the affected PDF reports the expected sidecar: path and a nonzero images: count. The user-visible symptoms of a missing or incomplete bundle are:

• A missing .textbundle looks like no sidecar, so bob ref scan performs marker/frontmatter planning but generates no image annotations.
• A .textbundle without text.md or text.markdown surfaces as unsupported textbundle sidecar.
• A bundle whose text.md references images it does not actually contain under assets/ surfaces as image asset not found.

Two Markdown sidecar shapes are supported. The simple shape is:

• Page labels are Markdown headings such as ## Page 12 or ## p. 12.
• Annotation spacers are horizontal rules such as ---.
• A highlight starts with one or more blockquote lines.
• Non-heading text after a highlight is rendered as a highlight comment. A leading Comment: or Note: label is stripped.
• A standalone note is non-heading text outside a blockquote. A leading Note: label is stripped.
• A Markdown image line such as ![alt](assets/figure.png) outside a blockquote is an image selection when the target has a supported image extension. Text after the image is treated as its comment. Multiple images in one chunk render as separate image annotations. Non-image targets such as PDFs remain ordinary standalone note text.
• Text before the first page heading is document preamble (a setext title, an author line, blank lines), never an annotation. Sidecars without any page heading have no preamble.
• A standalone note whose text is a status/parent marker list, or a linked-page highlight whose comment is one, is the PDF marker mirrored into the sidecar — never one of your annotations — and is excluded from generated content no matter where it sits in the sidecar. Color-grouped exports with no page headings may prefix that list with a setext title (raw two-line title + === underline, or the already-rendered single-line title ===... form); both newly parsed and legacy persisted mirrors are excluded, and the next normal scan/sync removes their phantom ^h- IDs without tombstones. Ordinary notes in sidecars without page headings are still retained.

Simple sidecar fragment:

## Page 12

- status: wip
- parent: obsidian

---

> Latency is not throughput.

Comment: Compare this with SLO notes.

---

Note: Keep this standalone observation.

Image selection sidecar fragment:

## Page 12

![Latency figure](assets/figure.png)

Comment: Compare this figure with the latency table on p.14.

The linked-page Highlights export shape is also supported:

• Page labels may be Markdown-link headings such as #### [Page 1](highlights://book#page=1). The link label, Page 1, is used as the rendered page heading.
• Non-page headings such as the document title and dated metadata headings are ignored by the annotation parser.
• Annotation spacers may use any Markdown horizontal rule style, including ***.
• Highlights may be hard-wrapped so that only the first physical line starts with >. Immediate nonblank continuation lines are kept as highlight text, including heading-shaped wrap lines such as # Set default agent. Page headings, color-group headings (#### Yellow), annotation-date headers (##### 2026-06-03:), marker-list fields, and explicit Comment:/Note: labels leave the quote instead. A blank line also ends the quote and starts the comment side. Color-grouped exports with no page headings share this hard-wrap rule; simple ## Page N sidecars keep treating bare lines after the quote as the comment.
• User comments exported as Markdown list items after a highlight are rendered without the list marker.
• A blockquoted marker mirror title followed by marker-list fields such as status and parent is excluded from generated content.

Linked sidecar fragment:

# Highlights Reference Note Sync

#### [Page 1](highlights://highlights-ref-sync#page=1)

##### 2026-06-03:

> Highlights Reference Note Sync

- status: ready
- parent: obsidian

***

#### [Page 2](highlights://highlights-ref-sync#page=2)

##### 2026-06-03:

> It only writes the PDF marker when marker write-back is needed
and the matching write opt-in is supplied.

- Support sase tool call replay?

The generated region is the only body region the tool owns:

<!-- highlights:begin -->

<!-- highlights:end -->

Manual content outside those markers must be preserved. User edits inside the generated region may be overwritten.

Legacy (v1) generated notes include a title, a PDF wikilink Obsidian task line with the #hide tag and the stable ^ref block ID, an Obsidian audio embed when a same-stem companion exists, and ## Highlights. ## Tasks is not part of that skeleton; the section is created later, directly below the ^ref task, when the first annotation task is imported. New v2 notes instead show a managed embed pointing at the parent-note reading task; see Reading tasks in parent notes (v2). Existing notes must already contain the managed begin/end markers; otherwise sync fails instead of guessing where generated content belongs.

Generated annotations render as Obsidian callouts. Highlight text uses a [!quote] callout; a highlight or image comment is nested inside that quote as a [!note] Comment callout so the annotation's trailing ^h-... block ID still covers the quote/image and the comment together. Image selections use [!quote] Image with a vault-relative Obsidian embed. Image notes (a local image path on its own line in the annotation's note) render as embeds inside the same callout, directly below the highlighted text. Standalone non-marker notes render as [!note] callouts. Removed annotation tombstones render under ### Removed highlights as [!warning] Removed highlight callouts.

Example generated body:

### Page 12

> [!quote] Latency is not throughput.
>
> > [!note] Comment Compare this with SLO notes.

^h-2b91f0a4c7de

> [!note] Keep this standalone observation.

^h-8f42a61a90cc

Image selections copy their asset into a per-note assets directory beside the reference note. For ref/books/example.md, the destination is ref/books/example.assets/h-<id>.<ext>, and the generated block embeds it with a vault-relative wikilink:

### Page 12

> [!quote] Image ![[ref/books/example.assets/h-2b91f0a4c7de.png]]
>
> > [!note] Comment Compare this figure with the latency table on p.14.

^h-2b91f0a4c7de

Asset filenames and image block IDs are content-addressed from the source PDF path and image bytes, so re-exported or renamed TextBundle assets keep the same ^h-... block and stored filename when the image bytes are unchanged. Existing matching assets are skipped; an existing destination with different bytes is a write failure instead of an overwrite. When an image selection is removed from the sidecar, the generated block is tombstoned like other removed annotations, but the copied asset file is left in place. Asset garbage collection is future work.

Image notes¶

An image note is a line of an annotation's note whose entire content is one local image path. Put the path on its own line in a highlight comment, a comment on a TextBundle image selection, or a standalone note.

Accepted forms, after trimming, stripping one list marker (-, *, +), stripping a leading Comment:/Note: label, stripping one wrapper (backticks, angle brackets, straight or smart quotes), and stripping a trailing ., ,, or ;:

~/tmp/screenshots/20261018_104022.png
/tmp/screenshots/20261018_104022.png
file:///tmp/Screenshot%202026-10-18.png

Spaces in the path are fine when the whole line is the path. Several whitespace-separated paths on one line become several images. A path in the middle of a sentence, or a relative path, stays ordinary comment text.

Supported extensions are png, jpg, jpeg, gif, webp, and svg (the formats Obsidian renders and the vault gitignore tracks). heic, heif, tif, tiff, bmp, and avif render an "Image type not supported" notice instead of staying silent text.

The copy lives beside TextBundle assets in ref/<type>/<stem>.assets/, named <safe-stem>-<hash8>.<ext>. hash8 is the first 8 hex digits of SHA-256 over the canonical source text (~/tmp/screenshots/20261018_104022.png is never host-expanded, so every host uses the same filename). Example:

ref/papers/foo.assets/20261018_104022-07154fc6.png

The same path on several annotations shares one copy. Images render directly below the highlighted text, inside the [!quote] callout, each preceded by a bare > spacer; remaining comment text follows as the usual nested Comment callout:

> [!quote] Latency is not throughput.
>
> ![[ref/books/example.assets/20261018_104022-07154fc6.png|420]]
>
> > [!note] Comment Compare with the p99 chart.

^h-2b91f0a4c7de

These states are self-healing: the next scan that can read the file copies it and replaces the warning with the image. They never fail the PDF or change the exit status.

Retina screenshots (PNG pHYs denser than about 90 dpi) emit a |<logical> width hint when the logical CSS-pixel width is below the pixel width and at most 600, so a 840× region capture at 144 dpi renders at |420. A full-screen 2880 px capture gets no hint and fits the callout. Non-PNG and malformed files get no hint.

Snapshot semantics: the vault copy is written once and never overwritten. If the destination exists, later edits to the file at the pasted path are ignored. To refresh an image, delete its file under <stem>.assets/ or paste a new screenshot path; the next scan re-imports it. Once the source is gone, the vault copy remains.

The Mac scan resolves macscrot paths (~/tmp/screenshots/YYYYMMDD_HHMMSS.png) locally. athena and apollo hold the same uploads, so a scan there can copy a screenshot that has already been synced across hosts.

Troubleshooting:

• bob ref sync <pdf> --dry-run shows image_notes: and image_note_unresolved: on the verbose plan.
• bob ref doctor lists unresolved_image_note under library diagnostics with example paths.

Continuation highlights (<)¶

A Highlights note that starts with < groups that text highlight with the nearest earlier text highlight when bob ref renders the reference note. Type the marker on the later piece, while looking at the overflow. The group keeps every piece's words, page, images, and comment together. It is stateless and deterministic, and deleting the < removes the group. Nothing is written to the PDF or the sidecar.

If you still see a <, it did not form a group. A dangling marker keeps its raw note, so the < stays visible in the generated comment.

The grammar runs only on text highlights, never on sticky notes (StandaloneNote) or on a TextBundle image selection's caption. Those < characters are literal text: they are not matched, escaped, or counted.

It reads the highlight's task_source after a leading Comment:/Note: label has already been stripped:

1. Take the first line whose trimmed text is non-empty. Every other line stays byte-identical.
2. Set aside leading whitespace and at most one unordered-list prefix (-, *, or +, then whitespace).
3. At the start of the remaining body, test these spellings in this order. Each marker must be followed by whitespace (including NBSP) or the end of the line:
4. Marker: < (ASCII), <… (U+2026), or <.... macOS smart punctuation turns ... into …, so both longer spellings match.
5. Escape: \ immediately followed by one of those spellings that meets the same boundary rule. Remove that one backslash and treat the note as ordinary. Only a would-be marker is unescaped, so \<div> stays exactly as typed.

The effective first line is the set-aside prefix plus the text after the token, with leading whitespace trimmed. If that text is empty, the line is dropped. A dangling marker keeps the raw note; the effective note is used only when the follower actually forms a group, or for an escape.

Previous is the nearest earlier annotation, in sidecar order (Highlights' sidebar order), that is a text highlight and not a marker mirror. The walk skips marker mirrors, sticky notes, and image selections; those keep their own blocks and render after the group, in sidecar order. If that nearest text highlight is itself a follower, the new piece becomes another follower of the same head, so A, <B, <C is one group. If no earlier text highlight exists, the marker dangles and the highlight renders on its own, verbatim. There is no distance limit: a wide page jump shows up through the gap and the heading.

The group renders at the head's position. Exporting Highlights sorted by color or date changes sidecar order, so "previous" is no longer the visual predecessor. That sort is unsupported.

Each highlight is beautified on its own. Text is never fused, hyphens are never healed across pieces, and no […] is inserted. The <… and <... spellings have the same grouping behavior as <.

Heading ranges use the members' page labels: Pages 11–12, Pages 11–13, Pages 11, 14, Pages 11–12, 14. A p. prefix becomes pp.. A single shared label stays as-is. A non-numeric label falls back to the head's label.

Page-turn sidecar:

#### [Page 11](highlights://example#page=11)

> PR Sheriff Standing Orders. On every

***

#### [Page 12](highlights://example#page=12)

> session startup, the sheriff checks open PRs

< PR Merge Master?!

renders as one block with two visibly separate pieces:

### Pages 11–12

> [!quote|Page 11] PR Sheriff Standing Orders. On every
>
> > [!quote|Page 12] session startup, the sheriff checks open PRs
> >
> > > [!note] Comment PR Merge Master?!

^h-ba93bec527bb

Each piece keeps its own page, image notes, and Comment card. A longer group keeps all followers as siblings inside the head quote:

### Pages 6–8

> [!quote|Page 6] In Scion, the unit of work is **a “changeset,” an entry in a
> write-ahead log.** … along with
>
> > [!note] Comment Changesets again.
>
> > [!quote|Page 7] content-addressed pointers to the compressed file contents in blob
> > storage. …
> >
> > ![[ref/blogs/redesigning_source_control.assets/20261020_182622-dcaced15.png|568]]
>
> > [!quote|Page 8] support refs, but since neither creating nor fetching commits
> > involves them, …
> >
> > > [!note] Comment Changesets are what “patches” should be!

^h-40fb23af3c7b

A previously published follower that is later absorbed keeps a redirect stub under ### Removed highlights:

### Removed highlights

> [!info] Grouped highlight
>
> Now shown in [[#^h-ba93bec527bb|the Pages 11–12 group]].

^h-609a556c1cb0

The head ^h- ID is unchanged with or without followers. The follower ID is the same grouped and ungrouped. A stub appears only for a follower that was already published. Existing tasks never have their 🔖 rewritten: a task created before a group keeps pointing at the follower ID, whose stub points to the group. New tasks on a grouped piece keep a per-piece [h::] processed ID and a 🔖 link to the group's head block.

Troubleshooting:

• bob ref sync <pdf> --dry-run shows continuations: when any follower was grouped or any marker dangles, then one continuation_dangling: <page label | no page> (no earlier text highlight) line per dangling marker.
• Concise bob ref scan adds 1 continuation / N continuations and 1 dangling continuation / N dangling continuations after image-note details.
• Scan JSON includes notes[].continuations and notes[].continuations_dangling only when the count is non-zero. schema_version stays 1. highlights_count counts a group once.

Underline emphasis¶

A Highlights underline whose exported quote is fully wrapped in ** is treated as emphasis when its cleaned text occurs exactly once in exactly one text highlight on the same page. The underline's words become a ** run in that highlight's quote; its comment, image notes, and #task join the host block. A whole-quote underline still produces one block. The simple rule: an underlined run inside a quote means an underline merged; a wholly underlined quote means it stayed standalone; the note stores both as **.

For multi-word text, the cleaned underline key must occur in exactly one candidate highlight. Single-word underlines additionally require word boundaries, so proves does not match inside approves. If no host matches, more than one host matches, or the text repeats in its only host, the underline stays separate. Dry-run reasons are no highlight around it, inside N highlights, and text repeats inside its highlight.

The run expands to whole words and is split at line breaks. For example, underlining what didn’ in what didn’t change renders as **what didn’t** change. Comments retain member comments first, then absorbed underline comments in run order; when multiple sources or runs share the block, underline comments get a short quoted run anchor. Image notes follow member images, and tasks retain the underline's own processed identity while linking to the host block. A leading < continuation marker always wins: that underline remains standalone and renders as it does today. Escaped \\< is an ordinary note.

An absorbed underline keeps its original ID out of current_ids and is never tombstoned. A Merged emphasis redirect stub appears only when the ID is in the existing region and either already has a Removed highlights entry or is linked from a Markdown file in the vault. It points to the host quote, and remains a tombstone if the underline is later deleted.

Use bob ref sync <pdf> --dry-run to inspect underlines_merged and each underline_standalone reason before rescanning. Concise scans report merged and standalone counts; JSON adds notes[].underlines_merged and notes[].underlines_standalone only when nonzero. schema_version remains 1.

In Obsidian, enable the ref-highlights CSS snippet so underlined runs in quote callouts draw as semibold full-ink words over a thick gold rule, matching the Bob Mac Capture Refs inspector, while leaving nested Comment callouts alone:

/* bob ref: Highlights exports an underline as **bold**, and bob ref merges it
   into its highlight's [!quote] callout. Draw it as the underline it was:
   full-ink semibold words over a thick gold rule that passes under
   descenders, matching the Bob Mac Capture Refs inspector. Nested Comment
   callouts are not affected. */
.callout[data-callout="quote"] > .callout-title strong,
.callout[data-callout="quote"] > .callout-content > p strong {
  color: var(--text-normal);
  font-weight: var(--font-semibold, 600);
  text-decoration-line: underline;
  text-decoration-color: color-mix(
    in srgb,
    var(--color-yellow) 65%,
    var(--color-orange)
  );
  text-decoration-thickness: 0.15em;
  text-underline-offset: 0.12em;
  text-decoration-skip-ink: none;
}

/* bob ref: a `<` group tucks each later highlight into the first
   highlight's quote callout. Draw the group as one card: every tucked
   highlight opens at a dashed seam, captioned at the right with the page
   it was highlighted on, and keeps its own Comment callout. */
.callout[data-callout="quote"] .callout[data-callout="quote"] {
  margin: 0.8em 0 0;
  padding: 0.8em 0 0;
  border: 0;
  border-top: 1px dashed rgba(var(--callout-color), 0.6);
  border-radius: 0;
  background: transparent;
}
.callout[data-callout="quote"]
  .callout[data-callout="quote"][data-callout-metadata]:not(
    [data-callout-metadata=""]
  )::before {
  content: attr(data-callout-metadata);
  float: right;
  margin: -0.55em 0 0 1em;
  font-size: var(--font-smallest);
  font-variant: all-small-caps;
  letter-spacing: 0.06em;
  line-height: 1;
  color: var(--text-faint);
}

/* bob ref: a `# Name` sticky note opens a section. Its card sits right
   under the section heading: a quiet violet rule, a faint small-caps line
   with the @tag, count, and pages, then your description in muted ink.
   Yellow quotes and gold underlines stay the loudest marks on the page. */
.callout[data-callout="summary"][data-callout-metadata="section"] {
  --callout-color: var(--color-purple-rgb);
  margin: -0.35em 0 1.2em;
  padding: 0.1em 0 0.1em 0.85em;
  border: 0;
  border-left: 3px solid rgba(var(--callout-color), 0.55);
  border-radius: 0;
  background: transparent;
}
.callout[data-callout-metadata="section"] > .callout-title {
  font-size: var(--font-smallest);
  font-variant: all-small-caps;
  letter-spacing: 0.06em;
  color: var(--text-faint);
}
.callout[data-callout-metadata="section"] > .callout-title .callout-icon {
  display: none;
}
.callout[data-callout-metadata="section"] > .callout-content {
  color: var(--text-muted);
}

The source snippet is ~/bob/.obsidian/snippets/ref-highlights.css; add "ref-highlights" to enabledCssSnippets in appearance.json. The underline rule applies to strong text in quote callouts. The nested quote rules turn group pieces into one card with dashed, page-captioned seams. Absorbed underlines share their host highlight's ^h- ID, so underlining a published passage never changes the host ID or later blocks on that page.

Sections (# and @)¶

• A sticky note or a comment whose first line is # Name starts a section. For a sticky, the rest is the description. For a comment, the annotation joins the section, and the rest stays its comment.
• Tag a note with @name or #name at the start or end of a line to file it there. If you can still see the tag, it did not group.
• The space matters: # Memory declares, and #memory joins.

In Highlights, write the declaration in a standalone sticky note or as the first line of a highlight, underline, or image comment. Its first non-empty line must be one #, a space, and a title, after an optional Comment: or Note: label. Exported date headings and an optional Note: label before a sticky line are ignored. Add optional aliases at the end, such as # Open questions @q or # Open questions #q; the first alias becomes the card's tag (which always shows the @ spelling), while both the title and aliases can match annotations. For a sticky, the remaining lines are its description. For a comment, the declaration line (and one blank line after it) is removed and any remaining text stays that annotation's own comment; an emptied comment means no Comment card, and comment declarations contribute no description. Only the first comment line declares; later # lines stay literal, and \# escapes. Rejected declarations remain visible as sticky notes (or literal escaped lines for comments).

A positional tag is a whole @name or #name token at the beginning or end of a line, after an optional Note: or Comment: label and one list marker; one line may mix both sigils. Matching ignores case and punctuation between letters and digits. A #name must end at whitespace or the end of the line, and must contain more than digits. #task is never a section tag, and nested Obsidian tags never group. Tags can be on highlight comments, underline comments, image comments, or ordinary sticky notes. Quoted PDF text and image paths are not scanned. Tags in the middle of prose, code, or a #task line do not group. A backslash escapes a tag. A matched #name is consumed, so it never becomes an Obsidian tag on the ref note. An unmatched one stays an ordinary Obsidian tag. \#name renders exactly as typed, which Obsidian shows as plain #name text.

Sections are real headings in the generated note, so they appear in Obsidian's Outline and can be folded, linked, or embedded. Each section card shows its tag, member count, pages, and description. For example:

### Memory

> [!summary|section] @memory · 1 highlight · Page 4
> Ideas to remember.

^s-memory

#### Page 4

> [!quote] A useful passage.

^h-496b1ca10117

### Other highlights

#### Page 2

> [!quote] A passage without a section tag.

^h-38a082b00de9

Each displayed block has one home. For a continuation group, the head's tags are checked first, then its followers, then absorbed underlines in sidecar order. The first matching section wins. Repeated tags for that section are consumed; tags for another section stay visible and report section_conflict. Removing a tag or declaration re-derives membership on the next scan.

An empty declaration still creates a section card that says “nothing tagged yet.” Sections render in declaration order, followed by Other highlights if any blocks remain, then Removed highlights. Stack the declaration stickies on page 1 to set the Outline order. Within each section, annotations keep sidecar order and page captions restart at each section heading.

Section tags do not change published ^h- IDs or recreate tasks. Section IDs use ^s- blocks; the declaration's #task items link to that section block. Nothing new is stored in frontmatter. Scan dry-run output reports section diagnostics beside the existing per-PDF details, and concise scans include section and unknown-tag counts. The JSON fields notes[].sections and notes[].section_tags_unknown are omitted when zero; the schema version stays 1.

Install the release on every host (Mac, athena, and apollo) before writing the first declaration; install everywhere first, then you hold the switch by adding a # sticky. Highlights on the Mac exports a comment's # Name line verbatim (observed in a real sidecar on 2026-10-24); sticky notes and iPad exports are still unverified.

The accepted costs are one home per displayed block, page order is grouped by section in a sectioned note, and Highlights has no section-tag autocomplete.

The existing ref-highlights snippet also styles section cards with a quiet violet rule and muted description. This CSS has not been tested in Obsidian. Check reading view and Live Preview, in light and dark themes and on mobile.

Annotation text is beautified only while rendering the generated region:

• Consecutive prose lines are reflowed into one line. Blank lines remain paragraph breaks.
• Markdown unordered list lines (-, *, +, including task checkboxes) start their own logical lines, and wrapped continuation text joins onto that list item.
• Hyphenated PDF line endings are healed at join points: lowercase next words drop the hyphen (through- + put -> throughput), while uppercase or digit next fragments keep it (Marie- + Curie -> Marie-Curie).
• Ligature glyphs, non-breaking spaces, soft hyphens, zero-width characters, doubled spaces, and tab runs are normalized away.
• For PDFs whose projection has return_links: true, highlight text drops return-link tag glyphs and ↩ p. N pill fragments before that cleanup.

The cleanup is render-only for display, but block identity uses the same normalized semantic text: return-link pill/tag glyphs and underline ** emphasis are stripped, PDF text is beautified, and whitespace is collapsed, so emphasis and return-link decoration never change an ID. Block IDs, annotation-task source links, and processed [h:: ...] markers use that stable content key, so unchanged sidecar annotations keep their existing ^h-... IDs when a note is regenerated with the callout format.

The legacy (v1) generated task line is the reference-reading lifecycle:

- [ ] #task #ref [[lib/books/example.pdf]] #hide ^ref

When a companion audio file is present, the native player sits directly below that task block (after its indented child lines, if any), before ## Highlights:

- [ ] #task #ref [[lib/books/example.pdf]] #hide ^ref

![[lib/books/example.mp3]]

## Highlights

The #ref tag immediately after #task marks this as the machine-managed reference lifecycle task so Obsidian task views can tell it apart from ordinary annotation-derived follow-up tasks. It is additive: legacy generated lines without #ref are still recognized.

The ^ref block ID remains part of the reference lifecycle and link contract; it does not create a freshness tracker or review group. Reference tasks use ordinary freshness and review rules (including #hide exclusion), as described in docs/freshness.md. PRE/POST checklist membership and recurring occurrence rules still take precedence. Sync never auto-confirms a task.

The mapping is bidirectional. On read, the checkbox is authoritative and moves frontmatter and the PDF marker to its status. On generation or write-back, the final status selects the matching checkbox. legacy has no lifecycle checkbox and falls back to [ ]; legacy notes without a generated ^ref task remain untouched. unread is accepted only as a deprecated alias of ready, while done remains a deprecated alias of read.

Blocked [?] is a status-neutral overlay, not a new reference status: it agrees with any open selected status (ready, next, wip, legacy) and contributes nothing, so the stored lane stays as is. With a terminal selected status it acts like a reopen and targets ready through the usual conflict check. Sync never rewrites a Blocked task to [ ], [*], or [/] when the final status is open, because bob task-status-hooks owns that recovery; when the hooks unblock the task, the next sync applies the new mark normally. To finish a blocked reference, check the ^ref task [x]; the hooks leave done tasks alone. Toggling only the checkbox to or from [?] counts as a checkbox-only body change.

Changing [x] or [-] back to [ ] therefore reopens a reference to ready; use [/] to resume it directly as wip. Metadata such as [cancelled:: 2026-06-04] is preserved when only the checkbox mark changes. If the marker or frontmatter also moved status to a different value from the stored base, the task signal reports a conflict rather than silently choosing a side. When editing status directly, move the visible ^ref task to the same state.

When sync itself closes the ^ref task, it stamps the close date on the line: a generated [x] carries [completion:: YYYY-MM-DD] and a generated [-] carries [cancelled:: YYYY-MM-DD], inserted immediately before the ^ref token and dated from BOB_NOW (else the local date). A line that already carries a bracket or emoji close date is never restamped, a checkbox you closed yourself gains no stamp, and reopening never removes the date.

If a lifecycle change would update the PDF marker, sync --dry-run previews pdf_marker_action: would-update, plain sync refuses before writes, and targeted sync --write-pdf writes the marker. scan --dry-run previews this work. A writing scan keeps the default note-only refusal unless --write-pdfs is supplied.

Highlight comments and standalone non-marker notes can also create actionable Obsidian tasks when the marker/frontmatter-selected PDF status is open (ready, next, or wip), before the generated PDF ^ref checkbox contributes another lifecycle status. A run that closes an open reference to read or abandoned still imports newly added annotation tasks in that same run, and reopening a terminal reference imports them too. Runs whose status is terminal (read, abandoned, or legacy) both before and after the reading-task signal skip task intake. Any unordered Markdown bullet line whose item text contains #task as a whitespace-delimited token is copied to an unchecked task:

- #task Compare this claim with the appendix.
- [x] #task Email the citation to Alex.
- #task Send the quote to Alice @alice

Legacy (v1) created tasks go into the note's ## Tasks section. If the note has no Tasks heading, one is created as ## Tasks directly below the generated PDF ^ref task block (the task line plus its indented child lines, e.g. a DEPENDS ON line), separated from it by a blank line. An existing Tasks heading at any level is reused. New tasks are appended after the tasks already in the section:

## Tasks

- [ ] #task Compare this claim with the appendix. [[#^h-2b91f0a4c7de|🔖]] [h:: 4c0a13d2...] [created::2026-06-07]
- [ ] #task Email the citation to Alex. [[#^h-2b91f0a4c7de|🔖]] [h:: 910f6ce7...] [created::2026-06-07]

If the final whitespace-delimited token is a strict @name route suffix, the suffix is removed from the created task text and the task is appended to the end of existing root-level note ~/bob/name.md instead. Routed tasks keep that append-at-end behavior; this command does not create a ## Tasks section on the routed note. Route names must match @([A-Za-z0-9][A-Za-z0-9_-]*); tokens with punctuation, dots, slashes, empty names, or path traversal content are ordinary task text. Routed target notes must already exist and must not be directories, because new root notes need explicit frontmatter that this command cannot infer. For example, @alice creates this line in ~/bob/alice.md:

- [ ] #task Send the quote to Alice [[ref/books/example#^h-2b91f0a4c7de|🔖]] [h:: 9b31c4a0...] [created::2026-06-07]

Each created task carries a block backlink to the annotation-level generated source block, rendered as an aliased 🔖 link between the task prose and processed/created properties. Same-note tasks use compact same-file targets ([[#^h-...]]). Routed tasks use full vault-relative note targets ([[ref/books/example#^h-...]]) so the source note remains recoverable after the task moves to another note or done/. The managed Highlights region keeps the annotation-level ^h-... block and does not attach task-specific ^ht-... block IDs to source task lines:

> [!quote] Highlighted claim.
>
> > [!note] Comment #task Compare this claim with the appendix.
> > #task Email the citation to Alex.

^h-2b91f0a4c7de

The [h:: ...] processed ID is computed from the source reference note path, source annotation block, and normalized task identity. Duplicate identical #task bullets in the same annotation still create only one task. The property is the durable processed marker that moves with the task when it is completed, cancelled, edited, or archived by bob task archive (formerly bob move-done-tasks, still accepted), so later syncs do not recreate it and do not write any processed state back into the PDF or sidecar. Older tasks that already have [highlight_task:: ...] are still recognized for compatibility, and tasks created by the old ^ht implementation are recognized by their #^ht-... backlinks. New tasks do not write [highlight_task:: ...] or ^ht source anchors. If both processed properties and old source backlinks are removed from a moved or heavily edited task, the legacy normalized text fallback only covers unchanged text.

Annotation-created tasks are independent tasks, not subtasks of the PDF reading-status task. Later syncs preserve existing checkbox state and task properties such as [completion::], [cancelled::], [due::], or edited priority fields. Completing or cancelling these annotation-created tasks does not update the PDF marker or reference-note reading status.

Generated blocks use Obsidian block IDs beginning with ^h-. Text highlight and standalone-note IDs are deterministic content hashes over source PDF path, page label, annotation kind, and normalized quote/note text, with no sidecar order. The normalized text strips return-link pill/tag glyphs and underline ** emphasis, beautifies PDF text, and collapses whitespace, so insertion above, emphasis, and return-link decoration never change an ID. Identical text on the same page, kind, and page shares one content key with a deterministic collision disambiguator only within that group; the same text on different pages or kinds hashes differently. Existing live and removed IDs are reused one to one for provably unchanged annotations before allocating new IDs, and are reserved against accidental reuse, so inserting annotations never renumbers unrelated blocks. Highlight comments are not part of the hash, so editing only a comment updates the block without changing its ID. Image IDs use the source PDF path plus image bytes instead, so asset renames and nearby annotation order changes do not churn the image block. If a previously generated block disappears from the sidecar, the command keeps the old block ID under ### Removed highlights with a tombstone message. Absorbed continuation followers never tombstone: the group keeps the head ^h- ID, and a previously published follower gets a redirect stub under ### Removed highlights that links to the group. Annotation tasks keep a per-piece [h::] processed ID (so a < toggle does not recreate the task) and a 🔖 backlink to the group's head block. Previously leaked absorbed underlines are never tombstoned. A Merged emphasis stub is emitted only when the underline ID is in the existing region and is already listed in Removed highlights or linked from a Markdown file in the vault; it points to the host block. marker-mirror and preamble blocks are the exception: when the existing region shows a vanished block as mirror-shaped or sitting before the first page heading, sync drops it without a tombstone and every genuine block ID stays stable. Editing text highlight content itself mints a new block ID; a task created earlier keeps its original link, which then targets the tombstoned block under ### Removed highlights — still a valid, resolvable jump.

Reading Tasks In Parent Notes (v2)¶

A newly scanned reference files one ordinary reading task in its area, project, or inbox note; the reference note itself shows a managed embed pointing at that task. An existing note stays on this branch when the locator finds its task outside the reference note (live or archived) or the body already carries the managed embed. Notes with an in-note ^ref tracker, and trackerless legacy notes, keep the v1 behavior documented above byte-for-byte.

Birth inserts - [ ] #task #ref [[ref/books/example|Title]] [created::YYYY-MM-DD] ^ref-example into the resolved parent: the marker parent through the shared area/project/inbox resolver, or mac_inbox with a ⚠️ parent '<hint>' is not an open area or project child bullet when the hint is unresolvable, ambiguous, or terminal. A uniquely located orphan is adopted before any insertion; ambiguous or archive-open candidates refuse instead of guessing. IDs allocate as ref-<stem> with numeric suffixes on destination or archive collision, honoring BOB_NOW for the created date. Births carry no #hide or [fresh::] tags, and a birth at a closed status stamps its close date like any generated close. A repeated run after a successful sync is a no-op.

The reference note renders frontmatter parent from the task's residence, while stored sync bases and hashes exclude parent, so refiling the task repoints the parent without reading as a two-sided edit. The body carries exactly one managed embed, ![[<task-path-without-md>#^<id>]], on its own line one blank below the H1 (archive tasks address done/...). Sync heals a missing, stale, or deleted embed; authored material around the managed slot is preserved, and unrelated authored embeds elsewhere are untouched. Companion audio anchors directly after the managed embed on new notes.

Located-task status compares the task checkbox against the stored base: a marker- or frontmatter-only move drives a checkbox line edit, while incompatible moves on both sides still conflict. [?] stays a status-neutral overlay. A closed archive task supplies terminal state; a deliberate marker/frontmatter reopen files a fresh open task in the archive's source parent (or the inbox with a warning) and may keep the renamed address when it is free. An open v2 note with no task anywhere gains no replacement: sync reports open_ref_without_task with restore-from-git/set-abandoned guidance, and duplicate live candidates refuse status and parent writes. Scan never mutates done/.

Each PDF writes in parent-first order: destination reading-task and routed annotation insertions first, then the explicitly authorized PDF marker write, then the reference note with the actual final ID. The destination write rereads immediately before touching the line, so a moved or altered task fails with reading task changed during sync; rerun before any marker or ref-note write for that PDF. There is no multi-file transaction: when a later marker or ref write fails after the parent succeeded, rerunning adopts the existing task and finishes without duplication.

Destination notes have no git-dirty veto; capture's preimage-checked writer rebuilds from fresh bytes on bounded retries (preimage races only), so unrelated user additions survive. V2 default annotation destinations share reading-task notes and accept unrelated dirty edits there; explicit routes keep the legacy dirty refusal, and a mixed default/explicit share keeps the legacy policy. Routed dedup commits only after its destination write succeeds (or fresh disk proves the task exists), so a failed destination never consumes a later PDF's follow-up. Before any write, sync revalidates the selected reading line, the ref-note preimage, the authorized PDF preimage, and asset preconditions; a changed, deleted, or ambiguous line fails with the reread error before any destination, marker, or ref write. The ref-note guard additionally allows edits confined to the managed embed slot (including deletion and repointing) plus residence-only frontmatter changes; unrelated body edits still refuse, and the v1 guard is unchanged. A parent-only move never demands marker write-back; writing runs refuse before any write when a non-parent synced field (including lifecycle status and normalization) needs marker write-back without --write-pdf / --write-pdfs, while dry runs preview it. Only the opt-in refreshes a stale marker hint to the residence. Ambiguous births and archive-open candidates refuse status, parent, task, and embed writes with per-PDF diagnostics (partial scans continue).

Unqualified annotation follow-ups file into the reading task's residence (or mac_inbox when there is none, never an archive) with full-path [[ref/<type>/<stem>#^h-…|🔖]] source links, rebased against fresh destination bytes at execution. Explicit @name routes and v1 local-note defaults are unchanged.

Concise human scans report 📖 N reading tasks created · <dest> (n) and ↻ N reading tasks updated from actual successful writes only: adoptions are not creations, unchanged tasks are not updates, and failures never count. Remaining unmigrated references print as N open v1 ref tasks · run bob ref migrate-tasks. Verbose, dry-run, and single-PDF reports name the destination, preview or final ID, adoptions, and planned or performed line edits. JSON output is unchanged.

MacBook Setup Guide¶

Run these steps on the MacBook. The intended checkout is ~/projects/bob-cli, and the intended vault paths are ~/bob/xlib for PDF intake, ~/bob/lib for archived PDFs, and ~/bob/ref for generated reference notes.

Some vaults also have a lit/ folder from an older layout. That is not the production library path. The defaults remain ~/bob/xlib (intake), ~/bob/lib (archived PDFs), and ~/bob/ref (generated notes). For a one-off test against lit, pass --lib-dir lit explicitly.

Install prerequisites if needed:

xcode-select --install
command -v cargo >/dev/null || curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"

Clone or update bob-cli, then install the local checkout:

mkdir -p ~/projects
if [ -d ~/projects/bob-cli/.git ]; then
  git -C ~/projects/bob-cli pull --ff-only
else
  git clone git@github.com:bobs-org/bob-cli.git ~/projects/bob-cli
fi
cargo install --path ~/projects/bob-cli --locked --force
bob ref --help

Create or confirm the vault layout:

mkdir -p ~/bob/xlib/chat ~/bob/lib/books ~/bob/ref
git -C ~/bob status --short

In Highlights Pro on the MacBook:

• By default, bob ref create files the PDF, ref note, and reading task together. A kept or queued capture remains in ~/bob/xlib/<ref_type>/ for bob ref scan -i or the host's local intake timer.
• Keep existing PDFs that should sync under ~/bob/lib/<ref_type>/, such as ~/bob/lib/books.
• Enable autosaved sidecars next to each PDF. Plain Markdown creates ~/bob/lib/books/example.md; TextBundle creates ~/bob/lib/books/example.textbundle/text.md.
• Use TextBundle export for PDFs where Highlights area/rectangle selections should sync, because plain Markdown export does not include image assets.
• For PDFs with image or area annotations, Highlights may fail to create the TextBundle the first time even with autosave on. Manually export/create example.textbundle/ beside example.pdf once, then confirm it with bob ref scan --dry-run. See Known Highlights TextBundle Creation Bug.
• Lock the Highlights Note Format to the sidecar contract above: page headings, --- annotation separators, highlights as blockquote lines, highlight comments as plain paragraphs, standalone notes as plain paragraphs, and image selections as Markdown image links.
• Add exactly one marker note as the first standalone /Text PDF note annotation on page 1.
• Put at least status and parent in that page-1 standalone note.

Use this marker as a starting point:

- status: ready
- parent: obsidian
- title: Example Title
- topics: [example]

Run the initial checks:

bob ref doctor
bob ref scan --dry-run
bob ref sync ~/bob/lib/books/example.pdf --dry-run
bob ref marker ~/bob/lib/books/example.pdf

MacBook validation checklist:

• cargo install --path ~/projects/bob-cli --locked --force installs the local checkout.
• bob ref doctor reports valid vault/library/ref paths, the xlib intake path and pending count, the pre-scan hook status, marker readability, PDF write coordination (on/off on macOS, n/a elsewhere), Git status, optional ob availability, and the library health and coverage rows. A missing ~/bob/xlib is warning-only because create creates it on demand.
• bob ref scan --dry-run lists the expected PDFs under ~/bob/lib and any pending ~/bob/xlib intake moves, reports the intended ~/bob/ref/<ref_type>/*.md targets, and prints writes: none. If scan_failures is non-zero, inspect the per-PDF plan_error lines while noting that valid PDFs were still reported.
• bob ref sync ~/bob/lib/books/example.pdf --dry-run shows the expected marker page/note, sync source, sidecar path, note action, and no writes.
• The first real note write creates or updates ~/bob/ref/books/example.md with parent, type: "[[ref]]", ref_type: books, pipeline metadata, manual sections, and the managed Highlights region.
• For a TextBundle image selection, the first real write also creates ~/bob/ref/books/example.assets/h-<id>.<ext> and embeds it from the generated callout with a vault-relative wikilink.
• For an image note (a screenshot path on its own line in a highlight comment), the first real write copies it into ~/bob/ref/<type>/<stem>.assets/<safe-stem>-<hash8>.<ext> and embeds it under the quote. bob ref sync <pdf> --dry-run reports image_notes: / image_note_unresolved:; bob ref doctor lists unresolved_image_note.
• For a continuation highlight (< on the later piece), the first real write groups that quote with the previous text highlight. bob ref sync <pdf> --dry-run reports continuations: / continuation_dangling:.
• A second run with unchanged inputs reports writes: none.
• If frontmatter edits or a generated lifecycle-task change require PDF marker write-back, a dry run reports pdf_marker_action: would-update before any targeted --write-pdf or bulk --write-pdfs run.
• git -C ~/bob status --short is reviewed before and after each write pass.

The sync model is deliberately asymmetric:

• Marker/frontmatter fields are 2-way and can conflict.
• Highlights, highlight comments, and standalone non-marker notes are PDF or sidecar to reference note only.
• Annotation-created #task bullets land in the v1 reference note, the v2 reading task's residence (or mac_inbox), or an explicit @name root note, and do not sync completion state back to the PDF marker.
• Edits inside the managed <!-- highlights:begin --> region in Obsidian may be overwritten.

Enable note writes only after reviewing the dry-run output:

git -C ~/bob status --short
bob ref sync ~/bob/lib/books/example.pdf
bob ref scan

scan does not enable PDF marker write-back by default. If a dry run reports pdf_markers_would_update, review those PDFs and back up the library before bulk write-back:

bob ref scan --dry-run --write-pdfs
bob ref scan --write-pdfs

For a single PDF, keep using the targeted singular flag:

bob ref sync ~/bob/lib/books/example.pdf --dry-run
bob ref sync ~/bob/lib/books/example.pdf --write-pdf

The intended frontmatter edit workflow is:

$EDITOR ~/bob/ref/books/example.md
bob ref sync ~/bob/lib/books/example.pdf --dry-run
bob ref sync ~/bob/lib/books/example.pdf --write-pdf

If the ref note is tracked in Git, the write-back command may update that dirty note only when the dirty changes are unstaged frontmatter edits and/or the exact generated ^ref checkbox toggle, and the file still matches what the command planned from. Other body edits, managed-region edits, staged changes, untracked notes, and dirty PDFs are still refused.

See PDFs open in Highlights for how marker writes behave when a PDF is open.

Scheduled Scan¶

The MacBook's staggered cron job runs the full bob ref scan for annotation sync and also processes the Mac's own xlib/ queue. It does not pull or read athena's or apollo's queue. See vault-git-sync.md for the managed schedule; do not replace it with a LaunchAgent.

On athena and apollo, chezmoi enables bob-ref-intake.timer, which runs bob ref scan --intake-only two minutes after boot and every two minutes. Each timer drains only its host's local queue. Check or recover a queue with:

bob ref doctor
bob ref scan -i --dry-run
bob ref scan -i

The intake-only command files queued PDFs and late audio without running the annotation pass or pre-scan hook. The full scan remains the Mac's annotation sync and can also process captures queued locally on the Mac.

Disable One PDF¶

There is no marker key such as sync: false. scan processes every .pdf under the configured library directory.

To disable one PDF, move that PDF and its sidecar out of ~/bob/lib:

mkdir -p ~/bob/lib-disabled
mv ~/bob/lib/books/example.pdf ~/bob/lib-disabled/
mv ~/bob/lib/books/example.md ~/bob/lib-disabled/ 2>/dev/null || true
mv ~/bob/lib/books/example.textbundle ~/bob/lib-disabled/ 2>/dev/null || true
git -C ~/bob status --short

The existing ~/bob/ref/books/example.md is no longer touched once the PDF is outside the scanned library. Leave it in place for archival use, or move it explicitly:

mkdir -p ~/bob/ref-disabled
mv ~/bob/ref/books/example.md ~/bob/ref-disabled/

Do not remove the managed Highlights markers as a disable mechanism. If a PDF is still scanned and its existing reference note lacks the managed markers, the command fails before guessing where generated content belongs.

Backup and Rollback¶

Before first writes, make the vault clean or intentionally checkpointed:

git -C ~/bob status --short
git -C ~/bob stash push -u -m "pre-highlights"
git -C ~/bob status --short

If the current vault changes should be kept as the baseline, commit them instead of stashing:

git -C ~/bob add ref lib
git -C ~/bob commit -m "Checkpoint before highlights sync"

Before enabling --write-pdf or --write-pdfs, keep a PDF backup outside the write path:

backup_dir=~/bob/backups/highlights/$(date +%Y%m%d-%H%M%S)
mkdir -p "$backup_dir/lib/books"
cp -p ~/bob/lib/books/example.pdf "$backup_dir/lib/books/"
cp -p ~/bob/lib/books/example.md "$backup_dir/lib/books/" 2>/dev/null || true

For a full library backup before broader testing:

backup_dir=~/bob/backups/highlights/$(date +%Y%m%d-%H%M%S)
mkdir -p "$backup_dir"
rsync -a --include='*/' --include='*.pdf' --include='*.md' --include='*.textbundle/***' --exclude='*' ~/bob/lib/ "$backup_dir/lib/"

Inspect note writes:

git -C ~/bob diff -- ref/books/example.md
git -C ~/bob status --short

Rollback a generated reference note:

git -C ~/bob restore -- ref/books/example.md

Rollback a PDF marker write from the backup:

cp -p "$backup_dir/lib/books/example.pdf" ~/bob/lib/books/example.pdf

Conflict Resolution¶

When the marker and frontmatter both changed since the last synced projection, the command compares both sides to highlights_marker_base. Non-overlapping field edits auto-merge and dry runs report sync_source: auto-merge. Same-field conflicts still fail and write nothing. Inspect both sides:

bob ref marker ~/bob/lib/books/example.pdf
sed -n '1,120p' ~/bob/ref/books/example.md
bob ref sync ~/bob/lib/books/example.pdf --dry-run

Choose the PDF marker as the source of truth:

bob ref sync ~/bob/lib/books/example.pdf --prefer marker

Choose the Obsidian frontmatter as the source of truth and write it back to the PDF marker:

bob ref sync ~/bob/lib/books/example.pdf --prefer frontmatter --write-pdf

If the only change is frontmatter or a generated lifecycle-task mark, or a dry-run auto-merge reports pdf_marker_action: would-update, review the marker first, back up the PDF, then run the targeted write:

bob ref sync ~/bob/lib/books/example.pdf --write-pdf

For reviewed bulk scan write-back, preview the library and then opt in:

bob ref scan --dry-run --write-pdfs
bob ref scan --write-pdfs

Expected Failures¶

Common failure snippets and fixes:

During scan, marker validation, sidecar validation, managed-region validation, --write-pdf refusals, and per-PDF write races are reported for the affected PDF while unrelated valid PDFs may still be planned or written. The summary shows result: partial-failure and the command exits non-zero when any PDF fails. Library discovery errors, output path collisions, and dirty target preflight failures remain hard global failures before writes.