Docs

Review analytics

How review actually behaves on your repo. Five rollups folded from the shape-only PR-review events RepoOps already ingests, joined to merged PRs from git history. No new ingestion, and no review prose is stored: the hosted webhook grades each body once at capture, then keeps only the number.

Where to find it

  • Localhost: /review-analytics.html?repo=<id> - a sub-tab of the Review queue (sidebar: Guard Review queue Review analytics)
  • API: GET /api/review-analytics (optional repo, windowDays 1 to 365, periodDays 1 to 90)
  • Keyboard: ⌘ K then review analytics

The five rollups

Cycles to merge.Each change-requested review forces another round, so a reviewed PR's cycle count is its change requests plus one. A PR that went straight to approval merged in one cycle. Reported as average and median over reviewed PRs.
Reviewer load.Per reviewer: review events, approvals, change requests, comments, and distinct PRs touched. Each reviewer carries a human, agent, or bot kind from the same classifier the attestation pipeline uses, so agent reviewers are visible, not folded into the humans.
Review depth (count).Substantive reviews (approvals plus change requests) per merged PR. A comment-only pass is counted in reviewer load but does not count as depth.
Review depth (score).A 0-100 grade of what each review actually said, computed at capture from the review body (see the rubric below). Averaged per PR, per reviewer, and per period, beside the count-based depth, never conflated with it.
Open to first review.Hours from PR open to the first review of any state. Honest-null when a timestamp is missing; a negative delta never fabricates a number.

The depth-score rubric

When the GitHub App webhook receives a review, it grades the review body under a deterministic rubric and stores only the number plus the rubric version (rubric-v1). The prose is read once inside that request and discarded; it never reaches the database, the export, or your desktop. The score is a capped sum of five signals:

  • Substance (up to 40): 2 points per word left after stripping boilerplate approval phrases ("LGTM", "looks good", "ship it", emoji) and code blocks
  • Code references (up to 24): 8 points per inline `code` span or fenced block
  • Suggestion blocks (up to 24): 12 points per ```suggestion fence, a concrete proposed change
  • Questions (up to 12): 4 points per question asked of the author
  • File and line anchors (up to 18): 6 points per file.ext:123, "line 45", or path reference

An empty review body (an approval submitted with no text) scores null, nothing to grade. "LGTM" alone scores about 0: it was graded, and it is shallow. A review that walks the diff, references code, proposes a suggestion, and asks questions scores high. The rubric is deterministic on purpose: the same body always grades the same, no paid key is involved, and because the prose is never stored, a score must be right at capture. A future smarter scorer would ship as a new version label; old scores keep the label that produced them. Reviews captured before scoring shipped stay null forever (their prose was never stored), so the tab reads "no scored reviews yet" until new reviews arrive.

Honest nulls

A merged PR with no stored review never enters an average: it reads n/a, not zero, and only lowers the coverage percentage (reviewed share of merged PRs). Averages over an empty set read null, never 0. Before any review events exist the tab renders its honest zero state and points at the real prerequisite below.

Prerequisite: the review-event pull

Review events arrive shape-only through the GitHub App webhook on the hosted side; the local store fills once you add a prReviewsPull block (the hosted export address plus a device token) to the repo's entry in repos.config.json. Off by default; a half-configured block never fires. The window select offers 30, 90, or 180 days, with daily, weekly, 2-week, or monthly buckets.

Read more

Last updated