Check the code your agents wrote
Four checks run on the machine where the agent worked: a scan of the diff it wrote, joined to the session that wrote it; a registry check of every dependency it added; an npm audit of every package surface; and a gate that refuses a secret before it reaches a file. Each one says what it checked, so an empty result is never mistaken for a clean one.
Watch it
Read the narration
- 0:00 An agent opened a branch. It looks clean.
- 0:03 Nothing has checked it yet.
- 0:06 RepoOps runs three checks on the developer's machine.
- 0:09 Each result names what ran.
- 0:13 A package the agent invented reads hallucinated.
- 0:16 An unreachable registry reads unverified, and a gate that is off says nothing was checked.
- 0:23 Teams see metadata only.
- 0:25 The secret never leaves the machine.
Synthetic example. Read the guide
How to
The RepoOps desktop app tracking the repository, with origin/main fetched.
Semgrep on the machine for the full ruleset (pip install semgrep). Without it a small built-in pass runs and says so.
npm and a package-lock.json in each package surface you want audited.
- Open AI Security, then the Secrets tab, pick the repository under The code the agent wrote, and press Re-scan on Agent code scan.
Semgrep runs over the working tree against origin/main, untracked files included. Each finding shows the path and line, the severity, the rule, and the session line: the newest captured session in the last 30 days and that session's whole cost. The match is by recency, not a per-line trace, and the page says so.
- Read Dependency audit below it.
The first open starts npm audit in the background over the repository root and each directory with its own lockfile, and the card fills when it lands. Each folder lists its advisories by severity, with the vulnerable range and whether a fix exists.
- Read Packages added since origin/main on the same tab.
It lists each package the branch added against origin/main and asks the npm registry about it. A name the registry does not know reads Hallucinated; one published under 30 days ago with under 50 downloads last month reads Slopsquat candidate; a registry the machine cannot reach reads Unverified.
- Pick a gate and copy the CI step.
On Dependency audit, choose Info only, Block production high or critical (the default) or Block all high or critical. The card says Pass, Fail or Unverified (a folder that could not be audited is never counted as clean), then Copy command or Copy YAML for your CI. The choice is not saved and blocks nothing on its own.
- Arm the write-time secret gate.
On Security, then Guardrails, tick Check every write against the secret patterns and press Send a test candidate write. The hook then checks each Write, Edit, MultiEdit and NotebookEdit call over loopback and refuses a match. It is registered on PreToolUse in this repository's
.claude/settings.json; add the same entry to another repository to cover it. - Turn on the team rollup, if your team should see it.
On Privacy and egress, under What this machine publishes to your team, turn on Agent code findings. The desktop publishes hourly to your team. It is off by default and needs this device bound and cloud sync on. On repoops.ai, /team/agent-code-security opens the Secrets tab of AI Security.
Use cases
A package the agent invented
- Situation
- An agent added a dependency nobody recognises. npm audit is clean, because audit only checks packages that exist.
- You do
- Open AI Security, then the Secrets tab, pick the repository, and read Packages added since origin/main.
- You see
- The package reads Hallucinated with the reason: the registry answered 404 for that name. Remove it before anyone installs a lookalike someone registers later.
A key the agent tried to write
- Situation
- An agent writes a config file with a live API key pasted into it, and the write-time secret gate is armed.
- You do
- Open AI Security, then the Secrets tab, and read the log under the secret gate.
- You see
- The write was refused before it reached disk. The event names the file and what kind of secret matched, with a redacted preview, never the key itself.
Expected outcome
Next step: Run pip install semgrep, then Re-scan. The built-in pass is a handful of rules, not the ruleset.
Next step: Semgrep read nothing, usually because the ruleset fetch failed. Set REPOOPS_SEMGREP_CONFIG in the data directory's .env to a local rule file and Re-scan.
Next step: Fetch the remote. Until origin/main resolves, nothing is claimed about added dependencies.
Next step: It has no lockfile, npm timed out after 90 seconds, or the registry was unreachable. Run npm install there, then Re-scan; the other surfaces still report.
Next step: No session was captured for this repository in 30 days. The finding is still real; RepoOps will not name a session it did not see.
What else to consider
Strict mode for the gate
A write the gate cannot check, for example while the dashboard is not running, is allowed with a warning. Refuse a write the gate could not check (REPOOPS_SECRET_GATE_STRICT=1) refuses it instead, which refuses every write while the dashboard is down.
The nightly whole-repo pass
REPOOPS_REPO_SCAN_TICK=1 runs Semgrep over everything git tracks at 02:00, once per commit, and reports under Whole repo, nightly. It is off by default because it costs minutes of CPU, and it refuses to run without Semgrep.
Egress
Semgrep's default config fetches its ruleset from semgrep.dev and sends Semgrep's own usage metrics; a local REPOOPS_SEMGREP_CONFIG stops that. npm audit and the provenance check call the npm registry. The gate posts candidate writes to 127.0.0.1 only. The hourly team snapshot, when turned on, carries severity, rule or package, verdict, counts and a machine hash; never the source line, the secret or the redacted preview.
Cost and plans
No model call and nothing metered. The local checks have no plan gate; publishing the rollup needs a paid team on the hosted plan or above.