CommitGuard
A policy engine that blocks AI-agent attribution in Git commit metadata — in local hooks, a GitHub Action and a webhook-driven GitHub App — measured against 9,174 labelled cases with every failing run kept.
- 9,174
- Labelled cases, none misjudged
- 4 of 15
- Benchmark runs that failed, kept
- 1,452
- Python tests
- 15
- AI agents in rule data
- Role
- Author
- Year
- 2026
- Status
- Open source
- Type
- Git commit provenance and policy engine
01Overview
CommitGuard reads what a commit says about its own origin — author, committer, message and trailers — and decides, according to a repository's policy, whether that commit is acceptable. The first policy is AI-agent attribution: a commit carrying an AI agent as co-author is blocked by default; one crediting a human co-author is not.
I built it alone over six days in September 2026 — 132 commits — and published it to PyPI as commitguardian and to the GitHub Marketplace as a composite Action. It is pre-alpha. The same engine runs in three places: Git hooks on a developer's machine, a GitHub Actions check, and a GitHub App with a web dashboard for organisation-wide policy.
The interesting part is that commit messages are attacker-controlled text. A check that matches strings is easy to evade with casing, look-alike letters or characters that render as nothing, so most of the work went into parsing hostile input and into measuring, in public, how often the parser got it wrong.
02The problem
AI coding agents increasingly write commits, and many record themselves in commit metadata. Some projects need to control that for contributor-agreement or licensing reasons, or simply to keep an accurate provenance record. Checking by hand does not scale.
Naive string matching fails in both directions. It is easy to evade — a Cyrillic letter in an agent's name, a variation selector inside the trailer key, a symbol before "Co-authored-by" that hides the line from a parser but not from a reader. It is also prone to false positives: a human who shares an agent's name, or an employee of the vendor, is not an AI agent.
And a local check is only advice. Anyone can pass --no-verify. A real boundary has to run server-side, and it has to be impossible for the pull request being judged to change the rules judging it.
03Architecture
- 01
Detection separate from policy
Four detectors — coauthor, identity, trailer and bot — report findings with rule, severity, confidence and evidence, and know nothing about configuration. A separate policy evaluator turns findings into allow, warn or block, with block > warn > allow regardless of detector order. Neither side has side effects, which is what lets detection accuracy be measured independently of any policy choice.
- 02
Rules as data
Four YAML files hold the rules: 15 AI agents, 11 vendor domains and 6 bots, plus trailer patterns. They ship inside the installed package and are never read from the repository being scanned. Matching is exact after normalisation — case, width and whitespace folded, every default-ignorable code point removed, Cyrillic and Greek look-alikes mapped to Latin. There is no substring matching, and a vendor domain on its own is never enough.
- 03
A trailer parser with no regular expressions
The trailer parser is an explicit linear scan with bounded key length, trailer count and prefix skipping. It never raises; instead it records named flags such as MISSING_SEPARATOR and LEADING_CHARACTERS that detectors can act on. Work stays linear in message size, and each bypass fix was a readable change to the scan rather than a new branch in an expression.
- 04
Three enforcement layers
Local hooks — pre-commit, commit-msg and pre-push — give immediate feedback and chain any existing hooks rather than replacing them. The composite Action scans every commit a pull request introduces, plus merge-queue entries and pushes, with SHA-pinned actions, hash-pinned dependencies and only contents: read. The GitHub App does the same from signed webhooks and publishes a Check Run. All three reach the same analyser.
- 05
GitHub App on the standard library
The App service is a WSGI server from Python's standard library over SQLite — 47 tables — with cryptography as its only extra dependency, for signing the App JWT. Webhooks are signature-checked and deduplicated by delivery ID. Installation tokens are scoped down to one repository per scan, and repositories are fetched as metadata-only mirrors, so repository code is never checked out or run.
- 06
Dashboard and control plane
React 19 and TypeScript over a /api/v1 served by the same process: 32 pages covering scans, violations, versioned organisation policy with audited rollback, scoped expiring exceptions, staged rollouts and policy simulation against recorded scans. Four roles — viewer, security manager, admin and owner — are granted in CommitGuard, and users see only repositories GitHub already lets them see.
- 07
Reproducible evidence
Four versioned datasets of labelled commits, the largest with 9,174 cases, are rebuilt and fingerprinted by one command, commitguard reproduce. Every benchmark result is written once to a timestamped file and never overwritten.
04Key decisions
Read policy from the base commit
If the check reads .commitguard.yaml from the pull request it is checking, the first commit of any pull request can be "disable the check". Server-side runs read policy from the base commit, and the workflow installs CommitGuard itself from that trusted commit. A pull request that tightens policy does not take effect until merged — that is the cost, and I accept it.
Every failure path blocks
A detector that raises, a finding with no matching policy, invalid configuration, a Git error — each becomes a block or an error, never an allow. The App publishes error, never success, and the dashboard shows UNKNOWN or STALE rather than a green tick. The only way to pass is a completed evaluation that found nothing.
Keep the failing runs
15 detection runs are committed, and four of them failed: one false negative, then two false positives introduced by the fix, then 11 and 15 misses found by property-based fuzzing. Each failing run was recorded before the fix. The final zero is only worth something because the runs that were not zero are sitting next to it.
Publish as commitguardian
commitguard and commitguard-cli on PyPI already belong to two unrelated projects. At one point the documentation told people to pip install commitguard, which would have installed someone else's code — a dependency-confusion problem of my own making. The distribution is commitguardian; the command is still commitguard.
05Stack
- Python 3.12
- Typer
- Pydantic
- PyYAML
- SQLite
- GitHub Apps
- GitHub Actions
- React 19
- TypeScript
- Vite
- TanStack Query
- pytest
- Hypothesis
- Playwright