Shipping Pull Request Review Without Annoying Anyone

The first version of automated review posted every finding it had. On a two hundred line pull request that was forty comments, and the team it was built for turned it off in a week.

Silence is a feature

The version that shipped posts at most six comments and ranks them by how likely a human reviewer is to raise the same point. Everything below the line is still computed and still available, behind a link nobody clicks, which is exactly right.

Suggestion blocks, not opinions

Every comment carries a patch. A reviewer accepts it the way they accept any other suggestion, and a comment that cannot produce a patch is usually a comment that was not worth making.

The review never approves and never blocks. Its job is to spend a human reviewer’s attention on the parts that need it.