Design: Octopus approves a clean review
Context
Section titled “Context”See openspec/changes/octopus-approve-clean-reviews/proposal.md for the problem and for the
decisions that Sean made on 2026-10-13.
Octopus is a fork (seanb4t/octopus) of octopusreview/octopus. The cluster runs the branch
upgrade/v1.2.14-analysis, which stacks on upgrade/v1.2.14-infra, on the upstream tag
v1.2.14. The fork master tracks upstream master with no extra commit. A patch for upstream
starts on a branch up/<topic> off upstream master, as the three merged pull requests
(#877, #878 and #879) did. These facts of the upstream code (master 580a930, release 1.2.16)
shape the design:
- One function decides the merge gate:
shouldFailReviewCheck()inapps/web/lib/review-helpers.ts. Its comment names it the single source of truth for the check run, the GitLab commit status and the review event. Two review paths read it:apps/web/lib/reviewer.ts(the standard path) andapps/web/lib/large-review-result.ts(the large-PR path). Each maps the result toREQUEST_CHANGESorCOMMENT. createPullRequestReview()inapps/web/lib/github.tsand theProviderClientinterface inapps/web/lib/provider.tsalready accept the eventAPPROVE. No caller passes it.reviewCheckResult()inapps/web/lib/review-coverage.tsis pure. It returnsfailurewhen the assessment is not complete,successwith the statenot-requiredwhen the repository policy excludes every changed file, and otherwise the gate result.- The standard path lists the earlier review comments of the pull request before it builds the
prompt. The type
PRReviewCommentcarriesid,user,inReplyToIdandisOutdated. The bot login is<app slug>[bot]. A re-review is the stateisReReview. - The re-review prompt tells the model that an empty findings list is the expected result and
forbids a repeat of an earlier finding. So zero findings on a re-review means nothing new, not
everything fixed. The only resolution signals are
isOutdatedon an earlier inline comment and a reply in its thread. - The organization setting
checkFailureThresholdshows the pattern for a setting: a column inpackages/db/prisma/schema.prisma, a migration underpackages/db/prisma/migrations/, a server action inapps/web/app/(app)/actions.tsbehind the permissionreviews:configure, and a card underapps/web/app/(app)/settings/reviews/. - The open-source bot-account path (
apps/web/lib/oss-review-post.ts) posts with a user token and hard-codesCOMMENT. - The GitHub App
octopus-fzymgc(installation 138713975) haspull_requests: write. GitHub needs no more for anAPPROVEreview. A dismissal on a protected branch needs an administrator or an allow-list, which is why the design does not dismiss. - The ruleset
main-branch-protectionof this repository requires one approving review and dismisses a stale review on push. Terraform manages it intf/core-services/github.tf.
This change adds no address and no prefix, so docs/reference/network.md does not change.
Goals / Non-Goals
Section titled “Goals / Non-Goals”Goals:
- One pure function decides whether a review is an approval, and both review paths call it.
- A change that upstream can merge: an organization setting, a UI card, tests, and no behaviour change for an organization that does not opt in.
- A fork that carries the change as one cherry-picked commit set until upstream releases it.
Non-Goals:
- Approval on GitLab, Forgejo or Bitbucket. Each has its own approval API, and upstream added GitLab parity for the merge gate in a later pull request (#632). The same split applies here.
- A per-repository override. The organization setting matches the scope of
checkFailureThreshold. - A dismissal of the earlier
CHANGES_REQUESTEDreview. An approval by the same login replaces it inreviewDecision. - A change to the re-review prompt or to the resolution signals.
Decisions
Section titled “Decisions”D-1. One pure helper decides the approval, next to the merge gate
Section titled “D-1. One pure helper decides the approval, next to the merge gate”apps/web/lib/review-helpers.ts gains shouldApproveReview(), directly under
shouldFailReviewCheck():
export function shouldApproveReview(input: { enabled: boolean; findings: number; // after the confidence filter and the cap coverageComplete: boolean; // reviewAssessmentComplete(coverage) and state !== "not-required" priorBotComments: { isOutdated: boolean; hasReply: boolean }[];}): boolean { return input.enabled && input.coverageComplete && input.findings === 0 && input.priorBotComments.every((c) => c.isOutdated || c.hasReply);}Each review path computes the event once:
const reviewEvent = approve ? "APPROVE" : shouldRequestChanges ? "REQUEST_CHANGES" : "COMMENT";findings is the count after the filters (findings.length), not the count of comments that
reach GitHub. The dedup against earlier inline comments can bring the visible count to zero while
findings exist. A fresh pull request has no earlier comment, so the last condition holds. The gate
reads the earlier comments on every review, not only on a re-review, so a comment that an earlier
partial attempt left also has to be outdated or replied to.
Alternative: a second branch inside shouldFailReviewCheck(). Rejected, because that function
returns one boolean that three outputs share, and an approval is a different question from a
failed check.
D-2. The resolution signal of an earlier inline comment
Section titled “D-2. The resolution signal of an earlier inline comment”hasReply is true when any comment in the same list has inReplyToId equal to the earlier
comment’s id and a user that is a person: not the bot login, and not another [bot] account.
isOutdated comes from GitHub. One pure helper, priorBotCommentSignals(), maps the full comment
list to the signals, and both review paths call it. If the list call fails, or the App slug is
unknown, the earlier comments are unknown, and an unknown list never approves. For that, the list
call listPullRequestReviewComments() pages through every comment and throws on a failed
response. Before, it read one page of 100 and returned an empty list on a failure.
A summary-table finding has no comment and no signal, so it does not enter the list. Sean decided this on 2026-10-13, because a pull request with one low-severity table entry would otherwise never get an approval.
D-3. Coverage gate
Section titled “D-3. Coverage gate”coverageComplete is reviewAssessmentComplete(coverage) and the assessment state is not
not-required. The first part mirrors the check run: an incomplete review never approves. The
second part keeps a pull request whose files the repository policy all excludes at COMMENT,
because no review ran. reviewCheckResult() reports success for that state, so the check run
still passes.
D-4. The organization setting
Section titled “D-4. The organization setting”packages/db/prisma/schema.prisma, model Organization, directly under checkFailureThreshold:
approveCleanReviews Boolean @default(false)Migration packages/db/prisma/migrations/<timestamp>_add_approve_clean_reviews/migration.sql:
ALTER TABLE "organizations" ADD COLUMN "approveCleanReviews" BOOLEAN NOT NULL DEFAULT false;The server action toggleApproveCleanReviews in apps/web/app/(app)/actions.ts follows
toggleReviewsPaused: the current organization from the cookie, the permission
reviews:configure, one prisma.organization.update, revalidatePath("/settings/reviews").
The card approve-clean-reviews-switch.tsx under apps/web/app/(app)/settings/reviews/ follows
reviews-paused-switch.tsx. It sits under the threshold card on the page. Its description states
the two conditions of D-1 in one sentence each, and this warning: the approval counts toward the
required approving reviews of a branch rule, and an author who replies to every finding gets an
approval.
Alternative: a fourth radio option in the threshold card. Rejected, because the threshold answers “when does the check fail”, and the approval is a separate question with its own warning.
Alternative: an environment variable. Sean rejected it on 2026-10-13, because the hosted product serves many organizations from one process.
D-5. The large-PR path
Section titled “D-5. The large-PR path”large-review-result.ts has no list of earlier comments. It gains the same list call that the
standard path makes (listPullRequestReviewComments) when the setting is on, and it applies D-1
and D-2 to the result. If the list call fails, priorBotComments is unknown, and the path posts
COMMENT. The path has no isReReview flag, so the list is the only signal. An empty list on a
first review approves. This keeps the two paths in step without a shared state.
D-6. The approval review
Section titled “D-6. The approval review”The approval is the same review call that posts the summary today, with the event APPROVE and
the commit_id of the head commit, which the call already passes. The body keeps the summary
line, so the approval states the coverage and 0 findings. An inline comment never accompanies
an approval, because an approval needs zero findings.
D-7. Tests
Section titled “D-7. Tests”- Unit tests for
shouldApproveReview()inapps/web/lib/__tests__/review-helpers.test.ts, one case per scenario of the spec: off, zero findings, one finding, incomplete coverage, every earlier comment outdated, every earlier comment replied, one earlier comment open. - One assertion in
apps/web/lib/__tests__/fixtures/review-follow-up-harness.ts: the mockedcreatePullRequestReviewrecords the event, and a run with the setting on and the fixture’s clean response postsAPPROVE. The harness already recordspublishedandcheckConclusions.
D-8. The fork branches and the upstream pull request
Section titled “D-8. The fork branches and the upstream pull request”The work starts on up/approve-clean-reviews off upstream/master. The pull request to
octopusreview/octopus follows .github/pull_request_template.md and CONTRIBUTING.md:
conventional commit feat(review): opt-in APPROVE review when a review finds nothing, lint, type
check, tests, build. The description names the author-reply risk and the counted-approval warning.
The same commits go onto upgrade/v1.2.14-infra with git cherry-pick -x, and from there onto
upgrade/v1.2.14-analysis. Both pushes build images. Upstream master moved past v1.2.14
in reviewer.ts, so the cherry-pick can need a conflict resolution around the event line. The
runbook rule applies: read the combined diff and apply it to the branch’s code.
D-9. The ruleset field
Section titled “D-9. The ruleset field”tf/core-services/github.tf, resource github_repository_ruleset.main_branch_protection, block
pull_request: require_last_push_approval = true. The provider integrations/github 6.13 sends
the ruleset as a full replace through go-github v88. The wire struct
PullRequestRuleParameters (github/rules.go) models the field as RequireLastPushApproval
with the JSON name require_last_push_approval. The live ruleset also carries
require_extra_approval_for_unattributed_changes: true, which that struct does not model, so the
apply can reset it. The task list re-reads it after the apply.
Octopus never pushes to a pull request branch, so its approval meets the rule. Sean pushes every commit, so his own approval would not.
Risks / Trade-offs
Section titled “Risks / Trade-offs”- [GitHub does not count an App approval toward the required count] → The acceptance of pull
request 2 proves it before pull request 3 changes the ruleset. If it fails, the ruleset keeps
require_last_push_approval = false, and the change still replaces the staleCHANGES_REQUESTEDverdict. - [
isOutdatedis loose: any change to the line counts as a fix] → Accepted. The check run has the same property today, and the re-review still runs the model on the new diff. - [An author replies to every finding and gets an approval] → Accepted for one developer, stated in the upstream pull request and in the card.
- [Upstream rejects or reshapes the change] → The fork carries it as a patch row in
docs/operations/octopus.md. The change is additive, so a later upstream version drops in. - [The cherry-pick onto
v1.2.14conflicts] → D-8 names the resolution rule. - [The Terraform apply resets
require_extra_approval_for_unattributed_changes] → The task list re-reads the field and names the fix (set it in the GitHub UI). - [A stale approval survives a push] →
dismiss_stale_reviews_on_pushis on, so GitHub dismisses it. The re-review approves again or leaves a comment.
Migration Plan
Section titled “Migration Plan”-
Fork: push
up/approve-clean-reviews, open the upstream pull request. No live change. -
Fork: cherry-pick onto
upgrade/v1.2.14-infraandupgrade/v1.2.14-analysis, push both. Wait forbuild-imageon the analysis commit. Rollback: none, the cluster still pins the old digests. -
Cluster: pin the web and migrate digests of the analysis commit in
argocd/app-configs/octopus/deployment.yaml. The migrate initContainer adds the column at the first pod start. Rollback: a pull request that restores the digestssha256:f717e9f575446cd6686a4b991ea7ce86da5677a9b6eb7b80309088088f37e331(web) andsha256:91b7708229102f97a6de8b75b4bbe367baf1fc5ac9ae9eac3c47dbf359a33d45(migrate). The column stays, with no reader, and harms nothing. -
Operator: turn the setting on under Settings > Reviews of the fzymgc-house organization. Rollback: turn it off in the same card, or on the Postgres primary:
Terminal window kubectl -n postgres exec "$(kubectl -n postgres get cluster main -o jsonpath='{.status.currentPrimary}')" \-c postgres -- psql -d octopus -c 'UPDATE organizations SET "approveCleanReviews" = false;' -
Operator: run the acceptance of the proposal on a test pull request. An approval that must go away is dismissed by an administrator:
Terminal window gh api -X PUT repos/fzymgc-house/selfhosted-cluster/pulls/<number>/reviews/<id>/dismissals \-f message='Dismissed during the acceptance of octopus-approve-clean-reviews' -
Cluster:
require_last_push_approval = true. Rollback: set it back tofalseand apply.