Skip to content

Design: Octopus approves a clean review

See openspec/changes/octopus-approve-clean-reviews/proposal.md for the problem and for the decisions that Sean made on 2026-10-13.

Octopus is a fork (seanb4t/octopus) of octopusreview/octopus. The cluster runs the branch upgrade/v1.2.14-analysis, which stacks on upgrade/v1.2.14-infra, on the upstream tag v1.2.14. The fork master tracks upstream master with no extra commit. A patch for upstream starts on a branch up/<topic> off upstream master, as the three merged pull requests (#877, #878 and #879) did. These facts of the upstream code (master 580a930, release 1.2.16) shape the design:

  • One function decides the merge gate: shouldFailReviewCheck() in apps/web/lib/review-helpers.ts. Its comment names it the single source of truth for the check run, the GitLab commit status and the review event. Two review paths read it: apps/web/lib/reviewer.ts (the standard path) and apps/web/lib/large-review-result.ts (the large-PR path). Each maps the result to REQUEST_CHANGES or COMMENT.
  • createPullRequestReview() in apps/web/lib/github.ts and the ProviderClient interface in apps/web/lib/provider.ts already accept the event APPROVE. No caller passes it.
  • reviewCheckResult() in apps/web/lib/review-coverage.ts is pure. It returns failure when the assessment is not complete, success with the state not-required when the repository policy excludes every changed file, and otherwise the gate result.
  • The standard path lists the earlier review comments of the pull request before it builds the prompt. The type PRReviewComment carries id, user, inReplyToId and isOutdated. The bot login is <app slug>[bot]. A re-review is the state isReReview.
  • The re-review prompt tells the model that an empty findings list is the expected result and forbids a repeat of an earlier finding. So zero findings on a re-review means nothing new, not everything fixed. The only resolution signals are isOutdated on an earlier inline comment and a reply in its thread.
  • The organization setting checkFailureThreshold shows the pattern for a setting: a column in packages/db/prisma/schema.prisma, a migration under packages/db/prisma/migrations/, a server action in apps/web/app/(app)/actions.ts behind the permission reviews:configure, and a card under apps/web/app/(app)/settings/reviews/.
  • The open-source bot-account path (apps/web/lib/oss-review-post.ts) posts with a user token and hard-codes COMMENT.
  • The GitHub App octopus-fzymgc (installation 138713975) has pull_requests: write. GitHub needs no more for an APPROVE review. A dismissal on a protected branch needs an administrator or an allow-list, which is why the design does not dismiss.
  • The ruleset main-branch-protection of this repository requires one approving review and dismisses a stale review on push. Terraform manages it in tf/core-services/github.tf.

This change adds no address and no prefix, so docs/reference/network.md does not change.

Goals:

  • One pure function decides whether a review is an approval, and both review paths call it.
  • A change that upstream can merge: an organization setting, a UI card, tests, and no behaviour change for an organization that does not opt in.
  • A fork that carries the change as one cherry-picked commit set until upstream releases it.

Non-Goals:

  • Approval on GitLab, Forgejo or Bitbucket. Each has its own approval API, and upstream added GitLab parity for the merge gate in a later pull request (#632). The same split applies here.
  • A per-repository override. The organization setting matches the scope of checkFailureThreshold.
  • A dismissal of the earlier CHANGES_REQUESTED review. An approval by the same login replaces it in reviewDecision.
  • A change to the re-review prompt or to the resolution signals.

D-1. One pure helper decides the approval, next to the merge gate

Section titled “D-1. One pure helper decides the approval, next to the merge gate”

apps/web/lib/review-helpers.ts gains shouldApproveReview(), directly under shouldFailReviewCheck():

export function shouldApproveReview(input: {
enabled: boolean;
findings: number; // after the confidence filter and the cap
coverageComplete: boolean; // reviewAssessmentComplete(coverage) and state !== "not-required"
priorBotComments: { isOutdated: boolean; hasReply: boolean }[];
}): boolean {
return input.enabled && input.coverageComplete && input.findings === 0
&& input.priorBotComments.every((c) => c.isOutdated || c.hasReply);
}

Each review path computes the event once:

const reviewEvent = approve ? "APPROVE" : shouldRequestChanges ? "REQUEST_CHANGES" : "COMMENT";

findings is the count after the filters (findings.length), not the count of comments that reach GitHub. The dedup against earlier inline comments can bring the visible count to zero while findings exist. A fresh pull request has no earlier comment, so the last condition holds. The gate reads the earlier comments on every review, not only on a re-review, so a comment that an earlier partial attempt left also has to be outdated or replied to.

Alternative: a second branch inside shouldFailReviewCheck(). Rejected, because that function returns one boolean that three outputs share, and an approval is a different question from a failed check.

D-2. The resolution signal of an earlier inline comment

Section titled “D-2. The resolution signal of an earlier inline comment”

hasReply is true when any comment in the same list has inReplyToId equal to the earlier comment’s id and a user that is a person: not the bot login, and not another [bot] account. isOutdated comes from GitHub. One pure helper, priorBotCommentSignals(), maps the full comment list to the signals, and both review paths call it. If the list call fails, or the App slug is unknown, the earlier comments are unknown, and an unknown list never approves. For that, the list call listPullRequestReviewComments() pages through every comment and throws on a failed response. Before, it read one page of 100 and returned an empty list on a failure.

A summary-table finding has no comment and no signal, so it does not enter the list. Sean decided this on 2026-10-13, because a pull request with one low-severity table entry would otherwise never get an approval.

coverageComplete is reviewAssessmentComplete(coverage) and the assessment state is not not-required. The first part mirrors the check run: an incomplete review never approves. The second part keeps a pull request whose files the repository policy all excludes at COMMENT, because no review ran. reviewCheckResult() reports success for that state, so the check run still passes.

packages/db/prisma/schema.prisma, model Organization, directly under checkFailureThreshold:

approveCleanReviews Boolean @default(false)

Migration packages/db/prisma/migrations/<timestamp>_add_approve_clean_reviews/migration.sql:

ALTER TABLE "organizations" ADD COLUMN "approveCleanReviews" BOOLEAN NOT NULL DEFAULT false;

The server action toggleApproveCleanReviews in apps/web/app/(app)/actions.ts follows toggleReviewsPaused: the current organization from the cookie, the permission reviews:configure, one prisma.organization.update, revalidatePath("/settings/reviews").

The card approve-clean-reviews-switch.tsx under apps/web/app/(app)/settings/reviews/ follows reviews-paused-switch.tsx. It sits under the threshold card on the page. Its description states the two conditions of D-1 in one sentence each, and this warning: the approval counts toward the required approving reviews of a branch rule, and an author who replies to every finding gets an approval.

Alternative: a fourth radio option in the threshold card. Rejected, because the threshold answers “when does the check fail”, and the approval is a separate question with its own warning.

Alternative: an environment variable. Sean rejected it on 2026-10-13, because the hosted product serves many organizations from one process.

large-review-result.ts has no list of earlier comments. It gains the same list call that the standard path makes (listPullRequestReviewComments) when the setting is on, and it applies D-1 and D-2 to the result. If the list call fails, priorBotComments is unknown, and the path posts COMMENT. The path has no isReReview flag, so the list is the only signal. An empty list on a first review approves. This keeps the two paths in step without a shared state.

The approval is the same review call that posts the summary today, with the event APPROVE and the commit_id of the head commit, which the call already passes. The body keeps the summary line, so the approval states the coverage and 0 findings. An inline comment never accompanies an approval, because an approval needs zero findings.

  • Unit tests for shouldApproveReview() in apps/web/lib/__tests__/review-helpers.test.ts, one case per scenario of the spec: off, zero findings, one finding, incomplete coverage, every earlier comment outdated, every earlier comment replied, one earlier comment open.
  • One assertion in apps/web/lib/__tests__/fixtures/review-follow-up-harness.ts: the mocked createPullRequestReview records the event, and a run with the setting on and the fixture’s clean response posts APPROVE. The harness already records published and checkConclusions.

D-8. The fork branches and the upstream pull request

Section titled “D-8. The fork branches and the upstream pull request”

The work starts on up/approve-clean-reviews off upstream/master. The pull request to octopusreview/octopus follows .github/pull_request_template.md and CONTRIBUTING.md: conventional commit feat(review): opt-in APPROVE review when a review finds nothing, lint, type check, tests, build. The description names the author-reply risk and the counted-approval warning.

The same commits go onto upgrade/v1.2.14-infra with git cherry-pick -x, and from there onto upgrade/v1.2.14-analysis. Both pushes build images. Upstream master moved past v1.2.14 in reviewer.ts, so the cherry-pick can need a conflict resolution around the event line. The runbook rule applies: read the combined diff and apply it to the branch’s code.

tf/core-services/github.tf, resource github_repository_ruleset.main_branch_protection, block pull_request: require_last_push_approval = true. The provider integrations/github 6.13 sends the ruleset as a full replace through go-github v88. The wire struct PullRequestRuleParameters (github/rules.go) models the field as RequireLastPushApproval with the JSON name require_last_push_approval. The live ruleset also carries require_extra_approval_for_unattributed_changes: true, which that struct does not model, so the apply can reset it. The task list re-reads it after the apply.

Octopus never pushes to a pull request branch, so its approval meets the rule. Sean pushes every commit, so his own approval would not.

  • [GitHub does not count an App approval toward the required count] → The acceptance of pull request 2 proves it before pull request 3 changes the ruleset. If it fails, the ruleset keeps require_last_push_approval = false, and the change still replaces the stale CHANGES_REQUESTED verdict.
  • [isOutdated is loose: any change to the line counts as a fix] → Accepted. The check run has the same property today, and the re-review still runs the model on the new diff.
  • [An author replies to every finding and gets an approval] → Accepted for one developer, stated in the upstream pull request and in the card.
  • [Upstream rejects or reshapes the change] → The fork carries it as a patch row in docs/operations/octopus.md. The change is additive, so a later upstream version drops in.
  • [The cherry-pick onto v1.2.14 conflicts] → D-8 names the resolution rule.
  • [The Terraform apply resets require_extra_approval_for_unattributed_changes] → The task list re-reads the field and names the fix (set it in the GitHub UI).
  • [A stale approval survives a push] → dismiss_stale_reviews_on_push is on, so GitHub dismisses it. The re-review approves again or leaves a comment.
  1. Fork: push up/approve-clean-reviews, open the upstream pull request. No live change.

  2. Fork: cherry-pick onto upgrade/v1.2.14-infra and upgrade/v1.2.14-analysis, push both. Wait for build-image on the analysis commit. Rollback: none, the cluster still pins the old digests.

  3. Cluster: pin the web and migrate digests of the analysis commit in argocd/app-configs/octopus/deployment.yaml. The migrate initContainer adds the column at the first pod start. Rollback: a pull request that restores the digests sha256:f717e9f575446cd6686a4b991ea7ce86da5677a9b6eb7b80309088088f37e331 (web) and sha256:91b7708229102f97a6de8b75b4bbe367baf1fc5ac9ae9eac3c47dbf359a33d45 (migrate). The column stays, with no reader, and harms nothing.

  4. Operator: turn the setting on under Settings > Reviews of the fzymgc-house organization. Rollback: turn it off in the same card, or on the Postgres primary:

    Terminal window
    kubectl -n postgres exec "$(kubectl -n postgres get cluster main -o jsonpath='{.status.currentPrimary}')" \
    -c postgres -- psql -d octopus -c 'UPDATE organizations SET "approveCleanReviews" = false;'
  5. Operator: run the acceptance of the proposal on a test pull request. An approval that must go away is dismissed by an administrator:

    Terminal window
    gh api -X PUT repos/fzymgc-house/selfhosted-cluster/pulls/<number>/reviews/<id>/dismissals \
    -f message='Dismissed during the acceptance of octopus-approve-clean-reviews'
  6. Cluster: require_last_push_approval = true. Rollback: set it back to false and apply.