[Code Review] Code Review in the AI Era — Who Reviews the Code After AI Writes It? | Software Engineering Transformation in the AI Age — Learn AI Slowly #174
Code Review in the Age of AI — Who Reviews When the Code Is Written by AI?
In the previous piece (AI173), I named “verification” as the third new bottleneck once code becomes nearly free, and closed with a line saying “I’ll cover it in its own section.” This piece delivers on that promise. Here is the conclusion upfront: looking back from mid-2026, the variable that AI coding tools actually delivers is neither license counts, nor seat counts, nor model benchmarks — it is review bandwidth.
AI173
Learn AI Slowly 197: Why “AI-Generated Code Is Full of Bugs” — and What the Data Actually Shows
CodeRabbit’s late-2025 report analyzed 470 open-source GitHub PRs and found that AI-assisted code ships with 1.7× more defects than human-only code (10.83 vs. 6.45 issues per PR, unpaired by file size or complexity), with security vulnerabilities running 1.57× to 2.74× higher by subcategory — XSS at 2.74×, improper cryptographic handling at 1.88×, insecure direct object references (IDOR) at 1.91×, and insecure deserialization at 1.82×. Logic/correctness issues ran 1.75× higher, readability defects 3×+, formatting 2.66×, and error handling nearly 2×.
Apiiro’s September 2025 scan of Fortune 50 enterprise repositories (covering December 2024 through June 2025) added the second half of the picture: AI-generated code drove monthly security findings from roughly 1,000 to over 10,000 — a 10× jump — with privilege-escalation vulnerabilities up 322% in absolute count (≈60–80% once normalized for code-volume growth) and architectural design flaws up 153%. Over the same period, syntax errors dropped 76% and logic bugs fell 60%.
The picture painted by these two data sets together matters most in a regulatory context: a significant share of those 322% privilege-escalation findings from Apiiro sit on authorization boundaries—and in financial services and telecom, those boundaries protect customer money and customer data. A lot of AI-generated code runs, but defects and vulnerabilities are rising proportionally, and the dangerous ones are creeping up quietly. (Methodology note: CodeRabbit is a vendor report; Apiiro’s data comes from an independent security vendor. The conclusions point the same direction, but the metrics need to be read with normalization differences in mind.)
What this means inside the enterprise triggers two counter-intuitive realities, each one the inverse of the story your tooling vendor is selling.
Two counter-intuitive truths
Counter-intuitive #1: The developer’s role has shifted from “writer of code” to “reviewer of code”—but reviewing is more exhausting than writing.
Takeaway: After AI widens the “writing” bottleneck, developers spend more time on “reading + evaluating”—reading unfamiliar code, judging compliance boundaries, checking business rules. The cognitive load is markedly heavier than writing your own code. 56% of senior engineers now rely on AI for over 70% of their work (Pragmatic, Feb 2026). The new way of working has become the default.
Counter-intuitive insight 2: The stronger AI tools get, the less organizations need more tools — and the more they need governance.
Learn AI Slowly 173 — The Audit Bandwidth Bottleneck: When AI Coding Moves the Constraint, Not Removes It
A 1.7× defect rate from CodeRabbit, a 322% jump in privilege-escalation findings from Apiiro. Read in isolation, these look like AI failures. Refracted through the lens of the Theory of Constraints (TOC), they become something more uncomfortable: the predictable result of widening a tool’s output capacity while leaving human review capacity exactly where it was. A system’s throughput is dictated by its narrowest stage. AI widened the “write” stage; the narrowest stage is now “review.” And because review bandwidth can’t scale linearly with generation speed, the faster the model writes, the more dangerous the debt the organization quietly accumulates. That is the diagnosis this edition offers: automation does not eliminate bottlenecks; it relocates them.
Applied to AI-assisted coding, this requires an important refinement. Software development is not a single pipeline with one bottleneck; it is a set of parallel bottlenecks that drift dynamically. TOC holds cleanly for serial production lines. In a parallel, multi-bottleneck domain like AI coding, the narrowest point has indeed drifted from “write” to “review,” but “review” itself fractures into three independent constraints, each capable of stalling the system on its own: validation, governance, and compliance review.
Learn AI Slowly 322: Safety Checkpoints Before Deploying Autonomous Agents
This principle has two layers of practical meaning. The first layer is that before you unleash autonomous agents, you must install four brakes at the organizational level — mandatory human code review, automated testing (anything AI touches has to still build and pass), security scanning at the same bar you apply to human-written code, and canary releases (roll AI-generated changes out to a small slice of traffic first). No AI pull request gets a free pass. This is the bare minimum that turns the question from “can AI write code” into the real engineering problem: “can AI write code and can the organization catch what it gets wrong.” Skip any one of those four and you have an exposed failure surface.
A frequently cited data point comes from Nicholas Carlini’s journal entries in January–February 2026: an Anthropic researcher ran 16 Claude Opus 4.6 agents in parallel for about two weeks, racking up roughly 2,000 sessions and around $20,000 in API spend. The agents wrote a 100,000-line Rust-based C compiler from scratch, capable of compiling the Linux 6.9 kernel and passing 99% of the GCC torture test suite.
Worth pausing on: that was a controlled experiment in a closed domain. Carlini never shipped the code to production. It is a valid “extreme no-review counterexample” — proof that with zero oversight the blast radius can still be finite — but reading it as a template for “go deploy autonomous agents today” will overestimate how well the result transfers. Drop that same workload into an organization with no code review, no automated tests, no security scanning, and no canary releases, and a bad day is a question of when, not if.
The second layer is subtler: the point of review isn’t to find bugs but to judge architectural alignment, compliance boundaries, and business correctness
The most common trap for senior engineers from earlier generations is equating review in the AI era with traditional code review. Traditional review asks, “Is this code wrong?” AI-era review asks, “Should this code exist in this file, in this project, within these compliance boundaries at all?” CodeRabbit’s 1.82–2.74× safer-vulnerability finding and Apiiro’s 322% privilege-escalation finding point to exactly this class of problem: the AI didn’t write wrong code, it wrote code in the wrong place, with the wrong permissions, behind the wrong defaults. These issues can’t be fixed in the IDE; they have to be understood at the review table.
The more widely accepted engineering practice is to mark GitHub/GitLab branch protection and CODEOWNERS rules in red for anything touching schema, auth, billing, or compliance boundaries, and route those changes to dual sign-off. In financial-services and telecom operations, this usually takes the form of a backup veto with spot-check ratios calibrated to risk tier, rather than a full review. Architecture Decision Records (ADRs), security and compliance baselines, and business-rule correctness — that’s where AI-era review time should actually go.
Stacking these two counter-intuitive points together, the picture becomes clear: code review in the AI era requires a company to adjust three things — pull engineering leadership into the review loop, encode compliance and architectural baselines into PR routing, and escalate governance metrics like failure rates to the board. These three map directly onto the “three lines of defense” in model governance (business, IT, compliance/audit) that regulators recognize on sight. We’ll break it down across four layers below.
II. Why “Now”: The Mechanism Behind Verification Becoming the New Bottleneck
Bottom line: Organizations that haven’t upgraded their review process before 2026 H2 will hit a wall — the Q4 peak / end-of-year code freeze / regulatory inspection window is where things will blow up. A three-layer model is the minimum bar, not a nice-to-have.
Fulfilling the promise from AI173’s Section 3 that “Section 4 will cover this in detail.” The specificity of the mid-2026 window: autonomous agents (Claude Code, Codex) are moving from “trial use” to “default use”; organizations that haven’t upgraded their review process before H2 will hit a wall during the Q4 peak / end-of-year code freeze / routine regulatory inspection cycle. First, I’ll explain why “verification” is the most underestimated piece of the new bottleneck, then place it alongside the other two new bottlenecks (defining the right problem, system integration) on a single diagram.
The root cause of this underestimation is that most AI-coding discussions default “verification” to CI/CD, unit tests, and lint passes. This is the world of internet products: code deploys to the cloud, unit tests are green, CI passes, the PR merges, and it ships to production. That flow works at internet-product cadence, but porting it to telecom, finance, manufacturing, or e-commerce does not: in these industries, “verification” means algorithmic filing, MLPS assessment (China’s multi-level protection scheme, comparable to a sector-specific security audit), cross-border data transfer review (mapped to Standard Contract Clauses under China’s Data Security Law, or GDPR DPIA in the EU), CAB change approval, reconciliation audits, and regulatory reporting—none of which has anything to do with code, yet each of which takes weeks. AI173 already showed the picture (coding speeds up, the bottleneck is verification), so I won’t repeat it here. The point is the question it leaves on the table: how many gates does AI-generated code have to clear before it can go to production?
Seven to start: automated testing + code review + security scanning + architecture/ADR review + business-rules review + compliance clearance + canary. Each one eats a slice of bandwidth. Stacked together, these seven are “the other side” of AI173’s chart—AI accelerates the segment with the lowest marginal cost (GPU time, license fees), while verification swallows the segment with the highest institutional cost (regulation, filing, reconciliation).
The second underestimated root cause is reducing “review” to mere “code review.” The two main lineages of code review—Weinberg’s egoless programming from The Psychology of Computer Programming (1971, rooted in NASA and academic practice), and Fagan Inspections from IBM in 1976—share a common assumption: code is written line by line, the author understands it best, and after writing, another human reads through it to catch errors. AI dismantles this assumption: code is produced by AI in seconds, the author (AI) does not transfer context, and the reader (the developer) faces an unfamiliar generated artifact. The original “error-catching” assumption no longer holds. The new review assumption becomes—Should this code exist in this file at all? Could it circumvent existing architectural decisions? Does it fall inside or outside the compliance boundary? Will its default configuration turn into a security vulnerability in production?
Each of these three questions demands people who understand the business, the architecture, and the regulatory landscape—tools only play a supporting role. This is the shift from treating review as a lint gate in CI/CD to treating it as a full-fledged engineering governance function.
III. Three-Layer Review Model: AI Pre-Review, Human Gatekeeping, Governance Rules
Bottom line: Scaling up review is not a tooling problem, it’s a routing problem — route each PR by risk tier to Layer 1 (automated), Layer 2 (human spot-check), or Layer 3 (governance sign-off). The three layers stack; each layer does its own job. Tooling, process, and governance move in parallel.
Compress the analysis above into an actionable structure. The three-layer model is not a substitution relationship — it’s additive. Every PR passes through all three layers simultaneously; each layer owns a distinct category of issues.
Layer 1 Runs on a Seconds-to-Minutes Cadence — Every AI-Written Line Goes Through Tools First
The moment a PR is opened, a stack of automated reviewers jumps on it. CodeRabbit, GitHub Copilot Review, Sourcery, Cursor BugBot, and Antigravity Review all deliver inline comments within tens of seconds to a few minutes of PR creation, flagging lint issues, security vulnerabilities, duplicated code, naming smells, and risky dependencies.
This layer is cheap to run. Whether a team ships two PRs a week or two hundred, the subscription cost stays flat. Coverage is universal because every PR gets scanned. In throughput terms, it is the foundation.
But the blind spots are just as clear as the benefits. Automated review cannot resolve architectural alignment, compliance boundaries, or business correctness. Tools like CodeRabbit position themselves as catching the majority of obvious issues automatically. The residual risks — default configurations, permission boundaries, error-handling paths hidden in the details — still require human judgment. This layer is the foundation, not the finish line.
Layer 2 — Hours to a Day
High-risk changes (touches to core modules, database schema migrations, authentication/billing/compliance logic) must be manually spot-checked by a small review group composed of an architect, the business owner, and a security lead. A meaningful share of the issues CodeRabbit reports (1.82–2.74× more security vulnerabilities) and Apiiro reports (322% more privilege-escalation findings) only surface at this layer: AI-generated code that looks right and even runs, but hides unsafe defaults, blurred permission boundaries, and unhandled edge-case paths in the details.
Low- and medium-risk changes can be handled by sampling. Based on what we’ve seen in internal-training engagements, a 20%–30% sampling rate works well — not an industry standard, just an empirical starting point. Not every PR needs a human pair of eyes. The point is to free human bandwidth from “review everything” to “review what matters most.”
The most common trap at this layer is silent downgrade. Teams quietly relax what counts as “high risk” so AI-generated PRs can move faster. Loosening the standard feels fine right up until an incident lands.
Learn AI Slowly #175 — Layer 3 (Days to Weeks)
Layer 3 covers the days-to-weeks time horizon — changes that touch compliance boundaries, regulatory reporting, cross-border data transfer, SLAs, or cross-team architecture. They go through a Change Advisory Board (CAB), filing reviews, MLPS / Cybersecurity Classified Protection assessment, and direct regulator engagement. This is the orange “AI cannot move this” block in the AI173 diagram, and the single most expensive cost item in heavily regulated industries.
The judgment call from AI174: AI cannot take on Layer 3, but if Layer 1 and Layer 2 are done well, they can intercept the vast majority of low-risk changes before they ever reach Layer 3 — roughly 80–90% by our internal training customer samples. The remaining 10–20% of high-risk changes still go through CAB, which shifts CAB’s attention from the whole company onto the changes that genuinely need governance. Shorter CAB queues, faster overall delivery — that is the “governance-bandwidth dividend” most often underestimated when review tiers are upgraded.
Layer 3 sign-off must be on paper. Every PR routed to Layer 3 has to leave behind a complete evidence chain: PR diff, review comments, dual sign-off from the business owner and the compliance owner, timestamps, and the model verification report as an attachment. Retention periods: 5 years for financial services, 3 years for telecom (reference: PRC PIPL Article 55 + CBIRC Order No. 9 of 2020 + MIIT Algorithm Filing Administrative Measures). This is hard evidence for regulator communication, not paperwork compliance.
Three Layers of Stacked Key Design: The trigger condition is encoded by risk level — not by lines of code or PR size. In practice, risk-level classification cannot rely on AI self-assessment — the model has no compliance awareness and doesn’t know that touching a customer ID field is a PIPL red line. It must come from a two-pronged mechanism: a manual checkbox in the PR template by the PR author (touched schema? auth? billing? compliance boundary?) plus CODEOWNERS rule confirmation. The checkbox outcome routes the PR to the corresponding layer: low-risk PRs go through Layer 1 auto-merge (whitelisted paths only, with an error-fuse mechanism — if any auto-merged PR within 30 days causes a production incident, auto-merge pauses and all such PRs fall back to full manual review); medium-risk PRs go through Layer 2 spot-check; high-risk PRs go through Layer 3 governance. This “risk-adaptive routing” is the highest form of review escalation.
IV. Review Tool Selection: CodeRabbit Isn’t the Only Answer, but It’s the Current De Facto Baseline
Conclusion: Rank selection dimensions as “rule customizability > PR comment quality > integration depth > price”; finance, government, defense, and telecom core domains require private deployment or self-hosting, but private deployment isn’t the end state — it must be paired with a PIPL §21 data entrustment processing agreement (a contractual safeguard equivalent to a Data Processing Agreement under GDPR Article 28).
Collapsing the three-layer model down to the tool level. This section only addresses Layer 1 selection — Layers 2 and 3 are largely an organizational and process problem, and tools don’t move the needle much there.
The top-installed AI review category on GitHub Marketplace is CodeRabbit (Series B in September 2025 at a $550M valuation, ARR of $40M projected by Q2 2026 per Sacra) — it embeds an “AI reviewer” directly into the PR comment stream, with each comment carrying click-through explanations, suggested fixes, and a severity tag. It’s particularly effective at surfacing unit-test blind spots. CodeRabbit has the deepest GitHub Actions integration, uses tiered per-PR pricing, and the enterprise tier adds private models, allowlists, and an internal knowledge base.
GitHub Copilot Review has only one reason to pick it: you’re already on GitHub Enterprise and don’t want another vendor. The dealbreaker is that its rules can’t be tuned deeply — over time its rule library will fall behind CodeRabbit’s.
Sourcery is the strongest automated reviewer in the Python ecosystem: it delivers refactoring suggestions directly at the PR stage (not just flagging issues, but rewriting code), and it’s particularly effective at completing type annotations and paying down technical debt. For polyglot teams it’s less useful — TypeScript and Go support is catching up, while coverage for other languages remains thin.
Cursor BugBot stands out by reading the conversation context inside the Cursor editor. Whatever you’ve been discussing with the AI, BugBot can see it, and it uses that context to deliver targeted reviews of the generated code. It’s unavailable for projects that aren’t developed in Cursor.
Antigravity Review is the review capability built into Google’s Antigravity platform, released in November 2025. It runs on Gemini 3 and inherits Google Cloud’s enterprise-compliance foundation. As of H1 2026, the product is still iterating quickly, the rule library isn’t as deep as CodeRabbit’s, and the enterprise pricing and deployment model are still being tuned.
Selection Criteria — Priority Order: rule customizability > PR comment quality > integration depth > price. For long-term use of a Layer 1 tool, if the rules can’t be customized, you’re locked into whatever safety model it ships with. Poor PR comment quality (e.g., an AI reviewer saying “something looks off here” without explaining why or how to fix) wastes developer time. Integration depth affects onboarding cost. Price ranks fourth — not because it’s unimportant, but because top-tier tools typically differ by under 30% in price, while the gaps in the first three dimensions are far larger.
Two Counterintuitive Selection Principles: First, in finance, government, defense, and telecom core domains, private deployment or self-hosting is the price of entry. But private deployment isn’t the finish line — review tools need to see your full code (PR diff + repository history), which effectively means handing your code to a third party for processing. That requires a third-party data-processing agreement (China’s PIPL §21 on entrusted processing) on top of technical isolation. Second, AI pre-review and human review aren’t an “either/or” — stacking two Layer 1 tools like CodeRabbit + GitHub Copilot Review is standard practice in large organizations. Their rule sets differ and the vulnerability types they catch are complementary; any single tool always has blind spots.
Five. Industry Rollout: How Review Upgrade Takes Shape in Each Regulatory Context
Bottom line: the tool layer (Layer 1) is shared across industries, but the process layers (Layer 2/3) must be redesigned per sector — telecom needs device security assessment, finance needs the three lines of defense for model governance plus independent MVU validation, manufacturing needs MES + supply-chain traceability, and e-commerce needs major-promotion surge handling plus tiered risk classification.
Telecom — Upgrading the Review Process for Plan/Billing Changes. A review from a regional carrier’s internal AI training program shared a diagram with me: every plan change has to pass through 11 checkpoints from coding to production. AI compressed the “coding” stage from 2 days to 0.5, but five other stages—CAB review, algorithm filing (because the billing model is involved), MLPS assessment (等保测评, China’s Multi-Level Protection Scheme evaluation), cross-border data transfer (a foreign model is used, so it falls under MIIT’s Industrial and Information Technology Data Security Management Measures (Trial) dedicated outbound negative list rather than the standard PIPL contract), and reconciliation audits—each consume anywhere from several days to a full month. Algorithm filing alone typically takes 4–6 months from document preparation to feedback from MIIT. That is the real bottleneck. End-to-end delivery timelines barely moved. The direction for upgrading the review process: Layer 1 tools must detect changes touching billing/authentication/compliance modules and automatically flag them as high-risk, routing them to a joint sign-off by the business owner and compliance owner at Layer 2. The CAB layer should conduct second-pass review only on changes that genuinely trigger regulatory reporting. The essence of this path is compressing CAB bandwidth from handling the full volume of all changes (including emergency patches), 5,000–8,000 per month, down to the changes that truly need governance (high-risk only), 100–200 per month. Before the upgrade, the review bandwidth bottleneck sat at the CAB; after the upgrade, the CAB actually becomes the fastest checkpoint because eight of the previous eleven stages are now handled by automated or rule-based pre-screening.
Learn AI Slowly 099 — The Telecom Industry’s Most Overlooked Pain Point Isn’t the Change Advisory Board; It’s Model Explainability
In telecom, the most hidden pain point is not the CAB—it is model explainability. Billing systems must be able to trace every line item on every invoice back to its rate source, and once a customer dispute lands, an AI black-box model has to be unpacked from top to bottom. When the top three 12300 complaint scenarios get fired off (number portability, billing reachability, and service suspension/resumption), they must clear group-level consumer-protection pre-review before launch. No Change Advisory Board can stand in for that.
Finance — Upgrading Credit Risk Model Review. The real workflow for pushing a risk model live inside a bank’s core system is strictly sequential, not parallel: MVU (Model Validation Unit) independent validation → Model Risk Committee approval → business unit applies for regulatory filing → regulator feedback → go-live after filing is approved. The surface area where AI coding can realistically speed things up is narrow (script generation, feature engineering code, data preprocessing code), but every change bumps up against a regulatory boundary — touching labels invokes “material model change requires re-filing” under Article 24 of the Administrative Measures for Internet Loans by Commercial Banks and CBIRC Order No. 9 of 2020. Where the review stack needs to evolve: Layer 1 must detect changes to features, labels, thresholds, or model weights and force a high-risk routing path; Layer 2 requires dual sign-off from a credit risk lead who understands the business and a data compliance lead, with the MVU remaining organizationally independent from both business and IT (a hard requirement under CBIRC Order No. 9 of 2020); Layer 3 adds model validation, EAST regulatory reporting, 1104 reporting, PIPL (PRC Personal Information Protection Law) assessment, and algorithmic fairness review (variables such as gender, age, and geography should not be used).
The Real Pain Point: Why People Before Tools
A joint-stock bank rolled out an AI feature engineering tool, expecting faster model development. Instead, Model Validation Unit (MVU) queues ballooned from 8 to 12 weeks. The reason was not the tool itself but the human review bottleneck that followed. Every AI-generated feature now required manual PSI/CSI drift checks, line by line, before sign-off.
On top of that, MVU and the data compliance team kept colliding over access. Validation analysts needed to see raw feature distributions to do their job; compliance, citing obligations under PIPL (China’s Personal Information Protection Law), would not let them touch customer-level data directly. Everything had to flow through a narrow channel: a model validation sandbox plus aggregated, masked features. Friction was baked in.
The lesson is straightforward. Staff Layer 2 properly before you talk about tools. No platform, however capable, substitutes for people who understand both the business and the regulatory perimeter and can spot-check what the model produces. Without them, any plan to step up model review rigor is built on sand.
Manufacturing — Upgrading Code Review for MES Process Changes. The appeal of AI-generated code in manufacturing is strong (line integration, quality-inspection models, production scheduling), but changes to the MES often touch safety interlocks, and shifting a single process parameter can bring an entire production line to a halt. Manufacturing know-how runs deeper than it appears on the surface: touching OEE (Overall Equipment Effectiveness) interlocks, SPC (Statistical Process Control) charts, batch-traceability logic, or return/replenishment flows is all high-risk — it cannot be judged by “process thresholds” alone. The direction for upgraded review: Layer 1 must flag anything touching safety interlocks, OEE, SPC, or batch traceability as maximum risk and block auto-merge; Layer 2 requires a co-sign from both a process engineer and a safety engineer; Layer 3 proceeds through trial runs and canary deployment (pilot on a single line with small batches first, verify no side effects on safety interlocks, then scale up). The bottleneck in this industry sits with Layer 2 people: senior process engineers are scarce, and their time is heavily consumed by production. The upgrade is, in effect, a reallocation of resources — pulling their attention away from routine line patrols toward high-risk PR review.
E-commerce — Escalated Review for Promotional Events. AI-generated code delivers the biggest productivity gains in e-commerce (frontend pages, promotional rules, data dashboards, recommendation logic), yet any code change during a major sale event touches the transaction pipeline, the risk-control pipeline, and the financial reconciliation pipeline. A single mistake can cost tens of millions in losses. The direction for review escalation: Layer 1 must flag any change touching sale-related modules, coupons, flash-sale logic, or inventory as the highest risk. Layer 2 requires joint sign-off from both the business owner and the risk-control owner. Layer 3 calls for canary release plus full-chain stress testing. What makes e-commerce distinctive is that promotions run on a fixed window — the days around events like Singles’ Day, 618, and the Lunar New Year shopping festival. During those two-week windows, the review bar should be higher than normal, yet the review bandwidth is actually at its tightest because production incidents compete for the same attention. The proven play here is a “calm in peacetime, strict in wartime” rhythm: about a week before the event window, lock down all high-risk changes so only bug fixes get through, and concentrate whatever review capacity remains on clearing the locked backlog. The goal is to keep high-risk changes from sneaking into the event window itself.
Looking across the four industries, the pattern is clear: the core of an upgraded review process is not buying tools, but redesigning the risk routing. The Layer 2/3 routing conditions differ by industry (telecom: Change Advisory Board + algorithm filing + model explainability; finance: independent MVU + model validation + EAST + algorithmic fairness; manufacturing: pilot runs + canary rollouts + OEE/SPC; e-commerce: peak-sale lock), but the Layer 1 tool logic can be shared: it is all about “identify high risk, auto-tag, force-route.” At the tooling layer, buying one or two Layer 1 solutions for cross-industry use is perfectly fine; at the process layer, you must redesign per industry.
VI. What This Means for Decision Makers
Reverse self-check—Is your team trusting AI output more, or less, over time? How does your AI PR get reviewed—100% full audit, risk-based sampling, or quietly rubber-stamped? How many times has your Layer 3 routing fired in the past 6 months? Of those, how many surfaced issues? How many surfaced incidents? If the board cannot pull up these three numbers, your governance is paper compliance.
Insight #1: A Review Upgrade Is an Organizational Capability Upgrade, Not a Technology Purchase. CodeRabbit Pro runs at $24/seat/month (Pro Plus at $48/seat/month, billed per developer who creates pull requests). For a 200-person team, that works out to roughly $58k a year, with enterprise licensing running another 3–5x higher; a rounding error against an R&D budget in the millions. What actually costs is staffing Layer 2 with real people and re-engineering the process at Layer 3. Those investments cannot be bought with a procurement order. They require the organization to be willing to change, and senior engineers to be willing to spend a portion of their time on review work. The teams that stall on this rollout almost always run it like an IT project: issue licenses, configure the tool, set KPIs. The teams that move it forward sit engineering leads and compliance leads at the same table and have them jointly define the PR routing rules. This is a budget signal that shifts governance from a cost center to a bandwidth asset—and that is when the money moves from “buy more licenses” to “fund the review bandwidth.”
Lesson Two: Before deploying autonomous agents, AI pre-review must already be in place. This is the other side of “fit the brakes before you talk about the engine.” Autonomous agents (such as Claude Code and Codex) can edit a dozen files, open pull requests, and run shell commands on their own. Before those capabilities go live, Layer 1 must be able to recognize which module is being touched and which boundary is being crossed, and then force-route the change to the right tier. A practical bar for “in place”: ≥95% auto-merge pass rate at Layer 1, ≥20% sampling coverage at Layer 2, and zero P0 incidents over three consecutive months. Carlini’s sample of a 100k-line Rust-based C compiler is not far from your reality—an autonomous agent can ship a production-grade project in two weeks, and it can also let a team without proper review accumulate 20,000 production-grade risks in two weeks. A more directly comparable peer case is Stripe’s “Minions” agent, which merges roughly 1,300 PRs per week—zero human-written code, with humans only reviewing. Fully AI-produced output plus human review-only is the signature of this model, and what a properly upgraded review process looks like.
Insight 3: The gains and losses of upgrading code review both scale with your review bandwidth.
Let’s redefine “review bandwidth.” It is not just the human-hours spent at the review table — it is the organization’s total capacity to spot risk, route it, and resolve it. The CodeRabbit finding that AI “catches most surface-level issues automatically” addresses only part of the picture. Whether AI actually helps depends on whether the remaining risks — architectural alignment, regulatory boundaries, business-logic correctness — still get enough human attention at Layer 2 and Layer 3.
The most common failure mode when teams “upgrade” review is the quiet relaxation of guardrails: letting AI-generated PRs auto-merge, easing the Layer 1 rules to make the AI productivity numbers look better, dropping Layer 2 to a 5% sample rate, and letting Layer 3 atrophy into a rubber stamp. The short-term metrics look great, but long-term incident rates climb — AI writes faster and you loosened review, so technical debt scales proportionally. CodeRabbit’s 1.7× defect spike paired with Apiiro’s 322% privilege-escalation surge are the joint price of that kind of loosening, not a single layer’s failure. Review bandwidth must scale with PR volume; any imbalance is a loss of control.
30-Day Implementation Checklist (down to the granularity of “which meeting to convene next Monday, which file to revise”):
- Week 1: Audit the existing PR routing rules and flag every one that touches schema, auth, billing, or compliance. Pull the past 90 days of Layer 3 triggers plus the average queue time, and lock that in as your baseline.
- Week 2: Bring in a Layer 1 tool (CodeRabbit or GitHub Copilot Review, hard-cut anything that fails the “self-hosted only” requirement), wire up the rules, and add a manual risk-tier checkbox to the PR template.
- Week 3: Build out the Layer 2 roster — business owner plus compliance owner for each domain — and set the spot-check sampling rate (20–30% is a sensible default). Backfill CODEOWNERS so every module has a named owner.
- Week 4: Promote five metrics into the PMO weekly report: average PR review time, change failure rate, post-review defect escape rate, average Layer 2/3 queue time, and the number of compliance events triggered by Layer 3 routing. In parallel, lock in three gating thresholds for autonomous-agent eligibility — Layer 1 pass rate ≥ 95%, Layer 2 spot-check coverage ≥ 20%, and zero P0 incidents sustained over three consecutive months.
Companion metrics must catch up: average PR review time, change failure rate, defect escape rate after review, average queue time at Layer 2/3, compliance events triggered by Layer 3 routing, and model validation queue time. I closed AI 173 with an observation: many large enterprises report AI coding ROI to leadership using “how many developers are covered” and “how many seats were bought” — which is precisely the way to hide every real bottleneck. Push these indicators up to the board deck (instead of seat counts and lines of code), and budget will finally shift from “buy more licenses” toward “staff up the review bottleneck.”
Shadow AI governance has to move in lockstep. UpGuard’s 2025 report frames it as “global employees using unapproved generative AI tools” — not just developers. About 80% of employees admit to using AI tools that IT has not approved, with business units sidestepping IT and using ChatGPT to write code on their own — the single biggest headache for compliance officers right now. Upgrading governance without parallel shadow AI controls is like managing “declared weapons” while ignoring “undeclared ones.”
Where this doesn’t apply: if your team has fewer than 50 people, sits outside a heavily regulated industry, doesn’t deploy autonomous agents, and ships fewer than 100 PRs per month, at least 60% of the judgments in this article won’t apply directly to you. Don’t force-fit the structure — land on Layer 1 tooling plus a few targeted spot-checks, and you’re covered.
What’s next
The next installment (AI175) covers the tooling layer: the AI tooling war ended in 2026, but whether the winners are actually usable is a separate question. It’s the story of the two on the throne (Claude Code / Codex), Copilot propped up by procurement inertia, and Antigravity still at the starting line — and the story of how “governance capability decides who gets to use what, and at which tier.” AI174 gives you the structure for upgrading your review process; AI175 gives you the structure for tooling selection. Read together, you get the full picture of “what happens to the organization after AI starts writing the code.”
After finishing this piece, we recommend reading section 3 of AI173 (the new-bottleneck diagnosis) plus the “Four Major Tools” section of AI175 (how governance maps to tooling capability) — three key judgments distributed across three articles.
Want to put these judgments to work in your company?
Learn AI Slowly 037 — AI Programming Tools in the Enterprise: From Pilot to Production
Once AI coding assistants move past the demo stage and into real engineering teams, the questions worth debating usually look like this: can the existing code review process absorb the volume of AI-generated changes? How thin do you staff the middle layer — by PR volume, by module count, or by FTE ratio? Does the CAB and approval workflow need to be re-engineered, or can it ride alongside the current change-management process? And which pilot metrics actually prove the rollout worked?
Diagnostic entry point. Before you spend a single dollar on tooling, pull five numbers from your team: average PR review cycle time, change-failure rate, defect escape rate after review, average queue time at Layers 2 and 3, and the count of compliance events triggered by Layer 3 routing. If you cannot produce even one of these, you are not ready for an AI pre-review tool.
Three engagement tracks we currently offer
Enterprise training. Built around your real codebase and delivery cadence, this engagement walks your team through the three-layer AI review model, Layer 1 tool selection (evaluated across four dimensions — private deployment, rule customizability, integration depth with your existing toolchain, and total cost — covering options such as CodeRabbit and GitHub Copilot Review), re-design of the Layer 2 and Layer 3 processes, and the supporting metrics layer. Deliverables: ① a current-state team scorecard covering review-bandwidth saturation, ② a three-layer rollout roadmap covering the next 3–6 months, ③ a Layer 1 tool-selection decision tree, and ④ a first-draft metrics dashboard. Three days on-site, approximately ¥90,000.
Specialized Consulting: Focused on a single, well-defined decision—for example, evaluating whether to bring in CodeRabbit, how to land a three-layer review model under heavy regulation (financial MVU independence + audit trail / telecom algorithm filing + 12300 complaint handling), or how to re-route AI pull requests through your existing Change Advisory Board cadence. Priced by decision topic (a 5–15 hour engagement counts as one consulting package). Deliverables: decision memo + implementation checklist + one-week follow-up. ¥5K/hour.
1-on-1 Coaching / Private Advisory Board: For VPs, directors, and senior engineers who are “ready to invest seriously in their own growth”—you are already using AI coding tools and want to grow the judgment behind review upgrades, team governance, and cross-functional politics inside your own organization. 12 sessions over 6 months, priced by topic. Deliverables: coaching session notes + phased action retrospectives. ¥180K–360K.
Executive Briefings & Industry Talks: Sessions on AI code review, organizational governance, enterprise AI transformation, and software engineering change. Half-day or full-day, scoped to the host’s needs.
These writings offer a general framework. Concrete implementation still has to be redesigned around each company’s data boundaries, compliance requirements, engineering maturity, and existing review workflow. To engage, reach out via coach@iaiuse.com.
Further reading: The Signboard Methodology v1.0 (Learn AI Slowly 187), a systematic 7-step framework for enterprise AI transformation.
About This Series
“The Transformation of Software Engineering in the AI Era” is a research series aimed at CIOs, CDOs, CTOs, and digital transformation leaders across telecommunications, finance, manufacturing, and e-commerce. It focuses on how AI coding tools are reshaping software delivery workflows, organizational structures, governance mechanisms, and management metrics.
Behind this publication is a small team—myself and one or two long-standing collaborators—dividing the work among AI coding tool research, organizational governance case studies, and coaching engagements. Most of the “projects we’ve walked enterprises through” are ones we have collectively delivered.
The series continuously tracks academic papers, vendor materials, and industry reports, with a research repository now exceeding 200 entries. We tag key judgments with evidence tiers, drawing clear lines between verified facts, vendor claims, industry observations, and author reasoning.
I bring nearly 8 years of consulting and business analysis experience at large enterprises, including a stint at IBM on projects spanning telecom, finance, insurance, and manufacturing. Since then, I have stayed close to the front lines—carrier-side product work, internet product development, and AI application engineering—handling requirements analysis, product design, and cross-functional execution.
References (per-item citation, evidence tier, stance flag)
The judgments in this series on review escalation, organizational governance, and process redesign are drawn from these practices and cross-validated against public research and industry cases. Project-specific details have been anonymized; some industry scenarios are extrapolations from typical problem patterns. Supporting sources are listed below.
CodeRabbit, State of AI vs Human Code Generation Report (2025.12.17, primary source, vendor-sponsored): Analyzed 470 open-source GitHub PRs, comparing AI- and human-authored code (unpaired by file size or complexity). Total defects were 1.7× higher in AI PRs (10.83 vs. 6.45 per PR on average). Security vulnerabilities ran 1.57–2.74× higher across subcategories — XSS 2.74×, improper handling of credentials 1.88×, insecure direct object references 1.91×, unsafe deserialization 1.82×. Logic and correctness issues were 1.75× higher (75% classified as high severity); code quality 1.64×; performance 1.42×; readability 3×+; formatting 2.66×; error handling ~2×; excessive I/O ~8×. This is CodeRabbit’s in-house study, so a vendor-perspective caveat applies; however, the sample and methodology are publicly disclosed. Full report: https://www.coderabbit.ai/whitepapers/state-of-AI-vs-human-code-generation-report. Also covered by The Register, 2025.12.17.
Apiiro 2025.9.4 (vendor perspective): Fortune 50 enterprise repository scan (data period Dec 2024 – Jun 2025). Monthly security findings in AI-generated code jumped from roughly 1,000 to 10,000+ (a 10× absolute-count increase), with privilege-escalation vulnerabilities up 322% in absolute terms and architectural-layer design flaws up 153%; normalized by code-volume growth, the estimated increase is roughly 60–80%. Syntax errors dropped 76%, logical bugs dropped 60%. Reported by The Register, Cloud Security Alliance Labs, and SiliconANGLE.
JetBrains AI Pulse Survey 2026.1 (first-party): 10,000+ professional developers across 8 languages. 90% of developers use at least one AI tool; 70% use 2–4. https://blog.jetbrains.com/research/2026/08/ai-coding-agent-adoption-2026/
Pragmatic Engineer Newsletter (Feb 2026, primary source): ~906 responses, reaching a 150,000-reader audience; 56% of senior engineers report that 70%+ of their engineering work depends on AI tools (self-rated heavy use, not a lines-of-code share). Claude Code leads at 46% favorability (vs. Cursor 19%, Copilot 9%); in companies under 10,000 employees, 75% pick Claude Code, while in companies above 10,000, 56% choose Copilot. https://newsletter.pragmaticengineer.com/p/ai-tooling-2026
GitHub Octoverse 2024 / 2025 (Tier 1): The Octoverse 2025 report reveals that the Copilot coding agent authored 1M+ PRs between May–September 2025; 80% of new developers use Copilot within their first week. The “40–60% PR participation rate” figure is an industry estimate, not direct Octoverse data. Synthesized from the GitHub Engineering Blog and The New Stack.
Stripe Minions (Mar 2026, primary source): Stripe’s “Minions” agent merges roughly 1,300 PRs per week, with zero human-written code (human review only) — the hallmark of this pattern is fully AI-generated output with humans in a review-only role. 500+ MCP tools, AWS EC2 devboxes, and Block Goose for branch strategy. https://stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents / covered by InfoQ on Mar 20, 2026.
Learn AI Slowly 004 — Anthropic Skills Architecture (January 2026, primary source, vendor perspective)
Anthropic has published its Skills design documentation. The core idea is modular task capabilities — folders that teach Claude specific tasks through a skill file combined with progressive context loading. Importantly, this mechanism has nothing to do with routing pull requests. The more common pattern for PR risk routing in the industry is handled by GitHub/GitLab branch protection and CODEOWNERS rules, which route reviews by path or code-owner. Source: Anthropic Engineering Blog.
Carlini / Anthropic (Jan–Feb 2026, Tier-1, primary research): Anthropic researcher Nicholas Carlini ran 16 Claude Opus 4.6 agents in parallel for two weeks, across roughly 2,000 sessions and about $20,000 in API spend, to write from scratch a 100,000-line Rust-based C compiler that compiles Linux 6.9 (x86/ARM/RISC-V) and passes 99% of the GCC torture test. This is closed-domain research that never reached production and shipped without a code review process. Reported by The Register on Feb 9, 2026, and by Ars Technica in February 2026.
METR 2026.2 update (Tier 1, pending verification): Early study covered 16 senior developers, 246 real-world tasks, using Cursor Pro + Claude 3.5/3.7 Sonnet. Result: AI slowed them down by 19% (95% CI 2%–39%), even though they self-reported feeling 20% faster. A follow-up published in February 2026 partially reversed this finding (newly added developers showed a -4% slowdown, and some results for senior developers flipped), so the exact numbers still need to be cross-checked against METR’s original report. https://metr.org/blog/2026-02-24-uplift-update
Microsoft FY26 Frontier Suite / EY Case (first-hand, vendor-aligned): EY rolled out Microsoft 365 Copilot to 150,000 employees and reported a 15% productivity gain (equivalent to 14 hours per person per week, redirected toward client delivery and learning). The deployment subsequently expanded to 400,000+ employees. In finance operations scenarios built on Microsoft Power Platform and Copilot Studio, end-to-end lead time improved by 95% with operating costs down 37% (these figures apply specifically to the finance operations scope, not company-wide). Source: Microsoft Customer Story 25760 / FY26 investor materials.
Atos Agent 365 Deployment (June 2026, primary source, vendor perspective): Atos rolled out Microsoft 365 Copilot to roughly 56,000 employees across 54 countries, using Agent 365 to host 19,000 internal AI agents. In Atos’s own framing, “governance and security are the first gate for agentic AI.” Sources: Microsoft News, June 9, 2026 / CDO Magazine.
Autonomous Agent Capabilities of Anthropic Claude Code and OpenAI Codex (primary sources, vendor perspective): Claude Code can edit a dozen or more files on its own, run shell commands, manage Git, and open pull requests; Codex can spin up several sub-agents that work in parallel on isolated copies of a repository and then merge the results. Sources: Anthropic and OpenAI engineering documentation.
CodeRabbit Company Fundamentals (2025–2026, primary sources): Top-tier in GitHub Marketplace’s AI code review segment; Series B in September 2025 at a valuation of ~$550M; ARR grew nearly 10× over 2025–2026 to ~$40M (Q2 2026, per Sacra); Pro at $24/seat/month, Pro Plus at $48/seat/month (priced per developer who creates PRs). Multi-sourced from Sacra / Reuters / TechCrunch. https://sacra.com/c/coderabbit
GitHub Copilot Review / Sourcery / Cursor BugBot / Antigravity Review (first-party, vendor perspective): Official docs and product pages for each Layer 1 review tool — usable to compare review coverage, rule customization, and integration depth. Antigravity reached GA on 2025-11-18, reported by VentureBeat / PCMag.
Origins of code review (L1): two main lineages — (1) Gerald Weinberg’s The Psychology of Computer Programming (1971), which introduced egoless programming (Weinberg himself was at NASA Goddard Space Flight Center plus a faculty appointment at the University of Nebraska — not an IBM background); (2) IBM Fagan Inspections, formalized by Michael Fagan at IBM in 1976 (Fagan was an IBM employee). The two traditions evolved in parallel. This is the historical anchor for comparing AI-era review with traditional review.
Financial regulation reference (primary): The Interim Measures for the Administration of Internet Loans by Commercial Banks (CBIRC Order No. 9 of 2020), Articles 39–42 (risk-model management) — the three-lines-of-defense model governance (business, IT, and compliance/audit) + an independent Model Validation Unit (MVU) + re-filing with the regulator for material model changes; EAST (Examination and Analysis System) submissions in monthly batches, plus 1104 regulatory reporting; PBOC personal-credit reporting requirements + algorithmic-fairness review (restrictions on variables such as gender, age, and geography).
Telecom regulatory references (primary sources): Ministry of Industry and Information Technology (MIIT) algorithm filing rules for billing and financial algorithms, which trigger dual regulatory oversight; MLPS (Multi-Level Protection Scheme) assessment timelines — Level 2 takes 30 working days, Level 3 takes 45 working days; the top three complaints fielded by hotline 12300 (number portability, bill accessibility, and suspension/restoration management); and the Data Security Management Measures for the Industrial and Information Technology Sector (Trial), including its cross-border data transfer negative list.
PIPL data entrustment and processing (primary source): Articles 21 and 55 of the Personal Information Protection Law (PIPL) — third-party processing agreements and a 3–5 year retention period for processing records, varying by sector.
Stack Overflow 2025 Developer Survey (primary source): a survey of more than 49,000 developers. The share of developers who trust AI accuracy dropped from 40% in 2024 to 29% in 2025 (down 11 percentage points), while 46% now actively distrust AI output (up from 31% in 2024). Code churn rose from 3.1% in 2020 to 5.7% in 2024. https://survey.stackoverflow.co/2025/
Shadow AI (UpGuard 2025, Tier 2): 80% of employees worldwide use unapproved generative AI tools (not just developers), and 68% of security leaders admit to unauthorized AI in the enterprise. Upgrading governance without addressing shadow AI in parallel is a compliance blind spot. https://www.upguard.com/resources/the-state-of-shadow-ai
Author’s own cases (desensitized): ① AI internal training at a regional carrier (Q4 2024, post-mortem across 11 checkpoints, desensitized) ② Credit risk review upgrade discussion at a joint-stock bank (H1 2025, desensitized) ③ Redesign of the MES process-change review workflow at a large manufacturer (H2 2025, desensitized) ④ Pre-sales traffic lock drill for a top e-commerce platform’s 11.11 promotion (2025, desensitized).
Note on case desensitization: The carrier, financial, manufacturing, and e-commerce cases referenced in this article draw on the author’s experience delivering AI training and tracking digital transformation teams in telecom-related contexts. They have been desensitized; the industry-implementation passages represent typical problem scenarios, not specific client engagements. Any citation should be marked as desensitized.
buymeacoffee
