0

Your cart

TOTAL excl.
TOTAL incl.
Pay
Go back

The Hidden Cost of AI in CI: When Agents Send Your Build Budget Through the Roof**

REXAutomatisationDéveloppementCI/CD

2026-12-09 | 12:00 AM - 12:25 AM

Register

Information

Context


AI in CI has become a category of its own. Automated pull request reviews, test generation, auto-remediation, changelog writing, issue classification, refactoring suggestions. Every vendor offers its own agent, and every team wants to plug in at least one. Product announcements all promise the same thing: more productivity, less human effort.
What the demos don’t show is what happens six months in. When the LLM bill exceeds the CI compute bill. When developers have started setting up Slack rules to hide comments from a particular bot. When an accidental workflow re-run burns through $400 worth of tokens without anyone noticing until the end of the month.
This talk tells a real-world story: six months of enthusiasm, one month of realization, and three months of rethinking and restructuring. With concrete numbers, identifiable cost patterns, and the guardrails we eventually put in place.

What the talk covers: a five-part REX


1. The hype. A timeline of the first six months. The first agents we deployed (PR reviews, test generation, issue classification). Enthusiastic adoption, the first wins celebrated in stand-ups, and the snowball effect. The warning signs we ignored: the “you can ignore this bot” comment showing up repeatedly in reviews, CI latency increasing by five minutes, and the monthly LLM provider report we stopped reading.
2. Anatomy of the bill. A breakdown of a real, anonymized monthly bill. Three patterns accounted for 80% of the cost: oversized contexts (the agent receives the entire repo on every call), an overpowered model for the task (a premium model used to classify labels), and silent re-runs (every workflow retry runs all the agents again). How each of these patterns crept in unnoticed. The brutal calculation: we were spending $1.40 per pull request on AI agents. With a team opening 60 PRs a day, that adds up quickly.
3. The invisible human cost. The part nobody measures. Time spent triaging false positives from agents, cognitive fatigue among reviewers who have to validate generated content without sufficient context, the gradual loss of trust in the tools, and—worst of all—real issues getting buried in the noise. One real-world example: a SAST agent flagged 47 vulnerabilities on a PR, 46 of which were false positives. The 47th was real and made it into production. The resulting trust debt is more expensive than technical debt.
4. The guardrails we put in place. Five concrete levers: pipeline-level budgets with alerts, task-specific model selection (no premium model for binary classification), aggressive caching of stable contexts, opt-in rather than systematic agents, and useful-signal metrics for each agent (suggestion acceptance rate, percentage of PRs modified following a comment). The guiding rule: an agent that can’t prove its value within 30 days gets disabled. Three of the five initial agents were cut at this stage.
5. The checklist before adding any new agent. The ten questions to ask before plugging a new AI agent into CI. Is there a hard budget cap? Do we have a useful-signal metric? Is the chosen model properly calibrated to the task? What happens on a re-run? Who will triage the false positives? These questions save us several thousand euros a month—and a lot of human fatigue.

Speakers