Coding & Engineering Business Operations

Brian Scanlan, Intercom's AI use case

AI engineer and R&D leader at Intercom

Built internal Claude Code workflows that improved Intercom’s engineering process: PR-description quality hooks, AI telemetry dashboards, self-improving flaky-test agents, and agent-friendly installation paths for product integration.

The problem

What was broken before AI

As AI coding tools spread through an engineering organization, the hard problem shifts from “can the model write code?” to “can the team trust, measure, and improve the way agents work?” Pull requests may have thin descriptions. Test failures may repeat without anyone capturing the lesson. Teams may not know where AI agents are helping or getting stuck. Products may also be hard for agents to install if the integration flow assumes a human is clicking through every step.

What changed

What the use case made possible

Brian’s team started treating AI coding as an engineering system with feedback loops. A PR-description hook uses Claude Code to assess whether a pull request description is good enough for review. Telemetry pipelines help the team understand AI tool usage and behavior. Flaky-test agents capture failures, fix issues where possible, and remember what happened. Intercom also redesigned install flows so agents can add the product to a codebase more easily.

Why this matters

Why this use case is worth studying

Brian’s workflow is valuable because it moves past the novelty of AI-generated code. The bigger opportunity is building the infrastructure that makes agent work visible, reviewable, and improvable. Intercom is not just asking developers to use AI more. It is shaping the surrounding process so the team can see what agents are doing, improve weak spots, and make their own product easier for agents to adopt.

Use this when

When this pattern applies

Use this pattern when AI coding has moved beyond individual experimentation and the team now needs trust, measurement, and repeatability. It works best when agent output affects shared engineering workflows like pull requests, tests, docs, setup, or product integrations.

Exponential Builder analysis

Agent adoption needs operating rails.

Once coding agents enter normal engineering work, the leverage shifts to the systems around them: review standards, telemetry, failure capture, and clear approval paths.

Handoffs are a high-value place to start.

A PR-description check looks small, but it protects reviewer attention and turns AI output into something teammates can evaluate faster.

Repeated failures deserve memory.

Flaky tests are a good target because the same pain returns over and over; having an agent investigate, attempt a fix, and preserve context makes the next failure less mysterious.

Who this is for

Best fit

Engineering leaders adopting AI coding tools

Developer productivity teams

Platform teams

QA and test infrastructure teams

SaaS teams building products agents need to install

Teams that want to measure AI coding impact instead of relying on anecdotes

What to avoid

Mistakes and warnings

Where this pattern can go wrong if you copy it too literally.

Do not assume more AI usage automatically means better engineering output.

Avoid judging agent performance only by anecdotes.

Do not let weak PR descriptions create extra work for reviewers.

Keep flaky-test fixes reviewable instead of letting agents silently patch behavior.

Make product setup instructions explicit; agents struggle when important steps are implicit.

Public workflow preview

The shape of the workflow

A high-level look at how the use case works, with the reusable pattern made clear.

Improve the review handoff

A Claude Code hook checks whether a pull request description gives reviewers enough context.

Measure how agents are used

Telemetry helps the team see where AI tools are saving time, failing, or creating new bottlenecks.

Turn flaky tests into a learning loop

Agents investigate failures, attempt fixes, and capture lessons so the same issue is easier to handle next time.

Make the product easier for agents to install

Intercom creates integration flows that coding agents can understand and execute.

Keep humans in the approval loop

The system improves first drafts and follow-up work, but human engineers still review the important changes.

Copy the pattern

The reusable idea

Pattern in one sentence

Build the operating layer around AI coding: review quality, telemetry, failure memory, and setup paths that make agent work easier to trust and improve.

Reusable idea

Brian’s use case is a good reminder that AI adoption gets better when it has operating infrastructure. Once a few people on the team are using coding agents, the next step is not just more usage. It is better handoffs, better measurement, better failure capture, and clearer boundaries for what agents should do. The leverage comes from improving the system around the agent, not only the prompt inside it.

Steal this workflow

Build a small “agent operations loop” around one engineering workflow:

Pick one handoff where AI output affects another person, such as PR descriptions, test fixes, or setup instructions.

Define the review standard in plain language: what context must be present, what evidence is required, and when the agent should stop.

Add a lightweight AI check before the human review step.

Log basic events: task type, outcome, failure reason, reviewer intervention, and repeated issue patterns.

For recurring failures, have the agent write a short incident note: what failed, likely cause, attempted fix, and what to recognize next time.

Review the logs weekly and improve one thing: the prompt, the docs, the install path, or the human approval rule.

Expand only after the first loop reliably improves handoff quality without hiding risk from reviewers.

Suggested prompt

“Review this engineering artifact before it goes to a human reviewer. Check whether it explains what changed, why it changed, how it was tested, and what the reviewer should pay attention to. Identify missing context, risky assumptions, unclear setup steps, or repeated failure patterns. If the artifact is not ready for review, suggest a concise improved version and list any questions the author must answer before approval.”

Field notes

Get new AI use cases in your inbox

A short weekly note on how real people are using AI to save time, make money, build tools, and run their lives.

Related use cases

Keep exploring nearby systems.

Browse all

Customer SupportSales & Outreach+1

Minerva CQ team

AI Research Division / product team at Minerva CQ

Real-time contact-center co-pilot workflow where AI listens to live voice support, detects intent and sentiment, retrieves context, updates customer profiles, and guides agents through next-best actions.

Minerva CQOpenSearchLLM APIs

Use case

View use case

Business OperationsCompany AI Adoption

Company S expense processing team

Finance operations team at Company S

Corporate expense-processing workflow where receipts are recognized with OCR/IDP, classified against policy data, routed through generative-AI exception handling, and finalized with human review.

Brity AutomationBrity Automation AI FlowSAP

Use case

View use case

Business OperationsKnowledge Management+1

Included Health clinical team

Clinical team using an internal ambient scribe workflow at Included Health

Telehealth ambient scribe workflow where visit audio is transcribed with Whisper and processed with GPT-4o into SOAP notes and patient instructions for clinician review.

WhisperChatGPTOpenAI Audio API+5

Use case

View use case