IT Ops Query: Block Broadens Goose's Loop with Builderbot for AI Agents

## Overview Brad Axen, head of AI capabilities at Block, discusses Builderbot, an internally developed orchestration layer for coordinating AI agents at enterprise scale. Block, the fintech company behind Square, Cash App, Afterpay, Tidal, and Bitcoin projects, released Builderbot this month to handle the complexity of hundreds of millions of lines of code and hundreds of services. The project emerged from Block's experimentation with applying LLMs to real-world engineering challenges—enabling any engineer to contribute to unfamiliar systems and rapidly diagnosing and fixing production incidents before human intervention becomes necessary.

## Key Technical Components Builderbot uses no custom model training; instead, it leverages frontier models and switches between them based on use case to optimize performance-cost tradeoffs. The core innovation is agentic search, moving beyond retrieval-augmented generation (RAG). Traditional search algorithms failed; what works is a reusable capability layer embedded in the agent's tool-calling loop, allowing agents to detect failed searches and retry with different terms. The system is built entirely on MCP (Model Context Protocol), which Axen describes as "a service mesh but for agents," grounding the models through tools and significantly reducing hallucinations.

The architecture employs two types of loops: inner loops (the core agent loop where an LLM makes an inference, detects tool calls, executes them, and iterates) and outer loops (workflows involving humans or multiple LLMs). Incident response exemplifies outer-loop automation: a new problem triggers an agent to find the root cause, seeks human approval for fixes, then executes changes in another process.

## Performance & Results Builderbot dramatically improves incident response metrics. Time to first response dropped from approximately 45 minutes (system-dependent) to 5 minutes, because AI can immediately check for potential problems. Time to remediation—how long to fully resolve and fix issues for customers—was cut in half, which significantly impacts customer experience. When tested against vendor end-to-end incident response solutions, Builderbot outperformed them because it knows Block's internal systems far better than any vendor could, enabling the agent to search relevant codebases and logs automatically.

## Quality Assurance & Limitations The capabilities layer (tools) grounds models effectively. When agents use tools, hallucinations are rare; code understanding is "basically never wrong." Data understanding, however, remains at "medium quality." Models write correct SQL queries but struggle when multiple definitions exist for the same concept—for example, five different payment tables at Square. Block's pragmatic solution combines human review and an internal system called Query Expert, which remembers correct queries from domain authorities, creating a feedback loop to improve accuracy.

For code changes to important services, humans always review before production deployment. Similarly, analytical queries for high-stakes decisions (e.g., board presentations) receive human validation. Block has begun implementing AI reviewers for lower-risk tier-three services, allowing automatic deployment if the AI approves changes.

## Scaling to Larger Enterprises Axen argues the value of AI agents grows for legacy systems with hundreds of millions of lines of code—precisely where traditional approaches fail. The barrier is largely organizational (compliance, SOC 2 requirements, procedures) rather than technical. Designers increasingly build functional prototypes directly in production systems, eliminating handoff delays. Multiple teams (sales, project management, customer support) now collaborate with AI from the outset, raising conversations to strategic questions rather than implementation details. Block generates a couple thousand internal prototypes monthly through this paradigm.

## Open Source Plans Block will open source components of Builderbot: the desktop app (under a new name), the Slack integration, and possibly other parts, while keeping Block-specific MCP servers proprietary. Buzz, an AI-native communication platform similar to Slack/Discord but with integrated agent capabilities, is already open source (blockbuzz). The desktop app works with any MCP-based setup and appeals to both enterprise users and individual open-source developers.

## Broader Vision Axen emphasizes collaboration across roles as central to enterprise AI's future. By bringing AI into conversations early—before lengthy back-and-forths—teams accelerate without replacing humans. Designers, engineers, sales, and operations work together in real-time, rapidly validating ideas and building higher-fidelity solutions. As models improve at spotting bugs (through patience and thoroughness), autonomous changes to lower-risk systems become viable, gradually scaling up as trust increases.