Bazel build analytics with ThoughtSpot

### Overview In this presentation, Shishir and Sahita from ThoughtSpot discuss dogfooding ThoughtSpot's analytics platform to extract actionable insights from Bazel build and test processes. Facing large volumes of complex, fragmented build and test logs, the team sought to avoid generic, non-customizable, ready-made dashboards. Instead, they applied ThoughtSpot to analyze build data, identify bottlenecks, optimize resource allocation, and improve build and test workflows using an open-source repository (Abseil C++ / Absil CPP) for demonstration.

### Key Topics The speakers focus on identifying core build and test metrics necessary for optimization: - **Cache Hit Rate:** Tracking the percentage of work avoided by pulling artifacts from remote cache. - **Critical Path Duration and Ratio:** Evaluating the longest sequence of dependent actions, which represents the lower limit of build duration, as well as the critical path ratio (critical path duration divided by total duration) to gauge the degree of build parallelism. - **Build and Test Duration Metrics:** Monitoring overall build duration, average test duration, and duration per invoked target to isolate slowest targets and tests. - **Test Health Metrics:** Evaluating test success, failure, flaky percentages, test count per component, and cache hit rates across individual tests.

### Technical Details During the live demonstration, Sahita showcases several specific KPIs, charts, and analysis workflows: - **Cache Hit Rate Trends:** Line charts illustrated a noticeable decrease in cache hit rates starting in July, which directly caused increased build durations. Concurrently, total actions exhibited an upward trend (rising 20% since September). - **Change Analysis:** By executing a change analysis and selecting a specific test attribute, the system demonstrated that changes in that single test were responsible for 7% of the total 20.2% increase in actions. - **Critical Path Optimization:** Aiming for a critical path duration rate close to 1 is presented as a strategy to optimize resource usage, lower contention, and speed up code iterations. To address low duration rates recorded in May, additional resources and pods were allocated. - **Target and Test Filtering:** Bar diagrams and monthly trend charts pinpoint which invoked targets and tests contribute the most to build and test delays. Users can dynamically filter by specific targets, test names, or components. - **Test Status Tracking:** While the demo repository showed no flaky tests, the success rate was below 100% due to a test consistently failing with a timeout.

### Takeaways Though implementation details were bypassed due to time constraints, the speakers emphasize that customizable KPIs, dashboards, and monitors make it straightforward to query build metrics. Beyond aggregate trends, the tooling enables teams to track failing or flaky tests across commit SHAs to locate culprit commits and make data-driven resource allocation decisions to control costs.