Planting Bazel in barren soil: A Perl Story

### Overview In this talk, Manuel from Booking.com describes how the organization replatformed part of its legacy Perl monorepo using Bazel. Founded in 1996, Booking.com initially ran a single large monolith where the entire codebase was deployed to every server. By 2017, the adoption of Kubernetes made shipping a four-gigabyte codebase with 50–60 dependencies per service unsustainable. The codebase contained around 10 million lines of Perl code, lacked dependency management, and suffered from a single `lib` directory containing roughly 48,000 unstructured files. Manuel details the technical journey of breaking down this monolith into manageable distributions and adopting Bazel without major disruptions.

### Key Topics - **Monorepo Evolution and Project Ravioli**: Transitioning from unstructured "spaghetti code" to "Project Ravioli," which organized the codebase into smaller, isolated distribution chunks (moving from 48,000 raw files in `lib` to roughly 6,000 distributions). - **Engineering Culture and CI Modernization**: The evolution of source control and CI at Booking.com, including adopting GitLab in 2017 to establish real CI and code ownership, transitioning away from on-demand Jenkins, and mandating merge requests in 2020. - **Adopting Bazel for Perl**: Writing custom Perl rules and tooling to manage Bazel targets and dependencies, addressing transitive test dependencies, and building associated assets like JavaScript. - **Automation and Developer Experience**: Developing custom automation—specifically a Starlark parser written in Perl—to auto-generate and manage `BUILD.bazel` files so developers don't have to manually write Bazel configurations. - **Build Infrastructure and Caching**: Addressing runner limitations in GitLab, deploying bare-metal runners, implementing remote caching and build event services (BES), and evaluating Remote Build Execution (RBE).

### Technical Details - **Timeline**: Project Ravioli launched in Q2 2019, initial Bazel work started in Q1 2020, and a "big bang" migration took place in Q1 2021. - **Rules and Symlinks**: Early custom Perl rules generated 20,000 to 40,000 symlinks for small "Hello World" targets. The team also adopted `bazel-dnf` to leverage existing CentOS RPMs. - **Circular Dependencies**: While Perl naturally permits circular dependencies, Bazel strictly prohibits them. The team temporarily tolerated runtime circular dependencies during transition while checking against new circular dependencies in CI. - **Automated Starlark Management**: Because tools like Gazelle plugins were unavailable or unfamiliar, the team wrote their own Starlark parser in Perl that understands Perl dependencies, achieving 100% automated management of Bazel build files. - **Runner and Cache Architecture**: Built a custom bare-metal Docker-based runner setup for GitLab. For remote caching and BES, they initially used JFrog, evaluated BuildBuddy open source, and used Buildbarn for RBE exploration. They also began packaging libraries into `.par` or tar/tgz-like chunks (similar to Python eggs) to ship a few thousand archives (~100 KB each) instead of 40,000 loose files.

### Takeaways - **Measurable Improvements**: 55% of the company's approximately 350 services have migrated to Bazel (with 11% of services created in the past 12 months). Docker image sizes dropped from 2.2 GB down to 500 MB for a basic service (still containing CentOS 7 and RPM support). - **Performance and Reliability**: CI jobs scaled from 100,000 per month in 2020 to 1.6 million per month, while p50 build/test times dropped from 22 minutes to 2 minutes. Test coverage also grew by 12.5% over a 12-month period. - **Developer Ergonomics**: Hiding Bazel configuration behind automated generation tooling minimized developer friction and avoided the burden of maintaining build files manually.