PLNOG22 - Piotr Wydrych - BGP w świecie SDN

### Overview In this presentation at PLNOG22, Piotr Wydrych discusses the architectural evolution of large content distribution network clusters, specifically moving away from traditional expensive core routers and transitioning from OpenFlow-based Software Defined Networking (SDN) to using BGP as the control and programming protocol. While SDN controllers and BGP were traditionally seen as disjoint networking paradigms, the team unified them by using centralized software controllers that communicate forwarding rules to merchant silicon switches via BGP.

### Key Topics - **Scaling Challenges in Modern Clusters:** Modern internet traffic is heavily concentrated in major metropolitan areas with high-bandwidth interconnects and exchange points. Traditional designs relying on massive, single-chassis routers face prohibitive costs and scaling limits when handling multi-terabit traffic loads. - **Limitations of OpenFlow in Production:** The initial SDN deployment attempted to use OpenFlow across commodity hardware. However, vendor-specific OpenFlow implementations, inconsistent rule installation confirmations, slow state dumps during reconnections, and lack of active vendor support made it impractical for robust, large-scale production use. - **BGP as a Transport Protocol for SDN:** The architecture was redesigned by swapping the execution layer from OpenFlow to BGP. Because BGP is mature, widely supported across switch hardware, and features deterministic session teardown and state synchronization, it serves effectively as a transport protocol to inject forwarding rules computed by the SDN controller directly into switch silicon. - **Handling Switch FIB Constraints:** Inexpensive commodity switches lack the memory capacity to hold the full Internet routing table (hundreds of thousands of prefixes). The SDN controller dynamically computes and injects only the most popular and relevant prefixes (e.g., 60,000 out of hundreds of thousands) based on traffic patterns and local CDN demands, leaving default routes or alternate paths for the remainder.

### Technical Details - **Pre-computed Forwarding vs. Reactive Controller Hits:** Unlike early academic OpenFlow designs where unknown packets triggered controller round-trips, the centralized controller pre-calculates routing tables and pushes them in advance. Unknown or unrouted packets are simply dropped silently in silicon, preventing switch CPU exhaustion. - **Redundancy and Path Determinism:** Controllers run redundantly in pairs. Because standard BGP path selection ties can lead to non-deterministic path choices across multiple controllers (especially with BGP Add-Path extensions), additional software criteria are applied to ensure consistent FIB state and prevent routing loops. - **Traffic Engineering and Load Balancing:** The central controller monitors port utilization across outgoing links (e.g., transit and peering) and samples prefix traffic. If a link approaches high utilization (e.g., >90%), the software can dynamically shift specific prefix subsets to alternate transits or peering links to balance load before packet loss occurs.

### Takeaways - OpenFlow's real-world vendor fragmentation and operational complexities led to its retirement in this environment. - BGP can serve reliably as a programmable transport protocol between external SDN controllers and switch hardware, leveraging decades of battle-tested hardware implementation. - Successful network modernization can be achieved incrementally, progressing from small pilot clusters to fully software-controlled multi-terabit production networks.