PLNOG22 - Jarosław Grabowski - The Age of Enlightenment is Coming, or Telemetry in Networks

### Overview During the PLNOG22 conference, Jarosław Grabowski presented a session on network telemetry as an evolution beyond traditional network monitoring mechanisms like SNMP. The talk focuses on the operational shortcomings of SNMP, architectural models for streaming telemetry from network devices, and practical ways network engineers can build and experiment with telemetry labs at minimal cost using virtual machines and open-source software.

### Key Topics - **Limitations of SNMP**: SNMP requires specific, manual polling of hierarchical OID trees. In large environments with hundreds of devices and thousands of ports, pulling data periodically creates high query overhead, provides only fragmented snapshots in time, and frequently misses transient state changes that happen between polling intervals. - **Push vs. Pull Model**: Telemetry places the transmission mechanism directly on the network device, streaming structured state information outward to one or more collectors in near real-time. - **Operating System Architecture**: Operating systems traditionally split telemetry into internal helper processes that poll other processes, which can degrade stability and recreate polling bottlenecks internally. In contrast, Arista's architecture centralizes state in an internal database (SysDB). Operational processes are stateless consumers; when any state changes, the central process automatically streams updates internally to subscribed agents and externally to collectors. - **Unified Software Images**: Arista utilizes a single binary operating system image across more than 200 hardware models and multiple ASIC families. This approach standardizes telemetry feeds, CLI, and APIs, avoiding behavior discrepancies caused by maintaining separate code branches.

### Technical Details - **Data Volume and Performance**: Even in a stable state, a single device typically generates 100 to 200 events per second, resulting in roughly 1 GB of telemetry data per device per day. Efficient encapsulation and compression are necessary to manage transport and storage. - **Lab Environment & Demo**: The speaker demonstrated telemetry streaming using GNS3 on a home server running virtualized Arista switches and Arista CloudVision. In the live demo, modifying an interface description (`hello plnog`) propagated instantly to CloudVision without manual polling. For open-source setups, engineers can combine Arista virtual machines with time-series databases and tools like Prometheus, Grafana, or Kibana. - **In-band Telemetry and Analytics**: Ongoing developments include In-band Telemetry (INT), where metadata is injected into packet headers, updated across network hops, and stripped upon network exit. Because the resulting dataset constitutes a big data challenge, automated analytic background processes ("turbines") continuously parse the database to detect anomalies and flag unexpected network behavior.

### Takeaways Streaming telemetry provides a real-time, high-fidelity view of network states that eliminates the visibility gaps inherent in periodic SNMP polling. Network engineers can test and validate end-to-end telemetry pipelines locally using virtual machines, GNS3, and open-source stacks like Prometheus and Grafana before implementing large-scale historical retention and automated anomaly detection.