E2E Observability for Connected Vehicle Services Including 5G Cellular... Masanori Itoh & Kota Endo

## Overview Masanori Itoh (Toyota Motor Corporation) and Kota Endo (K Corporation) presented a proof-of-concept for end-to-end observability in connected vehicle services operating over 5G networks. The work addresses the challenge that vehicle manufacturers can identify troubles on the vehicle and cloud/center sides, but have limited visibility into network-layer failures. By partnering with network operators like KDDI, Toyota aims to identify which vehicles are affected by network failures and take proactive actions.

## Key Technical Approach The system leverages open telemetry and distributed tracing to correlate vehicle identities with 5G network events. Initially, the team attempted two approaches: using REST APIs from 3GPP-defined network functions (SMF, UPF) and automatic instrumentation via OpenTelemetry. However, both proved insufficient—the network function APIs returned incomplete information, and Go language automatic instrumentation had limitations with context propagation.

As a pragmatic workaround, they developed a log-watching tool that monitors Session Management Function (SMF) log files in real-time. By parsing these logs, the system extracts PDU session identifiers, User Equipment (UE) IMSI numbers (phone identifiers), assigned IP addresses, and the affected User Plane Function (UPF). This information is exported to a trace aggregator, enabling correlation between network failures and specific connected vehicles.

## Implementation Details The proof-of-concept was built using Free5GC, an open-source 5G core implementation written in Go, deployed on virtual machines. The team separated UE registration (control plane) from PDU session establishment (user plane) to simplify testing. Two tracking areas were configured per test scenario. Upon receiving a UPF failure notification from the network operator, the system can extract mapping information to identify affected vehicles within seconds; UPF failure detection itself takes roughly 10 seconds to one minute.

Key extracted data includes: selected UPF identifier, allocated IP addresses, IMSI, and PDU session IDs. Current limitations include handling only gNB/base station level information and inability to track tiered UPF architectures. The solution operates without modifying the 3GPP protocol or existing commercial network function implementations.

## Challenges & Solutions Initial REST API approaches failed because commercial 5G implementations did not expose sufficient diagnostic data. Automatic instrumentation was limited by the Go language implementation of Free5GC. The team also noted that KDDI used manual instrumentation for non-HTTP protocols (PFCP, N1AP), which required special handling. Security considerations include avoiding transmission of dangerous identifiers like IMSI in context propagation; future work will substitute identifiers like IMI.

## Results & Takeaways The prototype successfully demonstrates the possibility of notifying the vehicle service provider about UPF failures with affected UE information, enabling proactive mitigation before failures impact driving vehicles. The approach requires no protocol modifications and leverages open-source components. Toyota and KDDI have already contributed fixes upstream to the OpenTelemetry community. Future work includes scalability testing, dashboard development, and exploring standard APIs as network function implementations mature. The collaboration model between vehicle manufacturers and network operators is positioned as essential for managing the complex observability requirements of connected vehicle services across hybrid cloud and cellular network infrastructure.