Network storage transports in containers

### Overview In this talk and open floor discussion, Chris Leech from Red Hat explores the difficulties encountered when running network storage transport control planes (such as iSCSI, NVMe-oF/TCP, and FCoE) inside containers. When users attempt to run user-space storage management daemons within network namespaces, the tools often fail because transport drivers and control interfaces were historically designed with the assumption of a single, initial network namespace. Leech presents findings from his investigations and patch efforts, sparking a technical debate among kernel and container developers regarding namespace lifetimes, persistence, and storage device isolation.

### Key Topics - **Transport Driver Limitations:** Drivers such as Fibre Channel over Ethernet (FCoE), ATA over Ethernet (AoE), and Distributed Replicated Block Device (DRBD) explicitly tie operations to the initial network namespace. Only iSCSI lacked explicit checks, but its user-space daemons (such as `iscsid`) fail when multiple instances run across different namespaces without visibility filtering. - **Control Plane Isolation vs. Visibility:** While NVMe/TCP utilizes a character device interface that operates in whatever namespace `nvme-cli` runs, established connections remain visible across namespaces. For iSCSI, Netlink control interfaces and transport objects require namespace-aware filtering. - **Namespace Lifetime Management:** When a container process terminates or restarts, associated storage transport connections in the kernel risk becoming orphaned or unmanageable if their lifecycle is tied to process IDs. - **Block Device Isolation and Block Namespaces:** The discussion examines why network storage control planes are challenging to fully containerize, noting the lack of block device namespaces and global device exposure in `devtmpfs`.

### Technical Details Leech described patching iSCSI Netlink control interfaces to listen per network namespace, followed by isolating transport objects to avoid conflicts when multiple `iscsid` processes manage different networks. While transport objects could be partitioned, underlying storage block devices remained visible system-wide due to the absence of block namespaces.

The conversation focused on how to handle the lifecycle of network namespaces referenced by storage connections. Attendees debated whether kernel block devices should hold a reference to the network namespace to keep it alive or whether the storage device should terminate when the container's PID 1 exits. Discussion also covered `nsfs` mechanisms in user space, such as bind-mounting `/proc/<pid>/ns/net` file descriptors to preserve a network namespace across container restarts, allowing subsequent containers to rejoin via `setns`.

Audience members also addressed the Kubernetes Container Storage Interface (CSI) model, where a container mounts network storage (such as NFS) and exposes it to other pods using bidirectional mounts. Participants noted that while NFS leverages mount namespaces, block devices (like iSCSI) expose global major/minor numbers in `devtmpfs`, rendering device cgroups merely an access restriction rather than complete namespace isolation.

### Takeaways Running network storage control planes inside containers exposes fundamental gaps at the intersection of Linux network namespaces and block storage subsystems. Resolving these challenges requires deciding whether device lifetimes should dictate namespace persistence, implementing robust user-space lifecycle management via persistent namespace file descriptors, and addressing the lack of block device namespaces in the kernel.