The NExt Big Sur(ge) on macOS

### Overview Presented at VB2020 by Avijit Kulkarni and Prakash Jagdari of Increx Technologies, this talk examines the transition away from legacy kernel extensions (KEXTs) toward modern System Extensions in macOS, particularly focusing on the transition leading up to macOS Big Sur. The presentation targets security vendors and developers who currently maintain or plan to build macOS security solutions, detailing why kernel extensions are being phased out, what replaces them, and practical workarounds for technical hurdles encountered during System Extension development.

### Key Topics - **Retirement of Kernel Extensions:** Kernel extensions run in kernel address space and load soon after boot. Because flawed KEXTs pose serious security and stability hazards to macOS, Apple announced at WWDC 2019 its initiative to deprecate third-party KEXTs. macOS Catalina is the last version fully supporting legacy extensions, introducing warnings in macOS 10.15.4+ regarding upcoming incompatibility with future releases like Big Sur. - **Modern System Extensions:** Supported on macOS Catalina and newer, System Extensions permit software to extend operating system capabilities without kernel-level execution. Because they execute strictly in user space, they cannot compromise the core operating system's stability or security. - **System Extension Framework Types:** - *Endpoint Security:* An alternative replacing Kauth-based kernel extensions to monitor and filter file system activity. - *Network Extension Framework:* Replaces legacy Network Kernel Extensions used to filter and block malicious network traffic. - *DriverKit Framework:* Modern alternative to IOKit-based drivers used to manage device access and block unauthorized devices.

### Technical Details and Implementation Challenges The speakers outline several distinct development challenges and their corresponding solutions or workarounds: 1. **Code Signing Errors with Developer ID:** When distributing via Developer ID certificates, ambiguous code signature errors arise. Developers must manually edit the entitlements file and append `-systemextension` to all entitlements. 2. **Concurrent Preference Saving in Multiple Network Extensions:** Network extensions load and save preferences to initiate. Concurrent calls to save preferences cause failures where only one extension successfully starts. Developers must serialize calls to save preferences. 3. **VPN Tunnel Boot Initialization:** For VPN transparent app proxies, starting a tunnel during the boot process via a daemon fails due to framework unpreparedness. The workaround is launching the process via a Launch Agent instead. 4. **Process ID (PID) Retrieval and Source App Identification:** System Extensions do not offer a direct mechanism to query a process ID for traffic flows. Instead of relying on PIDs (which carry security caveats), developers should inspect `NEFlowMetaData` fields and use the bundle identifier to identify the source application. 5. **Loopback/Self-Originating Flow Detection:** Identifying network traffic flows that originate from and return to the System Extension can be accomplished using `NEFlowMetaData` fields. 6. **Sandbox Restrictions:** Because System Extensions are sandboxed, restrictions apply (such as accessing global dispatch objects for incoming/outgoing connections). Developers must configure explicit sandbox exceptions in their entitlements file. 7. **Proxy Implementation Constraints:** Unlike legacy solutions that redirected traffic simply by altering IP addresses and ports, the transparent app proxy in the Network Extension framework does not support direct manipulation. Developers must implement their own proxy layer to interface with existing user-mode proxies or backend servers.

### Takeaways Apple's phase-out of kernel extensions makes migrating to System Extensions inevitable for all macOS security vendors. Developers targeting macOS Big Sur must adapt to user-space paradigms, carefully configure entitlements, work around framework initialization limitations, and begin porting their legacy KEXTs to System Extensions immediately.