ESXI下有一个Linux虚拟机,其中通过Docker跑了个Frigate作为家庭NVR,同时用于人脸识别给中控屏做人来亮屏。由于我的宿主机的核心是3代i5的3317u,比较羸弱,所以Frigate中ffmpeg软解视频比较吃力,TensorFlow识别物体也比较吃力,CPU使用率就飙上去了,所以考虑将GPU核显利用起来。本文测试使用的码流为TP摄像头创建的子码流:640*480的H.264
将GPU直通给Linux
首先设置GPU直通并重启ESXI,并给Linux虚拟机添加GPU为PCI设备
接着进入ESXI的ssh,让ESXI启动时不去获取显卡控制权
esxcli system settings kernel set -s vga -v FALSE
重启ESXI,此时即使机器插着显示器,ESXI启动也不输出画面了
接着配置Linux虚拟机:
禁用虚拟显卡:将svga.present由TRUE改为FALSE 让GPU驱动不知道在虚拟机中运行:添加hypervisor.cpuid.v0为FALSE
重启虚拟机,svga画面输出已经消失了。hypervisor.cpuid.v0一定要配置,否则Linux虚拟机无法完成启动。
启动后,通过ssh连接虚拟机,验证直通效果:sudo lshw -C display或者lspci -vnn | grep VGA -A 12
$ sudo lshw -C display *-display description: VGA compatible controller product: 3rd Gen Core processor Graphics Controller vendor: Intel Corporation physical id: 0 bus info: pci@0000:1b:00.0 version: 09 width: 64 bits clock: 33MHz capabilities: msi pm vga_controller bus_master cap_list configuration: driver=i915 latency=64 resources: irq:66 memory:fc800000-fcbfffff memory:d0000000-dfffffff ioport:2000(size=64)
$ lspci -vnn | grep VGA -A 12 1b:00.0 VGA compatible controller [0300]: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:0166] (rev 09) (prog-if 00 [VGA controller]) DeviceName: pciPassthru0 Subsystem: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:2010] Physical Slot: 256 Flags: bus master, fast devsel, latency 64, IRQ 66 Memory at fc800000 (64-bit, non-prefetchable) [size=4M] Memory at d0000000 (64-bit, prefetchable) [size=256M] I/O ports at 2000 [size=64] Capabilities: <access denied> Kernel driver in use: i915 Kernel modules: i915
可见GPU直通成功了。
配置Frigate
https://docs.frigate.video/configuration/hardware_acceleration#configuring-intel-gpu-stats-in-docker
https://docs.frigate.video/configuration/hardware_acceleration#via-vaapi
主要就是需要在docker参数中加上–device /dev/dri/renderD128和–privileged,然后在Frigate配置文件中加上hwaccel_args: preset-vaapi
进入Frigate查看效果:
GPU硬件加速成功,但是intel_gpu_top执行失败,这个原因未知,网上很多人有类似的问题:
Frigate log:
2024-03-12 15:21:21.941076931 [2024-03-12 15:21:21] frigate.util.services ERROR : Unable to poll intel GPU stats: Failed to detect engines! (No such file or directory) 2024-03-12 15:21:21.941085403 (Kernel 4.16 or newer is required for i915 PMU support.) 2024-03-12 15:21:21.941089240 timeout: the monitored command dumped core
直接运行intel_gpu_top:
$ intel_gpu_top Failed to initialize PMU! (Permission denied)
将Detectors交给GPU
需要使用openvino。openvino也支持CPU,效率比默认的CPU Detector高。从文档看,只支持6代以上的CPU。
https://docs.frigate.video/configuration/object_detectors#openvino-detector
https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html
用我的这个3代i5试试。
修改Frigate配置文件:
detectors: ov: type: openvino device: AUTO model: path: /openvino-model/ssdlite_mobilenet_v2.xml model: width: 300 height: 300 input_tensor: nhwc input_pixel_format: bgr labelmap_path: /openvino-model/coco_91cl_bkgr.txt
启动成功:
2024-03-12 15:28:42.137142043 [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO : Model Input Shape: [1,300,300,3] 2024-03-12 15:28:42.137157934 [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO : Model Output-0 Shape: [1,1,100,7] 2024-03-12 15:28:42.137176631 [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO : Model has 1 Output Tensors
使用CPU推理:
可见延迟有287ms。CPU占用非常高,FPS才1.4
使AUTO模式的OpenVINO推理:
可见延迟下降到80ms,CPU占用降低了很多,FPS达到了4
后续
由于上述OpenVINO处于AUTO模式,试着从AUTO改为GPU,发现启动失败了:
2024-03-12 15:50:55.499156863 Process detector:ov: 2024-03-12 15:50:55.502277448 Traceback (most recent call last): 2024-03-12 15:50:55.502450200 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap 2024-03-12 15:50:55.502457812 self.run() 2024-03-12 15:50:55.502467808 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run 2024-03-12 15:50:55.502478795 self._target(*self._args, **self._kwargs) 2024-03-12 15:50:55.502486642 File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector 2024-03-12 15:50:55.502521519 object_detector = LocalObjectDetector(detector_config=detector_config) 2024-03-12 15:50:55.502526047 File "/opt/frigate/frigate/object_detection.py", line 53, in __init__ 2024-03-12 15:50:55.502530992 self.detect_api = create_detector(detector_config) 2024-03-12 15:50:55.502534675 File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector 2024-03-12 15:50:55.502540247 return api(detector_config) 2024-03-12 15:50:55.502545383 File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 32, in __init__ 2024-03-12 15:50:55.502585844 self.interpreter = self.ov_core.compile_model( 2024-03-12 15:50:55.502634947 File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 399, in compile_model 2024-03-12 15:50:55.502646924 super().compile_model(model, device_name, {} if config is None else config), 2024-03-12 15:50:55.502661726 RuntimeError: Failed to create plugin /usr/local/lib/python3.9/dist-packages/openvino/libs/libopenvino_intel_gpu_plugin.so for device GPU 2024-03-12 15:50:55.502667511 Please, check your environment 2024-03-12 15:50:55.502671411 Check 'error_code == 0' failed at src/plugins/intel_gpu/src/runtime/ocl/ocl_device_detector.cpp:194: 2024-03-12 15:50:55.502677424 [GPU] No supported OCL devices found or unexpected error happened during devices query. 2024-03-12 15:50:55.502682322 [GPU] Please check OpenVINO documentation for GPU drivers setup guide. 2024-03-12 15:50:55.502686670 [GPU] clGetPlatformIDs error code: -1001
下次攒一个新点的板子来试试GPU模式的Detector。
日常使用
日常使用接入了2k的摄像头主码流,由于3代i5的GPU不支持H.265 HEVC硬解码,所以只能使用H.264编码。直接上数据