ESXI下有一个Linux虚拟机，其中通过Docker跑了个Frigate作为家庭NVR，同时用于人脸识别给中控屏做人来亮屏。由于我的宿主机的核心是3代i5的3317u，比较羸弱，所以Frigate中ffmpeg软解视频比较吃力，TensorFlow识别物体也比较吃力，CPU使用率就飙上去了，所以考虑将GPU核显利用起来。本文测试使用的码流为TP摄像头创建的子码流：640*480的H.264
将GPU直通给Linux
首先设置GPU直通并重启ESXI，并给Linux虚拟机添加GPU为PCI设备
接着进入ESXI的ssh，让ESXI启动时不去获取显卡控制权
esxcli system settings kernel set -s vga -v FALSE
重启ESXI，此时即使机器插着显示器，ESXI启动也不输出画面了
接着配置Linux虚拟机：
禁用虚拟显卡：将svga.present由TRUE改为FALSE 让GPU驱动不知道在虚拟机中运行：添加hypervisor.cpuid.v0为FALSE
重启虚拟机，svga画面输出已经消失了。hypervisor.cpuid.v0一定要配置，否则Linux虚拟机无法完成启动。
启动后，通过ssh连接虚拟机，验证直通效果：sudo lshw -C display或者lspci -vnn | grep VGA -A 12
$ sudo lshw -C display
*-display
description: VGA compatible controller
product: 3rd Gen Core processor Graphics Controller
vendor: Intel Corporation
physical id: 0
bus info: pci@0000:1b:00.0
version: 09
width: 64 bits
clock: 33MHz
capabilities: msi pm vga_controller bus_master cap_list
configuration: driver=i915 latency=64
resources: irq:66 memory:fc800000-fcbfffff memory:d0000000-dfffffff ioport:2000(size=64)
$ lspci -vnn | grep VGA -A 12 1b:00.0 VGA compatible controller [0300]: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:0166] (rev 09) (prog-if 00 [VGA controller]) DeviceName: pciPassthru0 Subsystem: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:2010] Physical Slot: 256 Flags: bus master, fast devsel, latency 64, IRQ 66 Memory at fc800000 (64-bit, non-prefetchable) [size=4M] Memory at d0000000 (64-bit, prefetchable) [size=256M] I/O ports at 2000 [size=64] Capabilities: <access denied> Kernel driver in use: i915 Kernel modules: i915
可见GPU直通成功了。
配置Frigate
https://docs.frigate.video/configuration/hardware_acceleration#configuring-intel-gpu-stats-in-docker
https://docs.frigate.video/configuration/hardware_acceleration#via-vaapi
主要就是需要在docker参数中加上–device /dev/dri/renderD128和–privileged，然后在Frigate配置文件中加上hwaccel_args: preset-vaapi
进入Frigate查看效果：
GPU硬件加速成功，但是intel_gpu_top执行失败，这个原因未知，网上很多人有类似的问题：
Frigate log：
2024-03-12 15:21:21.941076931 [2024-03-12 15:21:21] frigate.util.services ERROR : Unable to poll intel GPU stats: Failed to detect engines! (No such file or directory) 2024-03-12 15:21:21.941085403 (Kernel 4.16 or newer is required for i915 PMU support.) 2024-03-12 15:21:21.941089240 timeout: the monitored command dumped core
直接运行intel_gpu_top：
$ intel_gpu_top Failed to initialize PMU! (Permission denied)
将Detectors交给GPU
需要使用openvino。openvino也支持CPU，效率比默认的CPU Detector高。从文档看，只支持6代以上的CPU。
https://docs.frigate.video/configuration/object_detectors#openvino-detector
https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html
用我的这个3代i5试试。
修改Frigate配置文件：
detectors:
ov:
type: openvino
device: AUTO
model:
path: /openvino-model/ssdlite_mobilenet_v2.xml
model:
width: 300
height: 300
input_tensor: nhwc
input_pixel_format: bgr
labelmap_path: /openvino-model/coco_91cl_bkgr.txt
启动成功：
2024-03-12 15:28:42.137142043 [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO : Model Input Shape: [1,300,300,3] 2024-03-12 15:28:42.137157934 [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO : Model Output-0 Shape: [1,1,100,7] 2024-03-12 15:28:42.137176631 [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO : Model has 1 Output Tensors
使用CPU推理：
可见延迟有287ms。CPU占用非常高，FPS才1.4
使AUTO模式的OpenVINO推理：
可见延迟下降到80ms，CPU占用降低了很多，FPS达到了4
后续
由于上述OpenVINO处于AUTO模式，试着从AUTO改为GPU，发现启动失败了：
2024-03-12 15:50:55.499156863 Process detector:ov:
2024-03-12 15:50:55.502277448 Traceback (most recent call last):
2024-03-12 15:50:55.502450200 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-03-12 15:50:55.502457812 self.run()
2024-03-12 15:50:55.502467808 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-03-12 15:50:55.502478795 self._target(*self._args, **self._kwargs)
2024-03-12 15:50:55.502486642 File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
2024-03-12 15:50:55.502521519 object_detector = LocalObjectDetector(detector_config=detector_config)
2024-03-12 15:50:55.502526047 File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
2024-03-12 15:50:55.502530992 self.detect_api = create_detector(detector_config)
2024-03-12 15:50:55.502534675 File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2024-03-12 15:50:55.502540247 return api(detector_config)
2024-03-12 15:50:55.502545383 File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 32, in __init__
2024-03-12 15:50:55.502585844 self.interpreter = self.ov_core.compile_model(
2024-03-12 15:50:55.502634947 File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 399, in compile_model
2024-03-12 15:50:55.502646924 super().compile_model(model, device_name, {} if config is None else config),
2024-03-12 15:50:55.502661726 RuntimeError: Failed to create plugin /usr/local/lib/python3.9/dist-packages/openvino/libs/libopenvino_intel_gpu_plugin.so for device GPU
2024-03-12 15:50:55.502667511 Please, check your environment
2024-03-12 15:50:55.502671411 Check 'error_code == 0' failed at src/plugins/intel_gpu/src/runtime/ocl/ocl_device_detector.cpp:194:
2024-03-12 15:50:55.502677424 [GPU] No supported OCL devices found or unexpected error happened during devices query.
2024-03-12 15:50:55.502682322 [GPU] Please check OpenVINO documentation for GPU drivers setup guide.
2024-03-12 15:50:55.502686670 [GPU] clGetPlatformIDs error code: -1001
下次攒一个新点的板子来试试GPU模式的Detector。
日常使用
日常使用接入了2k的摄像头主码流，由于3代i5的GPU不支持H.265 HEVC硬解码，所以只能使用H.264编码。直接上数据
黑白静态时
全彩静态时
