ESXI下直通核显给Linux用于Frigate硬解视频和识别

ESXI下有一个Linux虚拟机,其中通过Docker跑了个Frigate作为家庭NVR,同时用于人脸识别给中控屏做人来亮屏。由于我的宿主机的核心是3代i5的3317u,比较羸弱,所以Frigate中ffmpeg软解视频比较吃力,TensorFlow识别物体也比较吃力,CPU使用率就飙上去了,所以考虑将GPU核显利用起来。本文测试使用的码流为TP摄像头创建的子码流:640*480的H.264

将GPU直通给Linux

首先设置GPU直通并重启ESXI,并给Linux虚拟机添加GPU为PCI设备

接着进入ESXI的ssh,让ESXI启动时不去获取显卡控制权

esxcli system settings kernel set -s vga -v FALSE

重启ESXI,此时即使机器插着显示器,ESXI启动也不输出画面了

接着配置Linux虚拟机:

禁用虚拟显卡:将svga.present由TRUE改为FALSE
让GPU驱动不知道在虚拟机中运行:添加hypervisor.cpuid.v0为FALSE

重启虚拟机,svga画面输出已经消失了。hypervisor.cpuid.v0一定要配置,否则Linux虚拟机无法完成启动。

启动后,通过ssh连接虚拟机,验证直通效果:sudo lshw -C display或者lspci -vnn | grep VGA -A 12

$ sudo lshw -C display
  *-display
       description: VGA compatible controller
       product: 3rd Gen Core processor Graphics Controller
       vendor: Intel Corporation
       physical id: 0
       bus info: pci@0000:1b:00.0
       version: 09
       width: 64 bits
       clock: 33MHz
       capabilities: msi pm vga_controller bus_master cap_list
       configuration: driver=i915 latency=64
       resources: irq:66 memory:fc800000-fcbfffff memory:d0000000-dfffffff ioport:2000(size=64)
$ lspci -vnn | grep VGA -A 12
1b:00.0 VGA compatible controller [0300]: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:0166] (rev 09) (prog-if 00 [VGA controller])
	DeviceName: pciPassthru0
	Subsystem: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:2010]
	Physical Slot: 256
	Flags: bus master, fast devsel, latency 64, IRQ 66
	Memory at fc800000 (64-bit, non-prefetchable) [size=4M]
	Memory at d0000000 (64-bit, prefetchable) [size=256M]
	I/O ports at 2000 [size=64]
	Capabilities: <access denied>
	Kernel driver in use: i915
	Kernel modules: i915

可见GPU直通成功了。

配置Frigate

https://docs.frigate.video/configuration/hardware_acceleration#configuring-intel-gpu-stats-in-docker

https://docs.frigate.video/configuration/hardware_acceleration#via-vaapi

主要就是需要在docker参数中加上–device /dev/dri/renderD128和–privileged,然后在Frigate配置文件中加上hwaccel_args: preset-vaapi

进入Frigate查看效果:

GPU硬件加速成功,但是intel_gpu_top执行失败,这个原因未知,网上很多人有类似的问题:

Frigate log:

2024-03-12 15:21:21.941076931  [2024-03-12 15:21:21] frigate.util.services          ERROR   : Unable to poll intel GPU stats: Failed to detect engines! (No such file or directory)
2024-03-12 15:21:21.941085403  (Kernel 4.16 or newer is required for i915 PMU support.)
2024-03-12 15:21:21.941089240  timeout: the monitored command dumped core

直接运行intel_gpu_top:

$ intel_gpu_top
Failed to initialize PMU! (Permission denied)

将Detectors交给GPU

需要使用openvino。openvino也支持CPU,效率比默认的CPU Detector高。从文档看,只支持6代以上的CPU。

https://docs.frigate.video/configuration/object_detectors#openvino-detector

https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html

用我的这个3代i5试试。

修改Frigate配置文件:

detectors:
  ov:
    type: openvino
    device: AUTO
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

启动成功:

2024-03-12 15:28:42.137142043  [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO    : Model Input Shape: [1,300,300,3]
2024-03-12 15:28:42.137157934  [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO    : Model Output-0 Shape: [1,1,100,7]
2024-03-12 15:28:42.137176631  [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO    : Model has 1 Output Tensors

使用CPU推理:

可见延迟有287ms。CPU占用非常高,FPS才1.4

使AUTO模式的OpenVINO推理:

可见延迟下降到80ms,CPU占用降低了很多,FPS达到了4

后续

由于上述OpenVINO处于AUTO模式,试着从AUTO改为GPU,发现启动失败了:

2024-03-12 15:50:55.499156863  Process detector:ov:
2024-03-12 15:50:55.502277448  Traceback (most recent call last):
2024-03-12 15:50:55.502450200    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-03-12 15:50:55.502457812      self.run()
2024-03-12 15:50:55.502467808    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-03-12 15:50:55.502478795      self._target(*self._args, **self._kwargs)
2024-03-12 15:50:55.502486642    File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
2024-03-12 15:50:55.502521519      object_detector = LocalObjectDetector(detector_config=detector_config)
2024-03-12 15:50:55.502526047    File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
2024-03-12 15:50:55.502530992      self.detect_api = create_detector(detector_config)
2024-03-12 15:50:55.502534675    File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2024-03-12 15:50:55.502540247      return api(detector_config)
2024-03-12 15:50:55.502545383    File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 32, in __init__
2024-03-12 15:50:55.502585844      self.interpreter = self.ov_core.compile_model(
2024-03-12 15:50:55.502634947    File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 399, in compile_model
2024-03-12 15:50:55.502646924      super().compile_model(model, device_name, {} if config is None else config),
2024-03-12 15:50:55.502661726  RuntimeError: Failed to create plugin /usr/local/lib/python3.9/dist-packages/openvino/libs/libopenvino_intel_gpu_plugin.so for device GPU
2024-03-12 15:50:55.502667511  Please, check your environment
2024-03-12 15:50:55.502671411  Check 'error_code == 0' failed at src/plugins/intel_gpu/src/runtime/ocl/ocl_device_detector.cpp:194:
2024-03-12 15:50:55.502677424  [GPU] No supported OCL devices found or unexpected error happened during devices query.
2024-03-12 15:50:55.502682322  [GPU] Please check OpenVINO documentation for GPU drivers setup guide.
2024-03-12 15:50:55.502686670  [GPU] clGetPlatformIDs error code: -1001

下次攒一个新点的板子来试试GPU模式的Detector。

日常使用

日常使用接入了2k的摄像头主码流,由于3代i5的GPU不支持H.265 HEVC硬解码,所以只能使用H.264编码。直接上数据

黑白静态时

全彩静态时

Share

You may also like...

发表回复

您的电子邮箱地址不会被公开。 必填项已用 * 标注