yolo v4 to TensorRt

參考: tensorrt_demos

$ cd TensorRT

$ git clone https://github.com/jkjung-avt/tensorrt_demos.git

$ cd tensorrt_demos

安裝環境

$ cd yolo

$ ./install_pycuda.sh

$ pip3 install onnx==1.4.1

$ cd ../plugins

$ make

下載 yolo 的 weights 和 cfg

$ cd ../yolo

$ ./download_yolo.sh

轉換 weights 到 trt

$ python3 yolo_to_onnx.py -m yolov4-tiny-416

$ python3 onnx_to_tensorrt.py -m yolov4-tiny-416

測試

$ cd ..

$ python trt_yolo.py --image doc/dog_trt_yolov4_416.jpg -m yolov4-tiny-416

使用 int8

$ cd yolo

$ ln -s yolov4-tiny-416.cfg yolov4-tiny-int8-416.cfg

$ ln -s yolov4-tiny-416.onnx yolov4-tiny-int8-416.onnx

$ mkdir calib_images

# and copy our image to calib_images

$ python3 onnx_to_tensorrt.py -v --int8 -m yolov4-tiny-int8-416

會產生如下錯誤

[03/02/2022-15:22:07] [TRT] [V] 001_convolutional + 001_convolutional_bn Set Tactic Name: sm70_xmma_fprop_implicit_gemm_f16f16_f16f16_f16_nhwckrsc_nhwc_tilesize128x256x32_stage1_warpsize2x4x1_g1_tensor8x8x4_t1r3s3 Tactic: 46202665595848747

[03/02/2022-15:22:07] [TRT] [V] Deleting timing cache: 2020 entries, 504 hits

[03/02/2022-15:22:07] [TRT] [E] 1: Unexpected exception

ERROR: failed to build the TensorRT engine!

$ vi onnx_to_tensorrt.py

將 from calibrator import YOLOEntropyCalibrator 移到程式開頭

Yolo tiny v4 to tensorflow and tflite

參考 tensorflow-yolov4-tflite

只能使用 tensorflow==2.3.0rc0

不要使用別的版本，也不要用 GPU

視情況修改 core/config.py

__C.YOLO.CLASSES

__C.YOLO.ANCHORS_TINY

for tensorflow format load by tf.saved_model.load()

$ python save_model.py --weights /your_path_to/weights/yolov4-tiny-vehicle-r_final.weights \

--output ./checkpoints/yolov4-tiny-416 \

--input_size 416 --model yolov4 --tiny

$ python convert_tflite.py --weights ./checkpoints/yolov4-tiny-416-tflite \

--output ./checkpoints/yolov4-tiny-416.tflite

for tensorflow tflite load  by tf.lite.Interpreter()

$ python save_model.py --weights /your_path_to/weights/yolov4-tiny-vehicle-r_final.weights \

--output ./checkpoints/yolov4-tiny-416-tflite \

--input_size 416 --model yolov4 --tiny --framework tflite

$ python convert_tflite.py --weights ./checkpoints/yolov4-tiny-416-tflite \

--output ./checkpoints/yolov4-tiny-416-fp16.tflite \

--quantize_mode float16

ubuntu install GTK3

$ sudo apt-get install libgtk-3-dev

$ sudo apt-get install libclutter-gst-3.0

deepstream with python

參考 DeepStream Python Apps

安裝說明 Deepstream python bindings

$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps.git

$ cd deepstream_python_apps/bindings/

$ git submodule update --init

$ cd ../3rdparty/gst-python/

$ ./autogen.sh

$ make

$ make  install

$ cd  ../../bindings/

$ mkdir build

$ cd build

$ cmake ..

$ make

$ cd ..

$ mkdir export_pyds

$ cp build/pyds*.whl export_pyds

$ pip3 install export_pyds/pyds-1.1.0-py3-none-linux_x86_64.whl

Install nvdec and nvh264enc as GStreamer plugins

$ sudo apt-get install gtk-doc-tools

$ git clone -b 1.14.5 https://github.com/GStreamer/gst-plugins-bad.git

download NVIDIA Video Codec SDK

$ unzip Video_Codec_SDK_11.1.5.zip

$ cp Video_Codec_SDK_11.1.5/Interface/nvEncodeAPI.h gst-plugins-bad/sys/nvenc/

$ cp Video_Codec_SDK_11.1.5/Interface/cuviddec.h gst-plugins-bad/sys/nvdec/

$ cp Video_Codec_SDK_11.1.5/Interface/nvcuvid.h gst-plugins-bad/sys/nvdec/

$ cd gst-plugins-bad/

$ ./autogen.sh --with-cuda-prefix="/usr/local/cuda"

$ cd sys/nvenc/

$ make

$ sudo cp .libs/libgstnvenc.so /usr/lib/x86_64-linux-gnu/gstreamer-1.0/

$ cd ../nvdec

$ make

$ sudo cp .libs/libgstnvdec.so /usr/lib/x86_64-linux-gnu/gstreamer-1.0/

$ gst-inspect-1.0 nvh264enc

$ gst-inspect-1.0 nvdec

Ubuntu 加入 RamDisk

$ sudo mkdir /mnt/ramdisk

$ sudo chmod 1777 /mnt/ramdisk

$ sudo vi /etc/fstab

tmpfs /mnt/ramdisk tmpfs rw,size=1G 0 0

$ sudo reboot

set frpc as service in Ubuntu

vi /etc/systemd/system/frpc.service

[Unit]

Description=frp client

Wants=network-online.target

After=network.target network-online.target

[Service]

ExecStart=/path_to/frpc -c /path_to/frpc.ini

[Install]

WantedBy=multi-user.target

$ sudo systemctl daemon-reload

$ sudo systemctl enable frpc

$ sudo systemctl start frpc

$ sudo systemctl status frpc

$ sudo systemctl staop frpc