Jetpack 6.0 更新 CUDA 12.2 至 12.6

本篇參考 安裝 Flux & ComfyUI 將其中的一些細節敘述清楚

參考 https://www.jetson-ai-lab.com/tutorial_comfyui_flux.html

參考 https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Installation.html#id10

$ sudo jtop

CUDA: 12.2.140

cuDNN: 8.9.4.25

TensorRT: 8.6.2.3

因為需要從 cuda-12.2 更新到 cuda-12.6

而在更新的過程中，會移除 Deepstream-7.0

所以需要自己安裝 Deepstream-7.1

記錄下目前的版本資料

$ dpkg -l>dekg_jp6.0.txt

$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh

$ chmod +x Miniconda3-latest-Linux-aarch64.sh

$ ./Miniconda3-latest-Linux-aarch64.sh

$ conda update conda

$ conda create -n comfyui python=3.10

$ conda init bash

$ cat .bashrc

$ conda activate comfyui

$ conda info --envs

$ conda deactivate

$ conda activate comfyui

$ conda list

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/arm64/cuda-keyring_1.1-1_all.deb

$ sudo dpkg -i cuda-keyring_1.1-1_all.deb

$ sudo apt-get update

$ sudo apt-get -y install cuda-toolkit-12-6 cuda-compat-12-6

$ update-alternatives --list cuda

$ update-alternatives --display cuda

$ sudo update-alternatives --config cuda

cuda-12.6

$ apt list -a cudnn

$ sudo apt-get install cudnn=9.10.2-1

$ apt list -a python3-libnvinfer

$ sudo apt-get install python3-libnvinfer=10.7.0.23+cuda12.6

The following packages will be REMOVED:

  deepstream-7.0 libnvparsers-dev nvidia-tensorrt-dev

$ sudo apt-get install python3-libnvinfer-dev=10.7.0.23-1+cuda12.6

$ sudo jtop

CUDA: 12.6.85

cuDNN: 9.10.2

TensorRT: 10.7.0.23

$ wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/org/nvidia/deepstream/7.1/files?redirect=true&path=deepstream-7.1_7.1.0-1_arm64.deb' -O deepstream-7.1_7.1.0-1_arm64.deb

$ sudo apt-get install ./deepstream-7.1_7.1.0-1_arm64.deb

$ export BNB_CUDA_VERSION=126

$ export LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64:$LD_LIBRARY_PATH

$ git clone https://github.com/timdettmers/bitsandbytes.git

$ cd bitsandbytes

$ pip uninstall numpy

$ pip install "numpy<2.0"

$ mkdir -p build

$ cd build

$ cmake .. -DCOMPUTE_BACKEND=cuda -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-12.6

$ make -j$(nproc)

$ cd ..

$ python setup.py install

$ pip install http://jetson.webredirect.org/jp6/cu124/+f/5fe/ee5f5d1a75229/torch-2.3.0-cp310-cp310-linux_aarch64.whl

$ pip install http://jetson.webredirect.org/jp6/cu124/+f/988/cb71323efff87/torchvision-0.18.0a0+6043bc2-cp310-cp310-linux_aarch64.whl

$ pip install http://jetson.webredirect.org/jp6/cu124/+f/0aa/a066463c02b4a/torchaudio-2.3.0+952ea74-cp310-cp310-linux_aarch64.whl

$ python3

>>> import bitsandbytes as bnb

>>> print(bnb.__version__)

$ git clone https://github.com/comfyanonymous/ComfyUI.git

$ cd ComfyUI

$ pip install -r requirements.txt

$ cd custom_nodes

$ git clone https://github.com/ltdrdata/ComfyUI-Manager.git

$ cd ..

$ mkdir workflows

$ cd workflows

$ wget https://www.jetson-ai-lab.com/assets/workflow_agx_orin_4steps.json

$ cd ..

到 https://huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main

下載 flux1-schnell.safetensors 放到 models/unet

下載 ae.safetensors 放到 models/vae/FLUX1

到 https://huggingface.co/stabilityai/stable-diffusion-3-medium/tree/main/text_encoders

下載 clip_l.safetensors 和 t5xxl_fp8_e4m3fn.safetensors 放到 models/clip

$ python main.py --port=8080

在 http://127.0.0.1:8080/ 網頁，按 Queue Prompt, 等待...

Load workflow_agx_orin_4steps.json 時，需修改 "Load VAE" 的 vae_name 到 FLUX1/ae.safetensors

記錄下目前的版本資料

$ dpkg -l>dpkg_jp6.0_comfyui.txt

若需要將 cuda-12.6 還回 cuda-12.2

參考記錄下來的版本資料 dpkg_jp6.0.txt dpkg_jp6.0_comfyui.txt

參考 https://repo.download.nvidia.com/jetson 下載 package

移除不必要的 package

$ sudo apt-get remove --purge package

安裝 package 時，常出現相依性錯誤，所以有時要變更安裝順序

甚至最後重新安裝一遍，確保全部安裝

$ cd jetpack_6.0

$ ./download.sh

$ ./rollback.sh

$ ./install.sh

參考 https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-cuda/tags

$ xhost +

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  nvcr.io/nvidia/l4t-cuda:11.4.19-runtime

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  nvcr.io/nvidia/l4t-cuda:12.2.12-runtime

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  nvcr.io/nvidia/l4t-cuda:12.6.11-runtime

docker: Error response from daemon: failed to create task for container: 

failed to create shim task: OCI runtime create failed: 

failed to create NVIDIA Container Runtime: failed to construct OCI spec modifier: 

requirements not met: unsatisfied condition: cuda>=12.6 (cuda=12.2): unknown.

因為 Jetpack 6.0 自帶 CUDA 12.2

使用的 docker image 不能比 CUDA 12.2 高 

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  -w /opt/nvidia/deepstream/deepstream-7.0 \

  nvcr.io/nvidia/deepstream-l4t:7.0-samples-multiarch

因為 Jetpack 6.0 自帶 CUDA 12.2

使用的 docker image 不能比 CUDA 12.2 高 

所以也無法使用 Deepstream 7.1

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  -w /opt/nvidia/deepstream/deepstream-7.1 \

  nvcr.io/nvidia/deepstream-l4t:7.1-samples-multiarch

deepstream create_pipeline.create_common_elements 分析

參考之前 https://yingrenn.blogspot.com/2023/03/deepstream-createpipeline.html

create_common_elements {

    common_elements.segvisual_bin = create_segvisual_bin

    common_elements.secondary_gie_bin = create_secondary_gie_bin {

        bin.sink - tee - queue - bin.src

        bin->sub_bins[] = create_secondary_gie {

            secondary_gie = nvinfer | nvinferserver

            if (is_parent || has_children) {

                queue

            }

            if (no_children) {

                queue - secondary_gie - fakesink

            } else {

                queue - secondary_gie - tee

            }

        }

        if (parent_index == -1) {

            tee - sub_bins[i].queue

        } else if (sub_bins[parent_index].tee) {

            sub_bins[parent_index].tee - sub_bins[i].queue

        } else {

            sub_bins[parent_index].secondary_gie - sub_bins[i].secondary_gie

        }

    }

    common_elements.secondary_preprocess_bin = create_secondary_preprocess_bin

    common_elements.dsanalytics_bin = create_dsanalytics_bin

    common_elements.tracker_bin = create_tracking_bin

    common_elements.primary_gie_bin = create_primary_gie_bin

    common_elements.preprocess_bin = create_preprocess_bin

    common_elements.msg_conv = gst_element_factory_make

    common_elements.tee = gst_element_factory_make

    

    sink_elem - preprocess_bin - primary_gie_bin - tracker_bin - dsanalytics_bin - secondary_preprocess_bin - secondary_gie_bin - segvisual_bin - msg_conv - tee - src_elem

}

  

安裝 Ubuntu 20.04

$ sudo apt-get update

$ sudo apt-get upgrade

$ sudo apt-get install ssh

$ sudo vi /etc/fstab

#中間的空格要使用 tab

ip:/share_folder /mnt/mount_folder nfs defaults,bg 0 0

$ cd /mnt

$ sudo mkdir QNAP_A QNAP_B

$ sudo mount -a

$ mkdir -p ~/.config/autostart

$ cp /usr/share/applications/vino-server.desktop ~/.config/autostart/

$ gsettings set org.gnome.Vino prompt-enabled false

$ gsettings set org.gnome.Vino require-encryption false

$ gsettings set org.gnome.Vino authentication-methods "['vnc']"

$ gsettings set org.gnome.Vino vnc-password $(echo -n 'ChangeToYourPasswd'|base64)

$ sudo vi /etc/gdm3/custom.conf

WaylandEnable=false

AutomaticLoginEnable = true

AutomaticLogin = UserLoginName

$ vi vino.sh

DISP=`ps -u $(id -u) -o pid= | \

    while read pid; do

        cat /proc/$pid/environ 2>/dev/null | tr '\0' '\n' | grep '^DISPLAY=:'

    done | grep -o ':[0-9]*' | sort -u`

echo $DISP

/usr/lib/vino/vino-server --display=$DISP

$ chmod +x vino.sh

依據 使用最新版本的 driver

CUDA Toolkit and Corresponding Driver Versions

https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html

dGPU Setup for Ubuntu

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Quickstart.html

Ubuntu 20.04

GStreamer 1.16.3

NVIDIA driver 525.125.06

CUDA 12.1

TensorRT 8.5.3.1

$ sudo ubuntu-drivers devices

$ sudo apt-get install nvidia-driver-535

$ sudo reboot

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.1-1_all.deb

$ sudo dpkg -i cuda-keyring_1.1-1_all.deb

$ sudo apt-get update

$ sudo apt-get -y install cuda-12-2

$ sudo apt-get -y install cuda-12-1

安裝 cuDNN

參考 https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

到 2.2. Downloading cuDNN for Linux(https://developer.nvidia.com/cudnn)

下載 Local Install for Ubuntu18.04 x86_64(Deb)

$ sudo apt-get install zlib1g

$ sudo dpkg -i cudnn-local-repo-ubuntu2004-8.9.5.29_1.0-1_amd64.deb

sudo cp /var/cudnn-local-repo-ubuntu2004-8.9.5.29/cudnn-local-98C06E99-keyring.gpg /usr/share/keyrings/

$ sudo apt-get update

$ apt list -a libcudnn8

$ sudo apt-get install libcudnn8=8.9.5.29-1+cuda12.2

$ sudo apt-get install libcudnn8-dev=8.9.5.29-1+cuda12.2

$ sudo apt-get install libcudnn8-samples=8.9.5.29-1+cuda12.2

$ update-alternatives --display libcudnn

$ cp -r /usr/src/cudnn_samples_v8/ .

$ cd cudnn_samples_v8/mnistCUDNN/

$ sudo apt-get install libfreeimage3 libfreeimage-dev

$ make clean && make

$ ./mnistCUDNN

...

Test passed!

安裝 TensorRT 8.6.1

https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-861/install-guide/index.html

$ sudo apt-get install python3-pip

$ sudo apt-get install python3.8.venv

$ python3 -m venv envs/tensorrt

$ source envs/tensorrt/bin/activate

$ pip3 install --upgrade pip

$ python3 -m pip install --extra-index-url https://pypi.nvidia.com tensorrt_libs

$ python3 -m pip install --extra-index-url https://pypi.nvidia.com tensorrt_bindings

$ python3 -m pip install --upgrade tensorrt

$ python3 -m pip install --upgrade tensorrt_lean

$ python3 -m pip install --upgrade tensorrt_dispatch

測試  TensorRT Python

$ python3

>>> import tensorrt

>>> print(tensorrt.__version__)

>>> assert tensorrt.Builder(tensorrt.Logger())

>>> import tensorrt_lean as trt

>>> print(trt.__version__)

>>> assert trt.Builder(trt.Logger())

>>> import tensorrt_dispatch as trt

>>> print(trt.__version__)

>>> assert trt.Builder(trt.Logger())

連結 https://developer.nvidia.com/tensorrt 按 GET STARTED

連結 https://developer.nvidia.com/tensorrt-getting-started 按 DOWNLOAD NOW

選擇 TensorRT 8

選擇 TensorRT 8.6 GA

TensorRT 8.6 GA for Ubuntu 20.04 and CUDA 12.0 and 12.1 DEB local repo Package

$ sudo dpkg -i nv-tensorrt-local-repo-ubuntu2004-8.6.1-cuda-12.0_1.0-1_amd64.deb

$ sudo cp /var/nv-tensorrt-local-repo-ubuntu2004-8.6.1-cuda-12.0/nv-tensorrt-local-9A1EDFBA-keyring.gpg /usr/share/keyrings/

$ sudo apt-get update

$ sudo apt-get install tensorrt

$ sudo apt-get install libnvinfer-lean8

$ sudo apt-get install libnvinfer-vc-plugin8

$ sudo apt-get install python3-libnvinfer-lean

$ sudo apt-get install python3-libnvinfer-dispatch

$ python3 -m pip install numpy

$ sudo apt-get install python3-libnvinfer-dev

$ python3 -m pip install protobuf

$ sudo apt-get install uff-converter-tf

$ python3 -m pip install numpy onnx

$ sudo apt-get install onnx-graphsurgeon

確認安裝

$ dpkg-query -W tensorrt

tensorrt        8.6.1.6-1+cuda12.0

安裝 DeepStream

$ sudo apt-get install libssl1.1

$ sudo apt-get install libgstreamer1.0-0

$ sudo apt-get install gstreamer1.0-tools

$ sudo apt-get install gstreamer1.0-plugins-good

$ sudo apt-get install gstreamer1.0-plugins-bad

$ sudo apt-get install gstreamer1.0-plugins-ugly

$ sudo apt-get install gstreamer1.0-libav

$ sudo apt-get install libgstreamer-plugins-base1.0-dev

$ sudo apt-get install libgstrtspserver-1.0-0

$ sudo apt-get install libjansson4

$ sudo apt-get install libyaml-cpp-dev

$ sudo apt-get install libjsoncpp-dev

$ sudo apt-get install protobuf-compiler

$ sudo apt-get install gcc

$ sudo apt-get install make

$ sudo apt-get install git

$ sudo apt-get install python3

$ git clone https://github.com/edenhill/librdkafka.git

$ cd librdkafka

$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a

$ ./configure

$ make

$ sudo make install

$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-6.3/lib

$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-6.3/lib

https://catalog.ngc.nvidia.com/orgs/nvidia/resources/deepstream

下載 deepstream-6.3_6.3.0-1_arm64.deb

$ wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/nvidia/deepstream/versions/6.3/files/deepstream-6.3_6.3.0-1_amd64.deb'

$ sudo apt-get install ./deepstream-6.3_6.3.0-1_amd64.deb

$ cd /opt/nvidia/deepstream/deepstream-6.3/samples/configs/deepstream-app

$ deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

安裝 Docker

https://docs.docker.com/engine/install/ubuntu/

$ sudo apt-get update

$ sudo apt-get install ca-certificates curl gnupg

$ sudo install -m 0755 -d /etc/apt/keyrings

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

$ sudo chmod a+r /etc/apt/keyrings/docker.gpg

$ echo \

  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \

  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \

  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

$ sudo apt-get update

$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

$ sudo docker run --rm hello-world

安裝 NVIDIA Container Toolkit

參考 https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \

  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \

    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \

    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list \

  && \

    sudo apt-get update

$ sudo apt-get install -y nvidia-container-toolkit

$ sudo nvidia-ctk runtime configure --runtime=docker

$ sudo systemctl restart docker

$ sudo groupadd docker

$ sudo usermod -a -G docker $USER

$ docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

安裝 NGC CLI

參考 https://ngc.nvidia.com/setup/installers/cli

$ wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/3.30.1/files/ngccli_linux.zip -O ngccli_linux.zip && unzip ngccli_linux.zip

$ find ngc-cli/ -type f -exec md5sum {} + | LC_ALL=C sort | md5sum -c ngc-cli.md5

$ sha256sum ngccli_linux.zip

$ chmod u+x ngc-cli/ngc

$ echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile && source ~/.bash_profile

$ ngc config set

# 直接 enter 即可

$ docker login nvcr.io

Username: $oauthtoken

Password: <Your API Key>

用 Docker 開發 DeepStream 6.3

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_docker_containers.html

$ sudo docker pull nvcr.io/nvidia/deepstream:6.3-gc-triton-devel

$ export DISPLAY=:0

$ xhost +

$ docker run -it --rm --net=host --gpus all -e DISPLAY=$DISPLAY --device /dev/snd -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/deepstream:6.3-gc-triton-devel

# cd samples/configs/deepstream-app

# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

# exit

$ sudo docker ps -a

$ sudo docker stop container_id

$ sudo docker rm container_id

$ sudo docker image list

$ sudo docker image rm image_id

DeepStream 之 nvinfer(primary mode) 執行 classifier, 使用在 ROI 上

試了很久, 無法利用 nvdspreprocess 的 ROI 放在 nvinfer 之前

參考 How to run classifier in primary mode only in specific ROI? 

發現直接使用 nvvideoconvert 的 ROI 可以正常運作

參數設定 src-crop="left:top:width:height", 如

g_object_set(G_OBJECT(pre_proc), "src-crop", "50:0:320:240", NULL);

YOLOv8 and TensorRT

參考 YOLOv8 GitHub 官網

參考 YOLOv8 文件

參考 DeepStream-Yolo

1. 下載 DeepStream-Yolo, Ultralytics YOLOv8

git clone https://github.com/marcoslucianops/DeepStream-Yolo.git

git clone https://github.com/ultralytics/ultralytics.git /mnt/Data/DeepStream/DeepStream-Yolo/ultralytics

2. 建立 deepstream_yolo docker container

docker_run.sh

xhost +

docker run --name='deepstream_yolo' --gpus all -it --net=host --privileged \

  -v /tmp/.X11-unix:/tmp/.X11-unix \

  -v /etc/localtime:/etc/localtime \

  -v /mnt/Data/DeepStream/DeepStream-Yolo/DeepStream-Yolo:/home/DeepStream-Yolo \

  -v /mnt/Data/DeepStream/DeepStream-Yolo/ultralytics:/home/ultralytics \

  -v /mnt/Data/DeepStream/DeepStream-Yolo/read_me:/home/read_me \

  -v /mnt/Data/DeepStream/DeepStream-Yolo/datasets:/home/datasets \

  -v /mnt/CT1000SSD/ImageData/Light:/home/Light \

  -e DISPLAY=$DISPLAY \

  -w /home/read_me \

  nvcr.io/nvidia/deepstream:6.2-devel

  

3. 在 Docker 內, 安裝 DeepStream-Yolo

apt-get install build-essential

/opt/nvidia/deepstream/deepstream/user_additional_install.sh

cd /home/DeepStream-Yolo

CUDA_VER=11.8 make -C nvdsinfer_custom_impl_Yolo

4. 在 Docker 內, 安裝 Ultralytics YOLOv8

#python3 -m pip install --upgrade pip

pip3 install --upgrade pip

pip3 install protobuf numpy

cd /home/ultralytics

#pip install -e .

pip3 install -r requirements.txt

python3 setup.py install

pip3 install onnx onnxsim onnxruntime

5. 在 Docker 內, 下載，轉換，測試 yolov8s.pt, yolov8s.pt 模型

cd /home/ultralytics

cp /home/DeepStream-Yolo/utils/export_yoloV8.py /home/ultralytics

wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s.pt

wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt

python3 export_yoloV8.py -w yolov8s.pt --dynamic

python3 export_yoloV8.py -w yolov8n.pt --dynamic

cp yolov8s.onnx labels.txt /home/DeepStream-Yolo

cp yolov8n.onnx labels.txt /home/DeepStream-Yolo

6. 移除 deepstream_yolo container

$ docker container rm deepstream_yolo

7. 重新進入 Docker

docker_attach.sh

xhost +

docker start deepstream_yolo

docker attach deepstream_yolo

8. 轉換模型格式為 onnx

yolov8n.py

from ultralytics import YOLO

# Load a model

model = YOLO("yolov8n.yaml")  # build a new model from scratch

model = YOLO("yolov8n.pt")  # load a pretrained model (recommended for training)

# Use the model

model.train(data="coco128.yaml", epochs=3)  # train the model

metrics = model.val()  # evaluate model performance on the validation set

results = model("https://ultralytics.com/images/bus.jpg")  # predict on an image

path = model.export(format="onnx")  # export the model to ONNX format

執行 python3 yolov8n.py 出現下列錯誤

ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).

修正方式

$ sudo systemctl stop docker

取得 container id

$ docker inspect deepstream_yolo | grep Id

"Id": "???????"

編輯 container 的 ShmSize

$ sudo vi /var/lib/docker/containers/your_container_id/hostconfig.json

"ShmSize":8589934592

$ sudo systemctl restart docker

$ ./docker_attach.sh

9. 在 DeepStream 中測試 onnx 模型

# cd /home/DeepStream-Yolo

# vi config_infer_primary_yoloV8.txt

onnx-file=yolov8s.onnx

onnx-file=yolov8n.onnx

# vi deepstream_app_config.txt

uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4

uri=rtsp://root:A1234567@192.168.0.107:554/live1s1.sdp

live-source=0

live-source=1

config-file=config_infer_primary.txt

config-file=config_infer_primary_yoloV8.txt

file-loop=0

file-loop=1

# deepstream-app -c deepstream_app_config.txt

10. 準備自己的圖形資料, PASCAL VOC(LabelImg 產生的 xml) 格式轉換成 txt

參考 Ultralytics 文件之 VOC Dataset 說明

參考 範例程式

prepare_detect.py

import cv2

import os

import random

import re

import xml.etree.ElementTree as ET

import numpy as np

LIGHT_CLASSES_LIST = [

    'forward_right',

    'others',

    'red',

    'red_left',

    'yellow',

    ]

        

def save_false_positives(img_org, iName, xName, tag, classIdx, 

        clip_x0, clip_y0, clip_x1, clip_y1):

    img_new = img_org[clip_y0:clip_y1, clip_x0:clip_x1]

    fPath, fName = os.path.split(iName)

    fName, fExt = os.path.splitext(fName)

    fName = fName + tag + fExt

    rndPaths = ['train', 'val', 'test']

    rndPath = random.choices(rndPaths, weights=(8,1,1))[0]

    iName = os.path.join('/home/datasets/Light/images', rndPath, fName)

    cv2.imwrite(iName, img_new)

        

def convert_box(size, box):

    dw, dh = 1. / size[0], 1. / size[1]

    x, y, w, h = (box[0] + box[1]) / 2.0 - 1, (box[2] + box[3]) / 2.0 - 1, box[1] - box[0], box[3] - box[2]

    return x * dw, y * dh, w * dw, h * dh

          

def save_file(img_org, iName, xName, tag, classIdx, 

        p0x, p0y, p1x, p1y, p2x, p2y, p3x, p3y, 

        img_w, img_h, xmin, ymin, xmax, ymax,

        clip_x0, clip_y0, clip_x1, clip_y1):

    img_new = img_org[clip_y0:clip_y1, clip_x0:clip_x1]

    fPath, fName = os.path.split(iName)

    fName, fExt = os.path.splitext(fName)

    fName = fName + tag + fExt

    rndPaths = ['train', 'val', 'test']

    rndPath = random.choices(rndPaths, weights=(8,1,1))[0]

    iName = os.path.join('/home/datasets/Light/images', rndPath, fName)

    cv2.imwrite(iName, img_new)

    

    w = clip_x1 - clip_x0

    h = clip_y1 - clip_y0

    xmin = xmin - clip_x0

    ymin = ymin - clip_y0

    xmax = xmax - clip_x0

    ymax = ymax - clip_y0

    bb = convert_box((w, h), (xmin, xmax, ymin, ymax))

    fPath, fName = os.path.split(xName)

    fName, fExt = os.path.splitext(fName)

    fName = fName + tag + '.txt'

    tName = os.path.join('/home/datasets/Light/labels', rndPath, fName)

    with open(tName, 'w') as f:

        f.write(" ".join([str(a) for a in (classIdx, *bb)]) + '\n')

        

def gen_img_yolo(iName, xName):

    tree = ET.parse(open(xName))

    root = tree.getroot()

    img_w = int(root.find('size').find('width').text)

    img_h = int(root.find('size').find('height').text)

    for idx, object in enumerate(root.findall('object')):

        name = object.find('name').text

        classIdx = LIGHT_CLASSES_LIST.index(name)

        #print(classIdx, name)

        bndbox = object.find('bndbox')

        p0x = int(bndbox.find('p0x').text)

        p0y = int(bndbox.find('p0y').text)

        p1x = int(bndbox.find('p1x').text)

        p1y = int(bndbox.find('p1y').text)

        p2x = int(bndbox.find('p2x').text)

        p2y = int(bndbox.find('p2y').text)

        p3x = int(bndbox.find('p3x').text)

        p3y = int(bndbox.find('p3y').text)

        xmin = int(bndbox.find('xmin').text)

        ymin = int(bndbox.find('ymin').text)

        xmax = int(bndbox.find('xmax').text)

        ymax = int(bndbox.find('ymax').text)

        if xmin != p0x or xmin != p3x or ymin != p0y or ymin != p1y or \

                xmax != p1x or xmax != p2x or ymax != p2y or ymax != p3y:

            print('error:bndbox', xName)

            exit()

        if idx > 0:

            print('error:object', xName)

            exit()

    img_org = cv2.imread(iName)

    if img_org.shape[0] != img_h or img_org.shape[1] != img_w:

        print(img_org.shape, (img_h, img_w))

        exit()

    img = np.copy(img_org)

    clip_x0 = random.randrange(0, int(xmin*0.5))

    clip_y0 = random.randrange(0, int(ymin*0.5))

    clip_x1 = random.randrange(int(xmax + (img_w-xmax)*0.5), img_w+1)

    clip_y1 = random.randrange(int(ymax + (img_h-ymax)*0.5), img_h+1)

    save_file(img_org, iName, xName, '', classIdx, 

            p0x, p0y, p1x, p1y, p2x, p2y, p3x, p3y, 

            img_w, img_h, xmin, ymin, xmax, ymax,

            clip_x0, clip_y0, clip_x1, clip_y1)

    ratio = (xmax - xmin) / img_w

    if ratio < 0.3:

        clip_x0 = random.randrange(int(xmin*0.3), int(xmin*0.8))

        clip_y0 = random.randrange(int(ymin*0.3), int(ymin*0.8))

        clip_x1 = random.randrange(int(xmax + (img_w-xmax)*0.2), int(xmax + (img_w-xmax)*0.7))

        clip_y1 = random.randrange(int(ymax + (img_h-ymax)*0.2), int(ymax + (img_h-ymax)*0.7))

        save_file(img_org, iName, xName, '_a', classIdx, 

                p0x, p0y, p1x, p1y, p2x, p2y, p3x, p3y, 

                img_w, img_h, xmin, ymin, xmax, ymax,

                clip_x0, clip_y0, clip_x1, clip_y1)

        clip_x0 = random.randrange(int(xmin*0.5), int(xmin*0.9))

        clip_y0 = random.randrange(int(ymin*0.5), int(ymin*0.9))

        clip_x1 = random.randrange(int(xmax + (img_w-xmax)*0.1), int(xmax + (img_w-xmax)*0.5))

        clip_y1 = random.randrange(int(ymax + (img_h-ymax)*0.1), int(ymax + (img_h-ymax)*0.5))

        save_file(img_org, iName, xName, '_b', classIdx, 

                p0x, p0y, p1x, p1y, p2x, p2y, p3x, p3y, 

                img_w, img_h, xmin, ymin, xmax, ymax,

                clip_x0, clip_y0, clip_x1, clip_y1)

        if xmin > (img_w - xmax):

            if ymin > (img_h - ymax):

                clip_x0 = random.randrange(0, int(xmin*0.8))

                clip_y0 = random.randrange(0, int(ymin*0.8))

                clip_x1 = random.randrange(int(xmin), int(xmin+(xmax-xmin)*0.8))

                clip_y1 = random.randrange(int(ymin), int(ymin+(ymax-ymin)*0.8))

                root.remove(object)

                save_false_positives(img_org, iName, xName, '_f0', classIdx, 

                        clip_x0, clip_y0, clip_x1, clip_y1)

            else:

                clip_x0 = random.randrange(0, int(xmin*0.8))

                clip_y0 = random.randrange(int(ymin+(ymax-ymin)*0.2), int(ymax))

                clip_x1 = random.randrange(int(xmin), int(xmin + (xmax-xmin)*0.8))

                clip_y1 = random.randrange(int(ymax+(img_h-ymax)*0.2), img_h)

                root.remove(object)

                save_false_positives(img_org, iName, xName, '_f1', classIdx, 

                        clip_x0, clip_y0, clip_x1, clip_y1)

        else:

            if ymin > (img_h - ymax):

                clip_x0 = random.randrange(int(xmin+(xmax-xmin)*0.2), int(xmax))

                clip_y0 = random.randrange(0, int(ymin*0.8))

                clip_x1 = random.randrange(int(xmax + (img_w-xmax)*0.2), img_w)

                clip_y1 = random.randrange(int(ymin), int(ymin+(ymax-ymin)*0.8))

                root.remove(object)

                save_false_positives(img_org, iName, xName, '_f2', classIdx, 

                        clip_x0, clip_y0, clip_x1, clip_y1)

            else:

                clip_x0 = random.randrange(int(xmin+(xmax-xmin)*0.2), int(xmax))

                clip_y0 = random.randrange(int(ymin+(ymax-ymin)*0.2), int(ymax))

                clip_x1 = random.randrange(int(xmax + (img_w-xmax)*0.2), img_w)

                clip_y1 = random.randrange(int(ymax+(img_h-ymax)*0.2), img_h)

                root.remove(object)

                save_false_positives(img_org, iName, xName, '_f3', classIdx, 

                        clip_x0, clip_y0, clip_x1, clip_y1)

    elif ratio < 0.7:

        clip_x0 = random.randrange(int(xmin*0.1), int(xmin*0.7))

        clip_y0 = random.randrange(int(ymin*0.1), int(ymin*0.7))

        clip_x1 = random.randrange(int(xmax + (img_w-xmax)*0.3), int(xmax + (img_w-xmax)*0.9))

        clip_y1 = random.randrange(int(ymax + (img_h-ymax)*0.3), int(ymax + (img_h-ymax)*0.9))

        save_file(img_org, iName, xName, '_c', classIdx, 

                p0x, p0y, p1x, p1y, p2x, p2y, p3x, p3y, 

                img_w, img_h, xmin, ymin, xmax, ymax,

                clip_x0, clip_y0, clip_x1, clip_y1)

        if xmin > (img_w - xmax):

            if ymin > (img_h - ymax):

                clip_x0 = random.randrange(0, int(xmin*0.8))

                clip_y0 = random.randrange(0, int(ymin*0.8))

                clip_x1 = random.randrange(int(xmin), int(xmin+(xmax-xmin)*0.8))

                clip_y1 = random.randrange(int(ymin), int(ymin+(ymax-ymin)*0.8))

                root.remove(object)

                save_false_positives(img_org, iName, xName, '_f4', classIdx, 

                        clip_x0, clip_y0, clip_x1, clip_y1)

            else:

                clip_x0 = random.randrange(0, int(xmin*0.8))

                clip_y0 = random.randrange(int(ymin+(ymax-ymin)*0.2), int(ymax))

                clip_x1 = random.randrange(int(xmin), int(xmin + (xmax-xmin)*0.8))

                clip_y1 = random.randrange(int(ymax+(img_h-ymax)*0.2), img_h)

                root.remove(object)

                save_false_positives(img_org, iName, xName, '_f5', classIdx, 

                        clip_x0, clip_y0, clip_x1, clip_y1)

        else:

            if ymin > (img_h - ymax):

                clip_x0 = random.randrange(int(xmin+(xmax-xmin)*0.2), int(xmax))

                clip_y0 = random.randrange(0, int(ymin*0.8))

                clip_x1 = random.randrange(int(xmax + (img_w-xmax)*0.2), img_w)

                clip_y1 = random.randrange(int(ymin), int(ymin+(ymax-ymin)*0.8))

                root.remove(object)

                save_false_positives(img_org, iName, xName, '_f6', classIdx, 

                        clip_x0, clip_y0, clip_x1, clip_y1)

            else:

                clip_x0 = random.randrange(int(xmin+(xmax-xmin)*0.2), int(xmax))

                clip_y0 = random.randrange(int(ymin+(ymax-ymin)*0.2), int(ymax))

                clip_x1 = random.randrange(int(xmax + (img_w-xmax)*0.2), img_w)

                clip_y1 = random.randrange(int(ymax+(img_h-ymax)*0.2), img_h)

                root.remove(object)

                save_false_positives(img_org, iName, xName, '_f7', classIdx, 

                        clip_x0, clip_y0, clip_x1, clip_y1)

    elif ratio < 1.0:

        pass

    return

def recursive_folder(path):

    files = os.listdir(path)

    files.sort()

    for file in files:

        fullName = os.path.join(path, file)

        if os.path.isfile(fullName):

            fPath, fName = os.path.split(fullName)

            fName, fExt = os.path.splitext(fName)

            if fExt in ('.jpg'):

                xPath = fPath + '.xml'

                xName = fName + '.xml'

                xFName = os.path.join(xPath, xName)

                if os.path.isfile(xFName):

                    gen_img_yolo(fullName, xFName)

                else:

                    print(xFName)

        else:

            recursive_folder(fullName)

def main():

    recursive_folder('/home/Light')

if __name__ == '__main__':

    main()

11. 訓練自己的模型

from ultralytics import YOLO

# Load a model

model = YOLO('yolov8n.pt')  # load a pretrained model (recommended for training)

# Train the model

model.train(data='VOC.yaml', epochs=100, imgsz=640)

12. 查詢 onnx 模型的輸出輸入層

import onnx

model = onnx.load('yolov8n.onnx')

g_in = model.graph.input

g_out = model.graph.output

DeepStream 之 nvdspreprocess 使用注意事項

nvdspreprocess 使用在 PGIE(Primary Gst Inference Engine)

g_object_set(G_OBJECT(pgie), "input-tensor-meta", 1, NULL);

or PGIE 的設定檔

input-tensor-from-meta=1

nvdspreprocess 的設定檔

# 需搭配 pgie_config.txt 的 model

# 0=NCHW, 1=NHWC, 2=CUSTOM

network-input-order=0

network-input-shape=2;3;384;1248

processing-width=1248

processing-height=384

tensor-name=Input

# 需搭配 pgie_config.txt 的 net-scale-factor

pixel-normalization-factor=1.0

Nvidia TAO Computer Vision Sample Workflows

參考: Nvidia TAO(Train, Adapt, and Optimize)

user@host:~$ pip3 install --upgrade pip

Traceback (most recent call last):

  File "/home/user/.local/bin/pip3", line 7, in <module>

    from pip._internal.cli.main import main

ModuleNotFoundError: No module named 'pip._internal'

user@host:~$ python3 -m pip --version

pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)

user@host:~$ python3 -m pip install --upgrade pip

Collecting pip

  Cache entry deserialization failed, entry ignored

  Using cached https://files.pythonhosted.org/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl

Installing collected packages: pip

Successfully installed pip-21.3.1

user@host:~$ pip3 install virtualenv

user@host:~$ pip3 install virtualenvwrapper

user@host:~$ mkdir .virtualenvs

user@host:~$ vi .bashrc

export WORKON_HOME=$HOME/.virtualenvs

export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3

source $HOME/.local/bin/virtualenvwrapper.sh

user@host:~$ source .bashrc

user@host:~$ mkvirtualenv tao -p /usr/bin/python3

(tao) user@host:~$ deactivate

user@host:~$ lsvirtualenv

user@host:~$ workon tao

(tao) user@host:~$ pip3 install nvidia-pyindex

(tao) user@host:~$ pip3 install nvidia-tao

(tao) user@host:~$ pip3 install jupyter

(tao) user@host:~$ tao info

到 https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/resources/cv_samples 網站

右上角 Download 選 WGET 或 CLI, 會將命令拷貝至剪貼簿，如下命令，並執行

ngc registry resource download-version "nvidia/tao/cv_samples:v1.4.1"

wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/cv_samples/versions/v1.4.1/zip -O cv_samples_v1.4.1.zip

(tao) user@host:~/Data/tao$ unzip -u cv_samples_v1.4.1.zip -d ./cv_samples_v1.4.1 && rm -rf cv_samples_v1.4.1.zip 

user@host:~/Data/tao$ cp -ar cv_samples_v1.4.1/yolo_v4_tiny/ yolo_v4_tiny_1.4.1

(tao) user@host:~/Data/tao$ cd cv_samples_v1.4.1/

(tao) user@host:~/Data/tao/cv_samples_v1.4.1$ 

(tao) user@host:~/Data/tao/cv_samples_v1.4.1$ jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root

依據命令返回說明，開啟網頁

進入 yolo_v4_tiny, 點選 yolo_v4_tiny.ipynb

修改下列環境變數到你真實的位置

%env LOCAL_PROJECT_DIR=YOUR_LOCAL_PROJECT_DIR_PATH

%env LOCAL_PROJECT_DIR=/home/user/Data/tao/yolo_v4_tiny_1.4.1

檢查工具版本

(tao) user@host:~/Data/tao/yolo_v4_tiny_1.4.1$ python3 --version

(tao) user@host:~/Data/tao/yolo_v4_tiny_1.4.1$ docker version

(tao) user@host:~/Data/tao/yolo_v4_tiny_1.4.1$ apt list -a nvidia-container-toolkit

(tao) user@host:~/Data/tao/yolo_v4_tiny_1.4.1$ apt list -a nvidia-docker2

(tao) user@host:~/Data/tao/yolo_v4_tiny_1.4.1$ nvidia-smi

(tao) user@host:~/Data/tao/yolo_v4_tiny_1.4.1$ docker login nvcr.io

到 https://catalog.ngc.nvidia.com/ 搜尋 object

可找到 TAO Pretrained Object Detection

進入，點選右上 Download, 選 CLI，將命令拷貝

可由命令查到真實位置

ngc registry model download-version "nvidia/tao/pretrained_object_detection:cspdarknet_tiny"

參考 https://catalog.ngc.nvidia.com/orgs/nvidia/containers/deepstream

user@host:~/Data/V2Pdetect$ docker run --gpus all -it --rm --net=host --privileged -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.2 nvcr.io/nvidia/deepstream:6.2-devel

查詢已安裝套件的版本

參考 https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Quickstart.html?highlight=compatibility#platform-and-os-compatibility

其中 nvidia driver 的版本, docker 是使用 host 的 driver

root@host:/opt/nvidia/deepstream/deepstream-6.2# pip list

root@host:/opt/nvidia/deepstream/deepstream-6.2# dpkg -l|grep nvinfer

root@host:/opt/nvidia/deepstream/deepstream-6.2# dpkg -l|grep cudnn

root@host:/opt/nvidia/deepstream/deepstream-6.2# dpkg -l|grep cudnn

root@host:/opt/nvidia/deepstream/deepstream-6.2# cat /etc/os-release 

root@host:/opt/nvidia/deepstream/deepstream-6.2# update-alternatives --display cuda

root@host:/opt/nvidia/deepstream/deepstream-6.2# nvidia-smi

root@host:/opt/nvidia/deepstream/deepstream-6.2# ./install.sh 

root@host:/opt/nvidia/deepstream/deepstream-6.2# ./user_additional_install.sh 

user@host:~/Data/DeepStream/deepstream_tao_apps$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git

user@host:~/Data/DeepStream/deepstream_tao_apps$ cd deepstream_tao_apps

user@host:~/Data/DeepStream/deepstream_tao_apps/deepstream_tao_apps$ git show-ref

user@host:~/Data/DeepStream/deepstream_tao_apps/deepstream_tao_apps$ cd ..

user@host:~/Data/DeepStream/deepstream_tao_apps$ mv deepstream_tao_apps deepstream_tao_apps-tao4.0_ds6.2ga

user@host:~/Data/DeepStream/deepstream_tao_apps$ cd deepstream_tao_apps-tao4.0_ds6.2ga/

user@host:~/Data/V2Pdetect$ docker run --gpus all -it --rm --net=host --privileged \

  -v /tmp/.X11-unix:/tmp/.X11-unix \

  -v /etc/localtime:/etc/localtime \

  -v /home/user/Data/DeepStream/deepstream_tap_apps/deepstream_tao_apps-tao4.0_ds6.2ga/:/home/deepstream_tao_apps \

  -v /home/user/Data/V2Pdetect/multi_rtsp:/home/multi_rtsp \

  -e DISPLAY=$DISPLAY \

  -w /opt/nvidia/deepstream/deepstream-6.2 \

  nvcr.io/nvidia/deepstream:6.2-devel

root@host:/opt/nvidia/deepstream/deepstream-6.2# cd /home/deepstream_tao_apps/

root@host:/home/deepstream_tao_apps# ./download_models.sh 

root@host:/home/deepstream_tao_apps# ls models/yolov4-tiny/

root@host:/home/deepstream_tao_apps# cd post_processor/

root@host:/home/deepstream_tao_apps/post_processor# make

Makefile:25: *** "CUDA_VER is not set".  Stop.

root@host:/home/deepstream_tao_apps/post_processor# dpkg -l | grep CUDA

root@host:/home/deepstream_tao_apps/post_processor# export CUDA_VER=11.8

root@host:/home/deepstream_tao_apps/post_processor# cd ../apps/tao_detection/

root@host:/home/deepstream_tao_apps/apps/tao_detection# make

自己準備個 sample_720p.h264，以便測試

root@host:/home/deepstream_tao_apps/apps/tao_detection# ./ds-tao-detection -c ../../configs/yolov4-tiny_tao/pgie_yolov4_tiny_tao_config.txt -i file:///home/deepstream_tao_apps/sample/streams/sample_720p.h264 -d

root@host:/home/deepstream_tao_apps/apps/tao_detection# ./ds-tao-detection -c ../../configs/yolov4-tiny_tao/pgie_yolov4_tiny_tao_config.txt -i rtsp://root:passwd@192.168.0.107:554/live1s1.sdp -d

將 tao 產生的 labels.txt, yolov4_cspdarknet_tiny_epoch_080.etlt 拷貝到 deepstream_tao_apps 下

並合併 nvinfer_config.txt 和 pgie_yolov4_tiny_tao_config.txt 

root@host:/home/deepstream_tao_apps/apps/tao_detection# ./ds-tao-detection -c ../../configs/yolov4-tiny_tao/pgie_light.txt -i file:///home/deepstream_tao_apps/sample/streams/sample_720p.h264 -d

root@host:/home/multi_rtsp# apt-get update

root@host:/home/multi_rtsp# apt-get install libopencv-dev

root@host:/home/multi_rtsp# apt-get install libclutter-gst-3.0-dev

deepstream create_pipeline 分析

/opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-app/deepstream_app.c

create_pipeline (AppCtx * appCtx,

    bbox_generated_callback bbox_generated_post_analytics_cb,

    bbox_generated_callback all_bbox_generated_cb, perf_callback perf_cb,

    overlay_graphics_callback overlay_graphics_cb)

{

    multi_src_bin = create_multi_source_bin {

        sub_bins[] = create_camera_source_bin | 

            create_uridecode_src_bin | 

            create_rtsp_src_bin | 

            create_uridecode_src_bin_audio;

        sub_bins[] - nvstreammux;

    };

    create_common_elements {

        secondary_gie_bin = create_secondary_gie_bin {

            sub_bins[] = create_secondary_gie {

                secondary_gie = nvinfer | nvinferserver

                if (num_children == 0) {

                    queue - secondary_gie - fakesink

                } else {

                    queue - secondary_gie - tee

                }

            };

            tee - queue

            tee - sub_bins[]...

        };

        dsanalytics_bin = create_dsanalytics_bin {

            queue - nvdsanalytics

        };

        tracker_bin = create_tracking_bin { nvtracker };

        primary_gie_bin = create_primary_gie_bin {

            queue - nvvideoconvert - (nvinfer | nvinferserver) 

        };

        preprocess_bin = create_preprocess_bin {

            queue | nvdspreprocess

        };

        common_elements = nvmsgconv - tee

        {   preprocess_bin - primary_gie_bin - tracker_bin - 

            dsanalytics_bin - secondary_gie_bin - common_elements - 

            tee

        };

        secondary_gie_bin(src) -> bbox_generated_post_analytics_cb()

        sink_elem = preprocess_bin

        src_elem = tee

    };

    dsexample_bin = create_dsexample_bin { 

        queue - nvvideoconvert - capsfilter - dsexample 

    };

    demuxer = { nvstreamdemux };

    instance_bins[] = create_processing_instance {

        osd_bin = create_osd_bin { 

            queue - nvvideoconvert - queue - nvdsosd 

        };

        sink_bin = create_sink_bin {

            sub_bins[] = create_render_bin | 

                create_encode_file_bin | 

                create_udpsink_bin | 

                create_msg_conv_broker_bin

            queue - tee - sub_bins[]

        };

        osd_bin - sink_bin

        (sink)osd_bin -> all_bbox_generated_cb()

        (sink)osd_bin -> overlay_graphics_cb()

    };

    demux_instance_bins[] = create_demux_pipeline {

        osd_bin = create_osd_bin {

            queue - nvvideoconvert - queue - nvdsosd 

        };

        demux_sink_bin = create_demux_sink_bin {

            sub_bins[] = create_render_bin |

                create_encode_file_bin |

                create_udpsink_bin |

                create_msg_conv_broker_bin

            queue - tee - sub_bins[]

        };

        osd_bin - demux_sink_bin

    };

    tiled_display_bin = create_tiled_display_bin {

        queue - nvmultistreamtiler

    };

    if (tiled_display_config.enable) {

        multi_src_bin - (sink_elem-src_elem) - dsexample_bin - tiler_tee

            tiler_tee - nvstreamdemux - demux_instance_bins[]

            tiler_tee - tiled_display_bin - instance_bins[]

    } else {

        multi_src_bin - (sink_elem-src_elem) - dsexample_bin - demuxer - instance_bins[]

    };

};

Ubuntu 18.04 重灌

因為 Ubuntu 18.04 只能安裝 DeepStream 6.0.1

在 Ubuntu 20.04 才能安裝 DeepStream 6.1

所以在 Ubuntu 18.04 上安裝 Docker Ubuntu 20.04

版本對應如下

===========================

DS 6.1

Ubuntu 20.04

GCC 9.4.0

CUDA 11.6.1

cuDNN 8.4.0.27

TRT 8.2.5.1

Display Driver:R510.47.03

GStreamer 1.16.2

OpenCV 4.2.0

deepstream:6.1

===========================

DS 6.0.1

Ubuntu 18.04

GCC 7.3.0

CUDA 11.4.1

cuDNN 8.2+

TRT 8.0.1

Display Driver:R470.63.01

GStreamer 1.14.5

OpenCV 3.4.0

deepstream:6.0.1

===========================

安裝作業系統

BIOS 選擇開機

Install Ubuntu

Installation type 選 Something else

Create partition/Mount point 選擇 /

Settings/Details/Users

Unlock

Automatic Login: ON

更新系統，安裝一些常用套件

$ sudo apt-get update

$ sudo apt-get upgrade

$ sudo apt-get install ssh

$ sudo apt-get install python3-pip

mount nfs

$ sudo apt-get install nfs-common

$ sudo mount -t nfs ip:/share_folder /mount_folder

$ sudo vi /etc/fstab

#中間的空格要使用 tab

ip:/share_folder /mnt/mount_folder nfs defaults,bg 0 0

$ sudo mount -a

vnc

$ sudo apt-get install x11vnc

$ sudo x11vnc -storepasswd

$ sudo chown user.group ~/.vnc/passwd

安裝顯示卡驅動，CUDA 和 CUDNN

$ sudo ubuntu-drivers devices

$ sudo apt-get install nvidia-driver-510

https://developer.nvidia.com/cuda-downloads

選擇 Archive of Previous CUDA Releases

選擇 CUDA Toolkit 11.4.1

選擇 deb(local), deb(network) 不可使用了

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin

$ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600

$ wget https://developer.download.nvidia.com/compute/cuda/11.4.1/local_installers/cuda-repo-ubuntu1804-11-4-local_11.4.1-470.57.02-1_amd64.deb

$ sudo dpkg -i cuda-repo-ubuntu1804-11-4-local_11.4.1-470.57.02-1_amd64.deb

$ sudo apt-key add /var/cuda-repo-ubuntu1804-11-4-local/7fa2af80.pub

$ sudo apt-get update

$ sudo apt-get -y install cuda-11-4

之後再安裝一遍 CUDA Toolkit 11.3.1(因為 TensorRT 8.0.1 需要 CUDA-11-3)

$ sudo apt-get -y install cuda-11-3

之後再安裝一遍 CUDA Toolkit 11.6.1(因為 DeepStream 6.1 需要 CUDA-11-6)

$ sudo apt-get -y install cuda-11-6

$ update-alternatives --display cuda

$ update-alternatives --config cuda

參考 https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

到 2.2. Downloading cuDNN for Linux

下載 Local Install for Ubuntu18.04 x86_64(Deb)

$ sudo apt-get install zlib1g

$ sudo dpkg -i cudnn-local-repo-ubuntu1804-8.4.1.50_1.0-1_amd64.deb

$ sudo cp /var/cudnn-local-repo-ubuntu1804-8.4.1.50/cudnn-local-BA71F057-keyring.gpg /usr/share/keyrings/

$ sudo apt-get update

$ apt list -a libcudnn8

$ sudo apt-get install libcudnn8=8.4.1.50-1+cuda11.6

$ sudo apt-get install libcudnn8-dev=8.4.1.50-1+cuda11.6

$ sudo apt-get install libcudnn8-samples=8.4.1.50-1+cuda11.6

安裝 TensorRT 8.0.1

因為使用 Docker 安裝 DeepStream 6.1, 所以不用安裝 TensorRT 8.2.5.1

參考 https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html

在 TensorRT Archives 切換安裝文件版本

選擇 8.0.1 的 TensorRT Installation Guide

在 3. Downloading TensorRT

連結 https://developer.nvidia.com/tensorrt 按 GET STARTED

連結 https://developer.nvidia.com/tensorrt-getting-started 按 DOWNLOAD NOW

選擇 TensorRT 8

選擇 TensorRT 8.0 GA

選擇 TensorRT 8.0.1 GA for Ubuntu 18.04 and CUDA 11.3 DEB local repo package

$ sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626_1-1_amd64.deb

$ sudo apt-key add /var/nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626/7fa2af80.pub

$ sudo apt-get update

$ sudo apt --fix-broken install

$ sudo apt-get upgrade

$ sudo apt-get install tensorrt

安裝 DeepStream

因為不選擇最新的版本

參考 https://docs.nvidia.com/metropolis/deepstream-archive.html

選擇版本 6.0.1 DeepStream SDK Development Guid

到 dGPU Setup for Ubuntu

$ sudo apt install libssl1.0.0

$ sudo apt install libgstreamer1.0-0

$ sudo apt install gstreamer1.0-tools

$ sudo apt install gstreamer1.0-plugins-good

$ sudo apt install gstreamer1.0-plugins-bad

$ sudo apt install gstreamer1.0-plugins-ugly

$ sudo apt install gstreamer1.0-libav

$ sudo apt install libgstrtspserver-1.0-0

$ sudo apt install libjansson4

$ sudo apt install gcc

$ sudo apt install make

$ sudo apt install git

$ sudo apt install python3

$ cd /usr/bin

$ sudo ln -s python3 python

$ git clone https://github.com/edenhill/librdkafka.git

$ cd librdkafka

$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a

$ ./configure

$ make

$ sudo make install

$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-6.0/lib

$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-6.0/lib

下載 Deepstream 6.0 dGPU Debian package

https://developer.nvidia.com/deepstream-6.0_6.0.1-1_amd64deb

$ sudo apt-get install ./deepstream-6.0_6.0.1-1_amd64.deb

$ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin

$ cd /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/

$ deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt

安裝  Nvidia Docker

參考 https://docs.docker.com/engine/install/ubuntu/

$ sudo apt-get update

$ sudo apt-get install ca-certificates

$ sudo apt-get install curl

$ sudo apt-get install gnupg

$ sudo apt-get install lsb-release

$ sudo mkdir -p /etc/apt/keyrings

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

$ echo \

  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \

  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

$ sudo apt-get update

$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin

$ sudo docker run --rm hello-world

參考 https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker

$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \

      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \

      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \

            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \

            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

$ sudo apt-get update

$ sudo apt-get install -y nvidia-docker2

$ sudo systemctl restart docker

$ sudo groupadd docker

$ sudo usermod -a -G docker $USER

$ sudo reboot

安裝 NGC CLI

參考 https://ngc.nvidia.com/setup/installers/cli

$ wget --content-disposition https://ngc.nvidia.com/downloads/ngccli_linux.zip && \

  unzip ngccli_linux.zip && \

  chmod u+x ngc-cli/ngc

$ find ngc-cli/ -type f -exec md5sum {} + | LC_ALL=C sort | md5sum -c ngc-cli.md5

$ echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile && source ~/.bash_profile

$ ngc config set

# 直接 enter 即可

$ docker login nvcr.io

Username: $oauthtoken

Password: <Your API Key>

使用 Docker 安裝 TensorRT OSS

開啟 https://github.com/nvidia/TensorRT

切換至 Tags 8.0.1

$ git clone -b master https://github.com/nvidia/TensorRT TensorRT_OSS-8.0.1

$ cd TensorRT_OSS-8.0.1/

$ git describe --tags

8.2.0-EA-2-g96e2397

$ git tag -l

$ git branch -r

$ git checkout 8.0.1

$ git log -1

$ git describe --tags

8.0.1

$ git submodule update --init --recursive

$ vi docker/ubuntu-18.04.Dockerfile

修改下列一行

RUN cd /usr/local/bin && wget https://ngc.nvidia.com/downloads/ngccli_cat_linux.zip && \

  unzip ngccli_cat_linux.zip && chmod u+x ngc-cli/ngc && \

  rm ngccli_cat_linux.zip ngc-cli.md5 && echo "no-apikey\nascii\n" | ngc-cli/ngc config set

$ cat docker/ubuntu-18.04.Dockerfile | grep CUDA_VERSION

ARG CUDA_VERSION=11.3.1

$ ./docker/build.sh --file docker/ubuntu-18.04.Dockerfile --tag tensorrt-ubuntu18.04-cuda11.3 --cuda 11.3.1

$ ./docker/launch.sh --tag tensorrt-ubuntu18.04-cuda11.3 --gpus all

/workspace$ cd $TRT_OSSPATH

/workspace/TensorRT$ mkdir -p build && cd build

/workspace/TensorRT/build$ cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out

/workspace/TensorRT/build$ make -j$(nproc)

/workspace/TensorRT/build$ exit

$ mkdir backup

$ sudo mv /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.8.0.1 backup/

$ sudo cp build/out/libnvinfer_plugin.so.8.0.1 /usr/lib/x86_64-linux-gnu/

另外還要再安裝 TensorRT 8.2.1 給 Docker Ubuntu 20.04

將 8.0.1 改成 8.2.1, 18.04 改成 20.04, 11.3.1 改成 11.4.2, 但不需安裝

用 Docker 開發 DeepStream 6.0.1

參考 https://docs.nvidia.com/metropolis/deepstream/6.0.1/dev-guide/text/DS_docker_containers.html

$ docker pull nvcr.io/nvidia/deepstream:6.0.1-devel

$ xhost +

access control disabled, clients can connect from any host

$ sudo docker run --gpus all -it --rm --net=host \

  -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime \

  -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel

# update-alternatives --display cuda

# cat /etc/os-release

# cd samples/configs/deepstream-app

# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

# cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-app

# export CUDA_VER=11.4

# make

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# gst-launch-1.0 uridecodebin uri=$URI ! nvvideoconvert ! nveglglessink

# exit

用 Docker 開發 DeepStream 6.1

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_docker_containers.html

$ docker pull nvcr.io/nvidia/deepstream:6.1-devel

$ xhost +

access control disabled, clients can connect from any host

$ sudo docker run --gpus all -it --rm --net=host \

  -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime \

  -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.1 nvcr.io/nvidia/deepstream:6.1-devel

# cd samples/configs/deepstream-app

# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

# cd /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-app

# export CUDA_VER=11.6

# make

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.1/lib/gst-plugins/

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# gst-launch-1.0 uridecodebin uri=$URI ! nvvideoconvert ! nveglglessink

# exit

用 Docker DeepStream 6.0.1 測試 Integrate TAO model with DeepStream SDK

$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git deepstream_tao_apps-tao3.0_ds6.0.1

$ sudo apt install gitk

$ cd deepstream_tao_apps-tao3.0_ds6.0.1/

$ git branch -r

$ git checkout release/tao3.0_ds6.0.1

$ git log -1

$ sudo docker run --gpus all -it --rm --net=host \

> -v /etc/localtime:/etc/localtime \

> -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \

> -v /home/mark/Data/TensorRT/TensorRT_OSS-8.0.1/:/home/TensorRT \

> -v /home/mark/Data/DeepStream/deepstream_tap_apps/deepstream_tao_apps-tao3.0_ds6.0.1/:/home/deepstream_tao_apps \

> -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel

# cd /home/deepstream_tao_apps/

# ./download_models.sh 

# export CUDA_VER=11.4

# make

# cp /home/TensorRT/build/out/libnvinfer_plugin.so.8.0.1 /usr/lib/x86_64-linux-gnu/

# ./apps/tao_detection/ds-tao-detection -c configs/frcnn_tao/pgie_frcnn_tao_config.txt -i /opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.h264 -d

# cd /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-nvdsvideotemplate/

# make

# cp libnvdsgst_videotemplate.so /opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/

# rm -rf ~/.cache/gstreamer-1.0/

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.0/lib/cvcore_libs/

# cd /home/deepstream_tao_apps/apps/tao_others/

# make

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# cd deepstream-bodypose2d-app/

# ./deepstream-bodypose2d-app 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./body2dout

# cd ../deepstream-emotion-app/

# ./deepstream-emotion-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-faciallandmark-app/

# ./deepstream-faciallandmark-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-gaze-app/

# ./deepstream-gaze-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./gazenet

# cd ../deepstream-gesture-app/

# ./deepstream-gesture-app 3 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./gesture

# cd ../deepstream-heartrate-app/

# ./deepstream-heartrate-app 3 $URI ./heartrate

# exit

用 Docker DeepStream 6.1 測試 Integrate TAO model with DeepStream SDK

$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git deepstream_tao_apps-tao3.0_ds6.1ga

$ sudo apt install gitk

$ cd deepstream_tao_apps-tao3.0_ds6.1ga/

$ git branch -r

$ git checkout release/tao3.0_ds6.1ga

$ git log -1

$ sudo docker run --gpus all -it --rm --net=host \

> -v /etc/localtime:/etc/localtime \

> -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \

> -v /home/mark/Data/TensorRT/TensorRT_OSS-8.2.1/:/home/TensorRT \

> -v /home/mark/Data/DeepStream/deepstream_tap_apps/deepstream_tao_apps-tao3.0_ds6.1ga/:/home/deepstream_tao_apps \

> -w /opt/nvidia/deepstream/deepstream-6.1 nvcr.io/nvidia/deepstream:6.1-devel

# cp /home/TensorRT/build/out/libnvinfer_plugin.so.8.2.1 /usr/lib/x86_64-linux-gnu/

# cd /usr/lib/x86_64-linux-gnu/

# rm libnvinfer_plugin.so.8

# ln -s libnvinfer_plugin.so.8.2.1 libnvinfer_plugin.so.8

# cd /home/deepstream_tao_apps/

# ./download_models.sh 

# export CUDA_VER=11.6

# make

# ./apps/tao_detection/ds-tao-detection -c configs/frcnn_tao/pgie_frcnn_tao_config_dgpu.txt -i /opt/nvidia/deepstream/deepstream-6.1/samples/streams/sample_720p.h264 -d

# cd /opt/nvidia/deepstream/deepstream-6.1/sources/gst-plugins/gst-nvdsvideotemplate/

# make

# cp libnvdsgst_videotemplate.so /opt/nvidia/deepstream/deepstream-6.1/lib/gst-plugins/

# rm -rf ~/.cache/gstreamer-1.0/

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.1/lib/cvcore_libs/

# cd /home/deepstream_tao_apps/apps/tao_others/

# make

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# cd deepstream-bodypose2d-app/

# ./deepstream-bodypose2d-app 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt 0 0 $URI ./body2dout

# cd ../deepstream-emotion-app/

# ./deepstream-emotion-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-faciallandmark-app/

# ./deepstream-faciallandmark-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-gaze-app/

# ./deepstream-gaze-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./gazenet

# cd ../deepstream-gesture-app/

# ./deepstream-gesture-app 3 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./gesture

# cd ../deepstream-heartrate-app/

# ./deepstream-heartrate-app 3 $URI ./heartrate

# exit

安裝其他元件

參考 Ubuntu 18.04 重灌 上的 安裝 CMake 和 安裝 OpenCV

Integrate TAO model with DeepStream SDK In Docker

參考 DeepStream Platform and OS Compatibility
參考 TensorRT Open Source Software

參考 Ubuntu 安裝 DeepStream 使用 Docker 部分

參考 Build TensorRT OSS(Open Source Software)

在 DeepStream Platform and OS Compatibility 文件中

DS 6.0.1

CUDA 11.4.1

cuDNN 8.2+

TRT 8.0.1

Driver R470.63.01

在 nvcr.io/nvidia/deepstream:6.0.1-devel Docker 中

Deepstream 6.0.1

TensorRT 8.0.1-1 + cuda11.3

替換 nvidia driver，不然會產生錯誤

Failed to establish dbus connection

$ apt list --installed|grep nvidia

$ sudo ubuntu-drivers devices

$ sudo apt-get remove --purge nvidia-driver-495

$ sudo apt-get install nvidia-driver-470

$ sudo reboot

確認 CUDA, TensorRT 安裝的版本

$ dpkg -l | grep -i cuda-cuda

$ dpkg -L cuda-cudart-11-5

$ dpkg -l | grep TensorRT

準備 DeepStream 開發用 Docker

$ docker pull nvcr.io/nvidia/deepstream:6.0.1-devel

$ xhost +

access control disabled, clients can connect from any host

$ sudo docker run --gpus ""device=0"" -it --rm --net=host -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel

# cd samples/configs/deepstream-app

# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

# cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps

/deepstream-app

# export CUDA_VER=11.4

# make

# exit

準備 TensorRT OSS 編譯環境 Docker

$ git clone -b master https://github.com/nvidia/TensorRT TensorRT_OSS-8.0.1

$ cd TensorRT_OSS-8.0.1

$ git tag -n

$ git checkout 8.0.1

$ git describe --tags

8.0.1

$ git submodule update --init --recursive

$ cat docker/ubuntu-18.04.Dockerfile | grep CUDA_VERSION

ARG CUDA_VERSION=11.3.1

$ ./docker/build.sh --file docker/ubuntu-18.04.Dockerfile --tag tensorrt-ubuntu18.04-cuda11.3.1 --cuda 11.3.1

$ ./docker/launch.sh --tag tensorrt-ubuntu18.04-cuda11.3.1 --gpus all

$ cd TensorRT

$ mkdir -p build && cd build

$ cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out

$ make -j$(nproc)

$ exit

下載主要程式，須注意版本

$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git

$ sudo apt install gitk

$ git branch -r

$ git checkout release/tao3.0_ds6.0.1

$ git log -1

$ sudo docker run --gpus all -it --rm --net=host \

 -v /etc/localtime:/etc/localtime \

 -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \

 -v /your_path_to/Data/TensorRT/TensorRT_OSS-8.0.1:/home/TensorRT \

 -v /your_path_to/Data/DeepStream/deepstream_tao_apps:/home/deepstream_tao_apps \

 -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel

# cd /home/deepstream_tao_apps/

# ./download_models.sh 

# export CUDA_VER=11.4

# make

# cp /home/TensorRT/build/out/libnvinfer_plugin.so.8.0.1 /usr/lib/x86_64-linux-gnu/

# ./apps/tao_detection/ds-tao-detection -c configs/frcnn_tao/pgie_frcnn_tao_config.txt -i /opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.h264 -d

# cd /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-nvdsvideotemplate/

# make

# cp libnvdsgst_videotemplate.so /opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/

# rm -rf ~/.cache/gstreamer-1.0/

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream/lib/cvcore_libs

# cd /home/deepstream_tao_apps/apps/tao_others/deepstream-gaze-app

# ./deepstream-gaze-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./gazenet

# cd ../deepstream-gesture-app

# ./deepstream-gesture-app 3 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./gesture

# cd ../deepstream-bodypose2d-app/

# ./deepstream-bodypose2d-app 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./body2dout

# cd ../deepstream-faciallandmark-app/

# ./deepstream-faciallandmark-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-emotion-app/

# ./deepstream-emotion-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

在 Jeston 上測試

$ docker pull nvcr.io/nvidia/l4t-base:r32.4.3

$ xhost +

$ sudo docker run -it --rm --net=host --runtime nvidia  -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-base:r32.4.3

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# gst-launch-1.0 uridecodebin uri=$URI name=d d. ! nvvidconv ! nvegltransform ! nveglglessink

# gst-launch-1.0 uridecodebin uri=$URI name=d d. ! nvvidconv ! nvoverlaysink