Jetpack 6.0 更新 CUDA 12.2 至 12.6

本篇參考 安裝 Flux & ComfyUI 將其中的一些細節敘述清楚

參考 https://www.jetson-ai-lab.com/tutorial_comfyui_flux.html

參考 https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Installation.html#id10

$ sudo jtop

CUDA: 12.2.140

cuDNN: 8.9.4.25

TensorRT: 8.6.2.3

因為需要從 cuda-12.2 更新到 cuda-12.6

而在更新的過程中，會移除 Deepstream-7.0

所以需要自己安裝 Deepstream-7.1

記錄下目前的版本資料

$ dpkg -l>dekg_jp6.0.txt

$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh

$ chmod +x Miniconda3-latest-Linux-aarch64.sh

$ ./Miniconda3-latest-Linux-aarch64.sh

$ conda update conda

$ conda create -n comfyui python=3.10

$ conda init bash

$ cat .bashrc

$ conda activate comfyui

$ conda info --envs

$ conda deactivate

$ conda activate comfyui

$ conda list

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/arm64/cuda-keyring_1.1-1_all.deb

$ sudo dpkg -i cuda-keyring_1.1-1_all.deb

$ sudo apt-get update

$ sudo apt-get -y install cuda-toolkit-12-6 cuda-compat-12-6

$ update-alternatives --list cuda

$ update-alternatives --display cuda

$ sudo update-alternatives --config cuda

cuda-12.6

$ apt list -a cudnn

$ sudo apt-get install cudnn=9.10.2-1

$ apt list -a python3-libnvinfer

$ sudo apt-get install python3-libnvinfer=10.7.0.23+cuda12.6

The following packages will be REMOVED:

  deepstream-7.0 libnvparsers-dev nvidia-tensorrt-dev

$ sudo apt-get install python3-libnvinfer-dev=10.7.0.23-1+cuda12.6

$ sudo jtop

CUDA: 12.6.85

cuDNN: 9.10.2

TensorRT: 10.7.0.23

$ wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/org/nvidia/deepstream/7.1/files?redirect=true&path=deepstream-7.1_7.1.0-1_arm64.deb' -O deepstream-7.1_7.1.0-1_arm64.deb

$ sudo apt-get install ./deepstream-7.1_7.1.0-1_arm64.deb

$ export BNB_CUDA_VERSION=126

$ export LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64:$LD_LIBRARY_PATH

$ git clone https://github.com/timdettmers/bitsandbytes.git

$ cd bitsandbytes

$ pip uninstall numpy

$ pip install "numpy<2.0"

$ mkdir -p build

$ cd build

$ cmake .. -DCOMPUTE_BACKEND=cuda -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-12.6

$ make -j$(nproc)

$ cd ..

$ python setup.py install

$ pip install http://jetson.webredirect.org/jp6/cu124/+f/5fe/ee5f5d1a75229/torch-2.3.0-cp310-cp310-linux_aarch64.whl

$ pip install http://jetson.webredirect.org/jp6/cu124/+f/988/cb71323efff87/torchvision-0.18.0a0+6043bc2-cp310-cp310-linux_aarch64.whl

$ pip install http://jetson.webredirect.org/jp6/cu124/+f/0aa/a066463c02b4a/torchaudio-2.3.0+952ea74-cp310-cp310-linux_aarch64.whl

$ python3

>>> import bitsandbytes as bnb

>>> print(bnb.__version__)

$ git clone https://github.com/comfyanonymous/ComfyUI.git

$ cd ComfyUI

$ pip install -r requirements.txt

$ cd custom_nodes

$ git clone https://github.com/ltdrdata/ComfyUI-Manager.git

$ cd ..

$ mkdir workflows

$ cd workflows

$ wget https://www.jetson-ai-lab.com/assets/workflow_agx_orin_4steps.json

$ cd ..

到 https://huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main

下載 flux1-schnell.safetensors 放到 models/unet

下載 ae.safetensors 放到 models/vae/FLUX1

到 https://huggingface.co/stabilityai/stable-diffusion-3-medium/tree/main/text_encoders

下載 clip_l.safetensors 和 t5xxl_fp8_e4m3fn.safetensors 放到 models/clip

$ python main.py --port=8080

在 http://127.0.0.1:8080/ 網頁，按 Queue Prompt, 等待...

Load workflow_agx_orin_4steps.json 時，需修改 "Load VAE" 的 vae_name 到 FLUX1/ae.safetensors

記錄下目前的版本資料

$ dpkg -l>dpkg_jp6.0_comfyui.txt

若需要將 cuda-12.6 還回 cuda-12.2

參考記錄下來的版本資料 dpkg_jp6.0.txt dpkg_jp6.0_comfyui.txt

參考 https://repo.download.nvidia.com/jetson 下載 package

移除不必要的 package

$ sudo apt-get remove --purge package

安裝 package 時，常出現相依性錯誤，所以有時要變更安裝順序

甚至最後重新安裝一遍，確保全部安裝

$ cd jetpack_6.0

$ ./download.sh

$ ./rollback.sh

$ ./install.sh

參考 https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-cuda/tags

$ xhost +

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  nvcr.io/nvidia/l4t-cuda:11.4.19-runtime

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  nvcr.io/nvidia/l4t-cuda:12.2.12-runtime

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  nvcr.io/nvidia/l4t-cuda:12.6.11-runtime

docker: Error response from daemon: failed to create task for container: 

failed to create shim task: OCI runtime create failed: 

failed to create NVIDIA Container Runtime: failed to construct OCI spec modifier: 

requirements not met: unsatisfied condition: cuda>=12.6 (cuda=12.2): unknown.

因為 Jetpack 6.0 自帶 CUDA 12.2

使用的 docker image 不能比 CUDA 12.2 高 

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  -w /opt/nvidia/deepstream/deepstream-7.0 \

  nvcr.io/nvidia/deepstream-l4t:7.0-samples-multiarch

因為 Jetpack 6.0 自帶 CUDA 12.2

使用的 docker image 不能比 CUDA 12.2 高 

所以也無法使用 Deepstream 7.1

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  -w /opt/nvidia/deepstream/deepstream-7.1 \

  nvcr.io/nvidia/deepstream-l4t:7.1-samples-multiarch

安裝 Flux & ComfyUI

參考 https://www.jetson-ai-lab.com/tutorial_comfyui_flux.html

$ sudo jtop

CUDA: 12.2.140

cuDNN: 8.9.4.25

TensorRT: 8.6.2.3

因為需要從 cuda-12.2 更新到 cuda-12.6

而在更新的過程中，會移除 Deepstream-7.0

所以需要自己安裝 Deepstream-7.1

記錄下目前的版本資料

$ dpkg -l>dekg_jp6.0.txt

$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh

$ chmod +x Miniconda3-latest-Linux-aarch64.sh

$ ./Miniconda3-latest-Linux-aarch64.sh

$ conda update conda

$ conda create -n comfyui python=3.10

$ conda init bash

$ cat .bashrc

$ conda activate comfyui

$ conda info --envs

$ conda deactivate

$ conda activate comfyui

$ conda list

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/arm64/cuda-keyring_1.1-1_all.deb

$ sudo dpkg -i cuda-keyring_1.1-1_all.deb

$ sudo apt-get update

$ sudo apt-get -y install cuda-toolkit-12-6 cuda-compat-12-6

$ update-alternatives --list cuda

$ update-alternatives --display cuda

$ sudo update-alternatives --config cuda

cuda-12.6

$ sudo apt-get install cudnn=9.5.1-1

$ sudo apt-get install python3-libnvinfer=9.5.1-1

The following packages will be REMOVED:

  deepstream-7.0 libnvparsers-dev nvidia-tensorrt-dev

$ sudo apt-get install python3-libnvinfer-dev=10.6.0.26-1+cuda12.6

$ sudo jtop

CUDA: 12.6.77

cuDNN: 9.5.1

TensorRT: 10.6.0.26

$ wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/org/nvidia/deepstream/7.1/files?redirect=true&path=deepstream-7.1_7.1.0-1_arm64.deb' -O deepstream-7.1_7.1.0-1_arm64.deb

$ sudo apt-get install ./deepstream-7.1_7.1.0-1_arm64.deb

$ export BNB_CUDA_VERSION=126

$ export LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64:$LD_LIBRARY_PATH

$ git clone https://github.com/timdettmers/bitsandbytes.git

$ cd bitsandbytes

$ pip uninstall numpy

$ pip install "numpy<2.0"

$ mkdir -p build

$ cd build

$ cmake .. -DCOMPUTE_BACKEND=cuda -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-12.6

$ make -j$(nproc)

$ cd ..

$ python setup.py install

$ pip install http://jetson.webredirect.org/jp6/cu124/+f/5fe/ee5f5d1a75229/torch-2.3.0-cp310-cp310-linux_aarch64.whl

$ pip install http://jetson.webredirect.org/jp6/cu124/+f/988/cb71323efff87/torchvision-0.18.0a0+6043bc2-cp310-cp310-linux_aarch64.whl

$ pip install http://jetson.webredirect.org/jp6/cu124/+f/0aa/a066463c02b4a/torchaudio-2.3.0+952ea74-cp310-cp310-linux_aarch64.whl

$ python3

>>> import bitsandbytes as bnb

>>> print(bnb.__version__)

$ git clone https://github.com/comfyanonymous/ComfyUI.git

$ cd ComfyUI

$ pip install -r requirements.txt

$ cd custom_nodes

$ git clone https://github.com/ltdrdata/ComfyUI-Manager.git

$ cd ..

$ mkdir workflows

$ cd workflows

$ wget https://www.jetson-ai-lab.com/assets/workflow_agx_orin_4steps.json

$ cd ..

到 https://huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main

下載 flux1-schnell.safetensors 放到 models/unet

下載 ae.safetensors 放到 models/vae/FLUX1

到 https://huggingface.co/stabilityai/stable-diffusion-3-medium/tree/main/text_encoders

下載 clip_l.safetensors 和 t5xxl_fp8_e4m3fn.safetensors 放到 models/clip

$ python main.py --port=8080

http://127.0.0.1:8080/

Load workflow_agx_orin_4steps.json 時，需修改 "Load VAE" 的 vae_name 到 FLUX1/ae.safetensors

記錄下目前的版本資料

$ dpkg -l>dekg_jp6.0_cuda-12.6.txt

若需要將 cuda-12.6 還回 cuda-12.2

$ cd jetpack_6.0

$ ./download.sh

$ ./install.sh

參考記錄下來的版本資料 dekg_jp6.0.txt dekg_jp6.0_cuda-12.6.txt

移除不必要的 package

$ sudo apt-get remove --purge package

參考 https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-cuda/tags

$ xhost +

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  nvcr.io/nvidia/l4t-cuda:11.4.19-runtime

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  nvcr.io/nvidia/l4t-cuda:12.2.12-runtime

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  nvcr.io/nvidia/l4t-cuda:12.6.11-runtime

docker: Error response from daemon: failed to create task for container: 

failed to create shim task: OCI runtime create failed: 

failed to create NVIDIA Container Runtime: failed to construct OCI spec modifier: 

requirements not met: unsatisfied condition: cuda>=12.6 (cuda=12.2): unknown.

$ docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  -w /opt/nvidia/deepstream/deepstream-7.0 \

  nvcr.io/nvidia/deepstream-l4t:7.0-samples-multiarch

docker run -it --rm --net=host --runtime nvidia\

  -e DISPLAY=$DISPLAY \

  -v /tmp/.X11-unix/:/tmp/.X11-unix \

  -w /opt/nvidia/deepstream/deepstream-7.1 \

  nvcr.io/nvidia/deepstream-l4t:7.1-samples-multiarch

install opencv4.6.0 with CUDA for Jetson

參考 https://forums.developer.nvidia.com/t/best-way-to-install-opencv-with-cuda-on-jetpack-5-xavier-nx-opencv-for-tegra/222777

下載 install_opencv4.6.0_jetson.sh 備份如下

#!/bin/bash

#

# Copyright (c) 2022, NVIDIA CORPORATION.  All rights reserved.

#

# NVIDIA Corporation and its licensors retain all intellectual property

# and proprietary rights in and to this software, related documentation

# and any modifications thereto.  Any use, reproduction, disclosure or

# distribution of this software and related documentation without an express

# license agreement from NVIDIA Corporation is strictly prohibited.

#

version="4.6.0"

folder="workspace"

for (( ; ; ))

do

    echo "Do you want to remove the default OpenCV (yes/no)?"

    read rm_old

    if [ "$rm_old" = "yes" ]; then

        echo "** Remove other OpenCV first"

        sudo apt -y purge *libopencv*

break

    elif [ "$rm_old" = "no" ]; then

break

    fi

done

echo "------------------------------------"

echo "** Install requirement (1/4)"

echo "------------------------------------"

sudo apt-get update

sudo apt-get install -y build-essential cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev

sudo apt-get install -y libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev

sudo apt-get install -y python3.8-dev python-dev python-numpy python3-numpy

sudo apt-get install -y libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev

sudo apt-get install -y libv4l-dev v4l-utils qv4l2 v4l2ucp

sudo apt-get install -y curl

echo "------------------------------------"

echo "** Download opencv "${version}" (2/4)"

echo "------------------------------------"

mkdir $folder

cd ${folder}

curl -L https://github.com/opencv/opencv/archive/${version}.zip -o opencv-${version}.zip

curl -L https://github.com/opencv/opencv_contrib/archive/${version}.zip -o opencv_contrib-${version}.zip

unzip opencv-${version}.zip

unzip opencv_contrib-${version}.zip

rm opencv-${version}.zip opencv_contrib-${version}.zip

cd opencv-${version}/

echo "------------------------------------"

echo "** Build opencv "${version}" (3/4)"

echo "------------------------------------"

mkdir release

cd release/

cmake -D WITH_CUDA=ON -D WITH_CUDNN=ON -D CUDA_ARCH_BIN="7.2,8.7" -D CUDA_ARCH_PTX="" -D OPENCV_GENERATE_PKGCONFIG=ON -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-${version}/modules -D WITH_GSTREAMER=ON -D WITH_LIBV4L=ON -D BUILD_opencv_python3=ON -D BUILD_TESTS=OFF -D BUILD_PERF_TESTS=OFF -D BUILD_EXAMPLES=OFF -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..

make -j$(nproc)

echo "------------------------------------"

echo "** Install opencv "${version}" (4/4)"

echo "------------------------------------"

sudo make install

echo 'export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH' >> ~/.bashrc

echo 'export PYTHONPATH=/usr/local/lib/python3.8/site-packages/:$PYTHONPATH' >> ~/.bashrc

source ~/.bashrc

echo "** Install opencv "${version}" successfully"

echo "** Bye :)"

將 cv2 加到 python3 中

$ export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.8/site-packages

$ python3

>>> import cv2

>>> cnt = cv2.cuda.getCudaEnabledDeviceCount()

>>> cnt

1 表示有 CUDA

Ubuntu 18.04 重灌

因為 Ubuntu 18.04 只能安裝 DeepStream 6.0.1

在 Ubuntu 20.04 才能安裝 DeepStream 6.1

所以在 Ubuntu 18.04 上安裝 Docker Ubuntu 20.04

版本對應如下

===========================

DS 6.1

Ubuntu 20.04

GCC 9.4.0

CUDA 11.6.1

cuDNN 8.4.0.27

TRT 8.2.5.1

Display Driver:R510.47.03

GStreamer 1.16.2

OpenCV 4.2.0

deepstream:6.1

===========================

DS 6.0.1

Ubuntu 18.04

GCC 7.3.0

CUDA 11.4.1

cuDNN 8.2+

TRT 8.0.1

Display Driver:R470.63.01

GStreamer 1.14.5

OpenCV 3.4.0

deepstream:6.0.1

===========================

安裝作業系統

BIOS 選擇開機

Install Ubuntu

Installation type 選 Something else

Create partition/Mount point 選擇 /

Settings/Details/Users

Unlock

Automatic Login: ON

更新系統，安裝一些常用套件

$ sudo apt-get update

$ sudo apt-get upgrade

$ sudo apt-get install ssh

$ sudo apt-get install python3-pip

mount nfs

$ sudo apt-get install nfs-common

$ sudo mount -t nfs ip:/share_folder /mount_folder

$ sudo vi /etc/fstab

#中間的空格要使用 tab

ip:/share_folder /mnt/mount_folder nfs defaults,bg 0 0

$ sudo mount -a

vnc

$ sudo apt-get install x11vnc

$ sudo x11vnc -storepasswd

$ sudo chown user.group ~/.vnc/passwd

安裝顯示卡驅動，CUDA 和 CUDNN

$ sudo ubuntu-drivers devices

$ sudo apt-get install nvidia-driver-510

https://developer.nvidia.com/cuda-downloads

選擇 Archive of Previous CUDA Releases

選擇 CUDA Toolkit 11.4.1

選擇 deb(local), deb(network) 不可使用了

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin

$ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600

$ wget https://developer.download.nvidia.com/compute/cuda/11.4.1/local_installers/cuda-repo-ubuntu1804-11-4-local_11.4.1-470.57.02-1_amd64.deb

$ sudo dpkg -i cuda-repo-ubuntu1804-11-4-local_11.4.1-470.57.02-1_amd64.deb

$ sudo apt-key add /var/cuda-repo-ubuntu1804-11-4-local/7fa2af80.pub

$ sudo apt-get update

$ sudo apt-get -y install cuda-11-4

之後再安裝一遍 CUDA Toolkit 11.3.1(因為 TensorRT 8.0.1 需要 CUDA-11-3)

$ sudo apt-get -y install cuda-11-3

之後再安裝一遍 CUDA Toolkit 11.6.1(因為 DeepStream 6.1 需要 CUDA-11-6)

$ sudo apt-get -y install cuda-11-6

$ update-alternatives --display cuda

$ update-alternatives --config cuda

參考 https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

到 2.2. Downloading cuDNN for Linux

下載 Local Install for Ubuntu18.04 x86_64(Deb)

$ sudo apt-get install zlib1g

$ sudo dpkg -i cudnn-local-repo-ubuntu1804-8.4.1.50_1.0-1_amd64.deb

$ sudo cp /var/cudnn-local-repo-ubuntu1804-8.4.1.50/cudnn-local-BA71F057-keyring.gpg /usr/share/keyrings/

$ sudo apt-get update

$ apt list -a libcudnn8

$ sudo apt-get install libcudnn8=8.4.1.50-1+cuda11.6

$ sudo apt-get install libcudnn8-dev=8.4.1.50-1+cuda11.6

$ sudo apt-get install libcudnn8-samples=8.4.1.50-1+cuda11.6

安裝 TensorRT 8.0.1

因為使用 Docker 安裝 DeepStream 6.1, 所以不用安裝 TensorRT 8.2.5.1

參考 https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html

在 TensorRT Archives 切換安裝文件版本

選擇 8.0.1 的 TensorRT Installation Guide

在 3. Downloading TensorRT

連結 https://developer.nvidia.com/tensorrt 按 GET STARTED

連結 https://developer.nvidia.com/tensorrt-getting-started 按 DOWNLOAD NOW

選擇 TensorRT 8

選擇 TensorRT 8.0 GA

選擇 TensorRT 8.0.1 GA for Ubuntu 18.04 and CUDA 11.3 DEB local repo package

$ sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626_1-1_amd64.deb

$ sudo apt-key add /var/nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626/7fa2af80.pub

$ sudo apt-get update

$ sudo apt --fix-broken install

$ sudo apt-get upgrade

$ sudo apt-get install tensorrt

安裝 DeepStream

因為不選擇最新的版本

參考 https://docs.nvidia.com/metropolis/deepstream-archive.html

選擇版本 6.0.1 DeepStream SDK Development Guid

到 dGPU Setup for Ubuntu

$ sudo apt install libssl1.0.0

$ sudo apt install libgstreamer1.0-0

$ sudo apt install gstreamer1.0-tools

$ sudo apt install gstreamer1.0-plugins-good

$ sudo apt install gstreamer1.0-plugins-bad

$ sudo apt install gstreamer1.0-plugins-ugly

$ sudo apt install gstreamer1.0-libav

$ sudo apt install libgstrtspserver-1.0-0

$ sudo apt install libjansson4

$ sudo apt install gcc

$ sudo apt install make

$ sudo apt install git

$ sudo apt install python3

$ cd /usr/bin

$ sudo ln -s python3 python

$ git clone https://github.com/edenhill/librdkafka.git

$ cd librdkafka

$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a

$ ./configure

$ make

$ sudo make install

$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-6.0/lib

$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-6.0/lib

下載 Deepstream 6.0 dGPU Debian package

https://developer.nvidia.com/deepstream-6.0_6.0.1-1_amd64deb

$ sudo apt-get install ./deepstream-6.0_6.0.1-1_amd64.deb

$ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin

$ cd /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/

$ deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt

安裝  Nvidia Docker

參考 https://docs.docker.com/engine/install/ubuntu/

$ sudo apt-get update

$ sudo apt-get install ca-certificates

$ sudo apt-get install curl

$ sudo apt-get install gnupg

$ sudo apt-get install lsb-release

$ sudo mkdir -p /etc/apt/keyrings

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

$ echo \

  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \

  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

$ sudo apt-get update

$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin

$ sudo docker run --rm hello-world

參考 https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker

$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \

      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \

      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \

            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \

            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

$ sudo apt-get update

$ sudo apt-get install -y nvidia-docker2

$ sudo systemctl restart docker

$ sudo groupadd docker

$ sudo usermod -a -G docker $USER

$ sudo reboot

安裝 NGC CLI

參考 https://ngc.nvidia.com/setup/installers/cli

$ wget --content-disposition https://ngc.nvidia.com/downloads/ngccli_linux.zip && \

  unzip ngccli_linux.zip && \

  chmod u+x ngc-cli/ngc

$ find ngc-cli/ -type f -exec md5sum {} + | LC_ALL=C sort | md5sum -c ngc-cli.md5

$ echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile && source ~/.bash_profile

$ ngc config set

# 直接 enter 即可

$ docker login nvcr.io

Username: $oauthtoken

Password: <Your API Key>

使用 Docker 安裝 TensorRT OSS

開啟 https://github.com/nvidia/TensorRT

切換至 Tags 8.0.1

$ git clone -b master https://github.com/nvidia/TensorRT TensorRT_OSS-8.0.1

$ cd TensorRT_OSS-8.0.1/

$ git describe --tags

8.2.0-EA-2-g96e2397

$ git tag -l

$ git branch -r

$ git checkout 8.0.1

$ git log -1

$ git describe --tags

8.0.1

$ git submodule update --init --recursive

$ vi docker/ubuntu-18.04.Dockerfile

修改下列一行

RUN cd /usr/local/bin && wget https://ngc.nvidia.com/downloads/ngccli_cat_linux.zip && \

  unzip ngccli_cat_linux.zip && chmod u+x ngc-cli/ngc && \

  rm ngccli_cat_linux.zip ngc-cli.md5 && echo "no-apikey\nascii\n" | ngc-cli/ngc config set

$ cat docker/ubuntu-18.04.Dockerfile | grep CUDA_VERSION

ARG CUDA_VERSION=11.3.1

$ ./docker/build.sh --file docker/ubuntu-18.04.Dockerfile --tag tensorrt-ubuntu18.04-cuda11.3 --cuda 11.3.1

$ ./docker/launch.sh --tag tensorrt-ubuntu18.04-cuda11.3 --gpus all

/workspace$ cd $TRT_OSSPATH

/workspace/TensorRT$ mkdir -p build && cd build

/workspace/TensorRT/build$ cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out

/workspace/TensorRT/build$ make -j$(nproc)

/workspace/TensorRT/build$ exit

$ mkdir backup

$ sudo mv /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.8.0.1 backup/

$ sudo cp build/out/libnvinfer_plugin.so.8.0.1 /usr/lib/x86_64-linux-gnu/

另外還要再安裝 TensorRT 8.2.1 給 Docker Ubuntu 20.04

將 8.0.1 改成 8.2.1, 18.04 改成 20.04, 11.3.1 改成 11.4.2, 但不需安裝

用 Docker 開發 DeepStream 6.0.1

參考 https://docs.nvidia.com/metropolis/deepstream/6.0.1/dev-guide/text/DS_docker_containers.html

$ docker pull nvcr.io/nvidia/deepstream:6.0.1-devel

$ xhost +

access control disabled, clients can connect from any host

$ sudo docker run --gpus all -it --rm --net=host \

  -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime \

  -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel

# update-alternatives --display cuda

# cat /etc/os-release

# cd samples/configs/deepstream-app

# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

# cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-app

# export CUDA_VER=11.4

# make

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# gst-launch-1.0 uridecodebin uri=$URI ! nvvideoconvert ! nveglglessink

# exit

用 Docker 開發 DeepStream 6.1

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_docker_containers.html

$ docker pull nvcr.io/nvidia/deepstream:6.1-devel

$ xhost +

access control disabled, clients can connect from any host

$ sudo docker run --gpus all -it --rm --net=host \

  -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime \

  -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.1 nvcr.io/nvidia/deepstream:6.1-devel

# cd samples/configs/deepstream-app

# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

# cd /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-app

# export CUDA_VER=11.6

# make

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.1/lib/gst-plugins/

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# gst-launch-1.0 uridecodebin uri=$URI ! nvvideoconvert ! nveglglessink

# exit

用 Docker DeepStream 6.0.1 測試 Integrate TAO model with DeepStream SDK

$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git deepstream_tao_apps-tao3.0_ds6.0.1

$ sudo apt install gitk

$ cd deepstream_tao_apps-tao3.0_ds6.0.1/

$ git branch -r

$ git checkout release/tao3.0_ds6.0.1

$ git log -1

$ sudo docker run --gpus all -it --rm --net=host \

> -v /etc/localtime:/etc/localtime \

> -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \

> -v /home/mark/Data/TensorRT/TensorRT_OSS-8.0.1/:/home/TensorRT \

> -v /home/mark/Data/DeepStream/deepstream_tap_apps/deepstream_tao_apps-tao3.0_ds6.0.1/:/home/deepstream_tao_apps \

> -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel

# cd /home/deepstream_tao_apps/

# ./download_models.sh 

# export CUDA_VER=11.4

# make

# cp /home/TensorRT/build/out/libnvinfer_plugin.so.8.0.1 /usr/lib/x86_64-linux-gnu/

# ./apps/tao_detection/ds-tao-detection -c configs/frcnn_tao/pgie_frcnn_tao_config.txt -i /opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.h264 -d

# cd /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-nvdsvideotemplate/

# make

# cp libnvdsgst_videotemplate.so /opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/

# rm -rf ~/.cache/gstreamer-1.0/

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.0/lib/cvcore_libs/

# cd /home/deepstream_tao_apps/apps/tao_others/

# make

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# cd deepstream-bodypose2d-app/

# ./deepstream-bodypose2d-app 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./body2dout

# cd ../deepstream-emotion-app/

# ./deepstream-emotion-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-faciallandmark-app/

# ./deepstream-faciallandmark-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-gaze-app/

# ./deepstream-gaze-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./gazenet

# cd ../deepstream-gesture-app/

# ./deepstream-gesture-app 3 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./gesture

# cd ../deepstream-heartrate-app/

# ./deepstream-heartrate-app 3 $URI ./heartrate

# exit

用 Docker DeepStream 6.1 測試 Integrate TAO model with DeepStream SDK

$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git deepstream_tao_apps-tao3.0_ds6.1ga

$ sudo apt install gitk

$ cd deepstream_tao_apps-tao3.0_ds6.1ga/

$ git branch -r

$ git checkout release/tao3.0_ds6.1ga

$ git log -1

$ sudo docker run --gpus all -it --rm --net=host \

> -v /etc/localtime:/etc/localtime \

> -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \

> -v /home/mark/Data/TensorRT/TensorRT_OSS-8.2.1/:/home/TensorRT \

> -v /home/mark/Data/DeepStream/deepstream_tap_apps/deepstream_tao_apps-tao3.0_ds6.1ga/:/home/deepstream_tao_apps \

> -w /opt/nvidia/deepstream/deepstream-6.1 nvcr.io/nvidia/deepstream:6.1-devel

# cp /home/TensorRT/build/out/libnvinfer_plugin.so.8.2.1 /usr/lib/x86_64-linux-gnu/

# cd /usr/lib/x86_64-linux-gnu/

# rm libnvinfer_plugin.so.8

# ln -s libnvinfer_plugin.so.8.2.1 libnvinfer_plugin.so.8

# cd /home/deepstream_tao_apps/

# ./download_models.sh 

# export CUDA_VER=11.6

# make

# ./apps/tao_detection/ds-tao-detection -c configs/frcnn_tao/pgie_frcnn_tao_config_dgpu.txt -i /opt/nvidia/deepstream/deepstream-6.1/samples/streams/sample_720p.h264 -d

# cd /opt/nvidia/deepstream/deepstream-6.1/sources/gst-plugins/gst-nvdsvideotemplate/

# make

# cp libnvdsgst_videotemplate.so /opt/nvidia/deepstream/deepstream-6.1/lib/gst-plugins/

# rm -rf ~/.cache/gstreamer-1.0/

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.1/lib/cvcore_libs/

# cd /home/deepstream_tao_apps/apps/tao_others/

# make

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# cd deepstream-bodypose2d-app/

# ./deepstream-bodypose2d-app 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt 0 0 $URI ./body2dout

# cd ../deepstream-emotion-app/

# ./deepstream-emotion-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-faciallandmark-app/

# ./deepstream-faciallandmark-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-gaze-app/

# ./deepstream-gaze-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./gazenet

# cd ../deepstream-gesture-app/

# ./deepstream-gesture-app 3 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./gesture

# cd ../deepstream-heartrate-app/

# ./deepstream-heartrate-app 3 $URI ./heartrate

# exit

安裝其他元件

參考 Ubuntu 18.04 重灌 上的 安裝 CMake 和 安裝 OpenCV

apt-get update error

# apt-get update

Err:1 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease

  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC

.

.

.

W: GPG error: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC

E: The repository 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease' is no longer signed.

# apt-key del 7fa2af80

# apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub

# apt-get update

CUDNN 安裝

顯卡 驅動 CUDA CUDNN 版本關係

https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html

安裝

https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

install pycuda

sudo pip3 install --global-option=build_ext --global-option="-I/usr/local/cuda-10.2/targets/aarch64-linux/include/" --global-option="-L/usr/local/cuda-10.2/targets/aarch64-linux/lib/" pycuda

https://docs.donkeycar.com/guide/robot_sbc/tensorrt_jetson_nano/

Install Tensorflow 1.15 on Ubuntu 18.04

原本 pip3 install tensorflow-gpu=1.15 即可
但發現與 cuda 10.2 不合， 需要安裝 cuda 10.0，搭配 cudnn v7.6.5

安裝 CUDA

https://developer.nvidia.com/cuda-downloads

選 Archive of Previous CUDA Releases

選 CUDA Toolkit 10.0, Linux, x86_64, Ubuntu, 18.04, deb(local)

按下 Download

sudo dpkg -i cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64.deb

sudo apt-key add /var/cuda-repo-10-0-local-10.0.130-410.48/7fa2af80.pub

sudo apt-get update

查詢可安裝版本

apt-cache policy cuda

apt-cache madison cuda

安裝正確版本

sudo apt-get install cuda=10.0.130-1

安裝 CUDNN

https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

https://developer.nvidia.com/cudnn

選擇 cuDNN v7.6.5 for CUDA 10.0

選擇 cuDNN Library for Linux (x86)

tar -xzvf cudnn-10.0-linux-x64-v7.6.5.32.tgz

sudo cp cuda/include/cudnn*.h /usr/local/cuda-10.0/include

sudo cp cuda/lib64/libcudnn* /usr/local/cuda-10.0/lib64

sudo chmod a+r /usr/local/cuda-10.0/include/cudnn*.h /usr/local-10.0/cuda/lib64/libcudnn*

sudo apt install python3-testresources

sudo apt install python-dev python-pip

mkdir envs; cd envs

python3 -m venv --system-site-packages tensorflow-1.15

source tensorflow-1.15/bin/activate

pip install --upgrade pip

pip install tensorflow-gpu==1.15

Ubuntu 18.04 重灌

依據 https://developer.nvidia.com/deepstream-getting-started

選擇各個版本

https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html

sudo apt-get install ssh

sudo apt install python3-pip

sudo ln -s /usr/bin/python3 /usr/bin/python

安裝 NVIDIA Driver

https://www.linuxbabe.com/ubuntu/install-nvidia-driver-ubuntu-18-04

sudo lshw -c display

sudo lshw -c video

sudo ubuntu-drivers devices

sudo ubuntu-drivers autoinstall

sudo reboot

sudo lshw -c display

安裝 CUDA Toolkit

https://developer.nvidia.com/cuda-downloads

請選擇 CUDA Toolkit 10.2

Linux/x86_64/Ubuntu/18.04/deb(local)

vi ~/.bashrc

export PATH=/usr/local/cuda-10.2:$PATH

export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64:$LD_LIBRARY_PATH

安裝 CUDNN

https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

https://developer.nvidia.com/cudnn

選擇 cuDNN v8.0.5 for CUDA 10.2

選擇 cuDNN Library for Linux (x86)

tar -xzvf cudnn-10.2-linux-x64-v8.0.5.39.tgz

sudo cp cuda/include/cudnn*.h /usr/local/cuda/include

sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64

sudo chmod a+r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn*

sudo ldconfig

/sbin/ldconfig.real: /usr/local/cuda-10.2/targets/x86_64-linux/liblibcudnn.so.8 is not a symbolic link

cd /usr/local/cuda-10.2/targets/x86_64-linux/lib

sudo rm liblibcudnn.so.8 liblibcudnn.so

sudo ln -s libcudnn.so.8.0.5 libcudnn.so.8

sudo ln -s libcudnn.so.8 libcudnn.so

sudo ldconfig

安裝 TensorRT

下載 TensorRT binary

https://developer.nvidia.com/nvidia-tensorrt-7x-download

選擇 TensorRT 7.1.3.4 for Ubuntu 18.04 and CUDA 10.2 TAR package

version="7.1.3.4"

os="Ubuntu-18.04"

arch=$(uname -m)

cuda="cuda-10.2"

cudnn="cudnn8.0"

tar -xvzf TensorRT-7.1.3.4.Ubuntu-18.04.x86_64-gnu.cuda-10.2.cudnn8.0.tar.gz

vi ~/.bashrc

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/Data/TensorRT/TensorRT-7.1.3.4/lib

source ~/.bashrc

cd TensorRt-7.1.3.4/python

sudo pip3 install tensorrt-7.1.3.4-cp36-none-linux_x86_64.whl

cd ../uff

sudo pip3 install uff-0.6.9-py2.py3-none-any.whl

cd ../graphsurgeon/

sudo pip3 install graphsurgeon-0.4.5-py2.py3-none-any.whl

安裝 GStreamer

https://yingrenn.blogspot.com/2020/09/gstreamer.html

安裝 DeepStream

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Quickstart.html#dgpu-setup-for-ubuntu

sudo apt install \

 libssl1.0.0 \

 libgstreamer1.0-0 \

 gstreamer1.0-tools \

 gstreamer1.0-plugins-good \

 gstreamer1.0-plugins-bad \

 gstreamer1.0-plugins-ugly \

 gstreamer1.0-libav \

 libgstrtspserver-1.0-0 \

 libgstrtspserver-1.0-dev \

 libjansson4

git clone https://github.com/edenhill/librdkafka.git

cd librdkafka

git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a

./configure

make

sudo make install

sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.0/lib

sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.0/lib

下載 DeepStream

https://developer.nvidia.com/assets/Deepstream/5.0/ga/secure/deepstream_sdk_5.0.1_x86_64.tbz2

sudo tar -xvf deepstream_sdk_v5.0.1_x86_64.tbz2 -C /

cd /opt/nvidia/deepstream/deepstream-5.0/

sudo ./install.sh

sudo ldconfig

安裝 CMake v3.13

wget http://www.cmake.org/files/v3.13/cmake-3.13.5.tar.gz

tar xpvf cmake-3.13.5.tar.gz cmake-3.13.5

cd cmake-3.13.5

sudo apt-get install zlib1g-dev

sudo apt-get install curl

sudo apt-get install libcurl3

sudo apt-get install libcurl4 libcurl4-openssl-dev

./bootstrap --system-curl

make -j$(nproc)

sudo make install

安裝 Docker

https://docs.docker.com/engine/install/ubuntu/

sudo apt-get update

sudo apt-get install \

    apt-transport-https \

    ca-certificates \

    curl \

    gnupg-agent \

    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo apt-key fingerprint 0EBFCD88

sudo add-apt-repository \

   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \

   $(lsb_release -cs) \

   stable"

sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io

安裝 NVIDIA Docker

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker

curl https://get.docker.com | sh \

  && sudo systemctl start docker \

  && sudo systemctl enable docker

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \

   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \

   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update

sudo apt-get install -y nvidia-docker2

sudo systemctl restart docker

sudo groupadd docker

sudo usermod -a -G docker $USER

sudo reboot

若 sudo apt update 出現

W: Target CNF (stable/cnf/Commands-all) is configured multiple times in /etc/apt/sources.list:52 and /etc/apt/sources.list.d/docker.list:1

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4773BD5E130D1D45

sudo rm /etc/apt/sources.list.d/docker.list

安裝 TensorRT 7.1 OSS

https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html

https://github.com/NVIDIA/TensorRT/tree/master

從 master 切換到 release/7.1

下載 TensorRT OSS

TensorRT OSS: 包含 TensorRT plugins, Caffe 和 ONNX parsers 等

git clone -b master https://github.com/nvidia/TensorRT TensorRT

cd TensorRT

git submodule update --init --recursive

參考之前的 TensorRT binary

cd TensorRT-7.1.3.4

export TRT_RELEASE=`pwd`

cd $TRT_SOURCE

./docker/build.sh --file docker/ubuntu.Dockerfile --tag tensorrt-ubuntu --os 18.04 --cuda 10.2

./docker/launch.sh --tag tensorrt-ubuntu --gpus all --release $TRT_RELEASE --source $TRT_SOURCE

cd $TRT_SOURCE

mkdir -p build && cd build

cmake .. -DTRT_LIB_DIR=$TRT_RELEASE/lib -DTRT_OUT_DIR=`pwd`/out

make -j$(nproc)

exit

mkdir backup

mv $TRT_RELEASE/targets/x86_64-linux-gnu/lib/libnvinfer_plugin.so.7.1.3 backup

cp $TRT_SOURCE/build/out/libnvinfer_plugin.so.7.2.1 $TRT_RELEASE/targets/x86_64-linux-gnu/lib/libnvinfer_plugin.so.7.1.3

安裝 OpenCV 4.4

git clone https://github.com/opencv/opencv.git opencv-4.4.0 -b 4.4.0 --depth 1

git clone https://github.com/opencv/opencv_contrib.git opencv_contrib-4.4.0 -b 4.4.0 --depth 1

sudo apt-get update

sudo add-apt-repository ppa:alex-p/tesseract-ocr

sudo apt install tesseract-ocr libtesseract-dev

sudo apt-get install libleptonica-dev

sudo apt-get install qt5-default

sudo apt-get install qtcreator

sudo apt-get install build-essential cmake unzip pkg-config

sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev

sudo apt-get install libxvidcore-dev libx264-dev

sudo apt-get install libgtk-3-dev

sudo apt-get install libatlas-base-dev gfortran

sudo apt-get install libhdf5-dev

sudo apt-get install python3-dev

mkdir build; cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \

-D CMAKE_INSTALL_PREFIX=/usr/local \

-D INSTALL_PYTHON_EXAMPLES=ON \

-D INSTALL_C_EXAMPLES=OFF \

-D WITH_TBB=ON \

-D BUILD_opencv_cudacodec=OFF \

-D ENABLE_FAST_MATH=1 \

-D CUDA_FAST_MATH=1 \

-D WITH_CUDA=ON \

-D WITH_CUBLAS=ON \

-D WITH_LIBV4L=ON \

-D BUILD_opencv_python3=ON \

-D BUILD_opencv_python2=OFF \

-D BUILD_opencv_java=OFF \

-D WITH_V4L=ON \

-D WITH_QT=ON \

-D WITH_OPENGL=ON \

-D WITH_GSTREAMER=ON \

-D WITH_GTK=ON \

-D OPENCV_GENERATE_PKGCONFIG=ON \

-D OPENCV_PC_FILE_NAME=opencv.pc \

-D OPENCV_ENABLE_NONFREE=OFF \

-D OPENCV_EXTRA_MODULES_PATH=/your_path_to/opencv/opencv_contrib-4.4.0/modules \

-D BUILD_EXAMPLES=ON \

-D WITH_CUDNN=ON \

-D CUDNN_VERSION="8.0.5" \

-D OPENCV_DNN_CUDA=ON \

-D CUDNN_INCLUDE_DIR=/usr/local/cuda/include \

-D CUDNN_LIBRARY=/usr/local/cuda/lib64/libcudnn.so.8.0.5 \

-D CUDA_ARCH_BIN=7.5 \

../opencv-4.4.0

#-D OPENCV_ENABLE_NONFREE=ON \

#-D OpenGL_GL_PREFERENCE=LEGACY \

make -j$(nproc)

sudo make install

sudo ldconfig

opencv_version -v

下載 Tesseract 的中文字 chi_tra.traineddata, chi_tra_vert.traineddate

https://github.com/tesseract-ocr/tessdata

放至 /usr/share/tesseract-ocr/4.00/tessdata

安裝 Xpra

https://www.xpra.org/trac/wiki/Building/Debian

sudo apt-get install libx11-dev libxtst-dev libxcomposite-dev libxdamage-dev \

 libxkbfile-dev python-all-dev

sudo apt-get install libgtk-3-dev python3-dev python3-cairo-dev python-gi-dev cython3

sudo apt-get install xauth x11-xkb-utils

sudo apt-get install libx264-dev libvpx-dev yasm

sudo apt-get install libnvidia-encode-440

sudo apt-get install libavformat-dev libavcodec-dev libswscale-dev

sudo apt-get install libturbojpeg-dev

sudo apt-get install libwebp-dev

sudo apt-get install uglifyjs brotli libjs-jquery libjs-jquery-ui gnome-backgrounds

sudo apt-get install python3-opengl python3-numpy python3-pil

sudo apt-get install python3-rencode python3-lz4 python3-dbus python3-cryptography \

 python3-netifaces python3-yaml

sudo apt-get install python3-setproctitle python3-xdg python3-pyinotify python3-opencv

sudo apt-get install libpam-dev quilt xserver-xorg-dev xutils-dev xvfb keyboard-configuration

sudo apt-get install python3-kerberos python3-gssapi

sudo apt-get install gstreamer1.0-pulseaudio gstreamer1.0-alsa \

 gstreamer1.0-plugins-base gstreamer1.0-plugins-good \

 gstreamer1.0-plugins-ugly

sudo apt-get install cups-filters cups-common cups-pdf python3-cups

sudo apt-get install openssh-client sshpass python3-paramiko

sudo apt-get install devscripts build-essential lintian debhelper

wget https://www.xpra.org/src/xpra-4.0.4.tar.xz

tar -xf xpra-4.0.6.tar.xz

cd xpra-4.0.6

vi setup.py

#!/usr/bin/env python3

sudo ./setup.py install

安裝 frp

https://yingrenn.blogspot.com/2020/03/frp.html

vi frps.ini

[common]

bind_port = 7000

dashboard_port = 7001

dashboard_user = user

dashboard_pwd = password

vi /etc/systemd/system/frps.service

[Unit]

Description=FRP Server Daemon

[Service]

Type=simple

ExecStartPre=-/usr/sbin/setcap cap_net_bind_service=+ep /home/mark/Data/frp/frp_0.34.3_linux_amd64/frps

ExecStart=/path_to_frp/frp_0.34.3_linux_amd64/frps -c /path_to_frp/frp_0.34.3_linux_amd64/frps.ini

Restart=always

RestartSec=20s

User=nobody

PermissionsStartOnly=true

LimitNOFILE=infinity

[Install]

WantedBy=multi-user.target

sudo systemctl start frps

sudo systemctl enable ftps

安裝 Gitea Tomcat Nginx

https://yingrenn.blogspot.com/2019/11/gitea-in-ubuntu.html

https://yingrenn.blogspot.com/search/label/tomcat

安裝 x11vnc

因為 Ubuntu 18.04 改 LightDM 到 GDM3, 所以要在 console login 後，才能動作

sudo apt install x11vnc

sudo x11vnc -storepasswd

sudo chown mark.mark ~/.vnc/passwd

sudo vi /etc/systemd/system/x11vnc.service

# Description: Custom Service Unit file

# File: /etc/systemd/system/x11vnc.service

[Unit]

Description="x11vnc"

Requires=display-manager.service

After=display-manager.service

[Service]

ExecStart=/usr/bin/x11vnc -loop -nopw -xkb -repeat -noxrecord -noxfixes -noxdamage -forever -rfbport 5900 -display :1 -auth /run/user/1000/gdm/Xauthority -rfbauth /home/mark/.vnc/passwd

ExecStop=/usr/bin/killall x11vnc

Restart=on-failure

RestartSec=2

[Install]

WantedBy=multi-user.target

安裝 VirtualBox

https://www.virtualbox.org/wiki/Downloads

到官網下載

virtualbox-6.1_6.1.16-140961~Ubuntu~bionic_amd64.deb

Oracle_VM_VirtualBox_Extension_Pack-6.1.16.vbox-extpack

sudo dpkg -i virtualbox-6.1_6.1.16-140961~Ubuntu~bionic_amd64.deb

sudo groupadd win10disk

sudo usermod -a -G win10disk youruser

sudo udevadm info /dev/sdX | grep UUID

E: ID_PART_TABLE_UUID=01234567-89ab-cdef-0123-456789abcde

vi /etc/udev/rules.d/99-win10disk.rules

ENV{ID_PART_TABLE_UUID}=="1234567-89ab-cdef-0123-456789abcde", GROUP="win10disk"

ls -l /dev/sdb

brw-rw---- 1 root win10disk 8, 16 Nov 4 23:33 /dev/sdb

VBoxManage internalcommands createrawvmdk -filename .VirtualBox/Crucial1T.vmdk -rawdisk /dev/sda

sudo usermod -a -G vboxusers youruser

拷貝舊的 VM, 修改 win10.vbox, win10.vbox-prev 內的 uuid

與 .VirtualBox/Crucial1T.vmdk 的 uuid 一致

開啟 VirtualBox

File/Preferences...

Extensions 按 +

選擇剛下載的 Oracle_VM_VirtualBox_Extension_Pack-6.1.16.vbox-extpack

只有登入畫面無法使用滑鼠和鍵盤，其餘正常

sudo apt-get instll xserver-xorg-input-all

Build OpenCV on Jetson NANO with CUDA CUDNN

參考 How to configure your NVIDIA Jetson Nano for Computer Vision and Deep Learning

make 時出現下列錯誤

error: 'CUDNN_CONVOLUTION_FWD_PREFER_FASTEST' was not declared in this scope

error: 'CUDNN_CONVOLUTION_BWD_PREFER_FASTEST' was not declared in this scope

原因為 OpenCV 4.4 以前，並不支援 CUDNN8.0

CUDNN 8.0 只能在 OpenCV 4.4 才能 build 成功

$ git clone https://github.com/opencv/opencv.git -b 4.4.0 --depth 1

$ git clone https://github.com/opencv/opencv_contrib.git -b 4.4.0 --depth 1

刪除舊的安裝

$ sudo apt purge libopencv-dev libopencv-python libopencv-samples libopencv*

$ sudo apt update

參考 Any OCR model run in Jetson Nano

參考 Installing tesseract 4.0 on Ubuntu 16.04

$ sudo apt install tesseract-ocr libtesseract-dev

參考 Jetson_Nano_OpenCV410_QT5

$ sudo add-apt-repository ppa:alex-p/tesseract-ocr

$ sudo apt-get update

$ sudo apt-get install libleptonica-dev

$ sudo apt-get install qt5-default

$ sudo apt-get install qtcreator

cmake \

 -D CMAKE_BUILD_TYPE=RELEASE \

 -D CMAKE_INSTALL_PREFIX=/usr/local \

 -D WITH_CUDA=ON \

 -D WITH_CUDNN=ON \

 -D WITH_CUBLAS=ON \

 -D WITH_V4L=ON \

 -D CUDNN_VERSION='8.0' \

 -D CUDNN_INCLUDE_DIR='/usr/include' \

 -D OPENCV_DNN_CUDA=ON \

 -D WITH_GTK=ON \

 -D WITH_QT=ON \

 -D INSTALL_C_EXAMPLES=OFF \

 -D WITH_GSTREAMER=ON \

 -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.4.0/modules/ \

 -D OPENCV_ENABLE_NONFREE=ON \

 -D BUILD_TESTS=OFF \

 -D BUILD_PERF_TESTS=OFF \

 -D BUILD_EXAMPLES=OFF \

 -D INSTALL_PYTHON_EXAMPLES=ON \

 -D PYTHON_EXECUTABLE=$(which python3) \

 -D BUILD_opencv_python2=OFF \

 -D BUILD_opencv_python3=ON \

 -D BUILD_opencv_java=OFF \

 -D PYTHON3_EXECUTABLE=$(which python3) \

 -D PYTHON3_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \

 -D PYTHON3_PACKAGES_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \

 -D Tesseract_INCLUDE_DIR=/usr/include/tesseract \

 -D Tesseract_LIBRARY=/usr/lib/aarch64-linux-gnu/libtesseract.so \

 -D Lept_LIBRARY=/usr/lib/aarch64-linux-gun/liblept.so \

 -D CUDA_ARCH_PTX="" \

 -D CUDA_ARCH_BIN="5.3,6.2,7.2" \

 -D OPENCV_GENERATE_PKGCONFIG=YES ..

$ make -j4

$ sudo make install

$ sudo ldconfig

$ opencv_version -v

安裝於 python 的 virtualenv 中

nano@nano-desktop:~/envs/py3cv4/lib/python3.6/site-packages$ ln -s /usr/lib/python3/dist-packages/cv2/python-3.6/cv2.cpython-36m-aarch64-linux-gnu.so cv2.so

測試 參考 如何讓Jetson Nano順利執行OpenCV中CUDA函式

在 mainwindow.cpp 檔內增加

#include <opencv2/opencv.hpp>

#include <opencv2/cudaimgproc.hpp>

測試二 參考 How to use OpenCV’s “dnn” module with NVIDIA GPUs, CUDA, and cuDNN

OpenCV ‘dnn’ with NVIDIA GPUs: 1549% faster YOLO, SSD, and Mask R-CNN

SSD 測試

(py3cv4) nano@nano-desktop:~/Data/opencv/opencv-dnn-gpu-examples$ python opencv-ssd-cuda/ssd_object_detection.py --prototxt opencv-ssd-cuda/MobileNetSSD_deploy.prototxt --model opencv-ssd-cuda/MobileNetSSD_deploy.caffemodel --input example_videos/guitar.mp4 --display 0 --use-gpu=1

[INFO] setting preferable backend and target to CUDA...

[INFO] accessing video stream...

[INFO] elasped time: 34.94

[INFO] approx. FPS: 7.07

(py3cv4) nano@nano-desktop:~/Data/opencv/opencv-dnn-gpu-examples$ python opencv-ssd-cuda/ssd_object_detection.py --prototxt opencv-ssd-cuda/MobileNetSSD_deploy.prototxt --model opencv-ssd-cuda/MobileNetSSD_deploy.caffemodel --input example_videos/guitar.mp4 --display 0

[INFO] accessing video stream...

[INFO] elasped time: 81.08

[INFO] approx. FPS: 3.05

Yolo 測試

(py3cv4) nano@nano-desktop:~/Data/opencv/opencv-dnn-gpu-examples$ python3 opencv-yolo-cuda/yolo_object_detection.py -y opencv-yolo-cuda/yolo-coco -i example_videos/guitar.mp4 --display=0 --use-gpu=0

[INFO] loading YOLO from disk...

[INFO] setting preferable backend and target to CUDA...

[INFO] accessing video stream...

[INFO] elasped time: 212.71

[INFO] approx. FPS: 1.16

(py3cv4) nano@nano-desktop:~/Data/opencv/opencv-dnn-gpu-examples$ python3 opencv-yolo-cuda/yolo_object_detection.py -y opencv-yolo-cuda/yolo-coco -i example_videos/guitar.mp4 --display=0

[INFO] loading YOLO from disk...

[INFO] accessing video stream...

[INFO] elasped time: 1275.95

[INFO] approx. FPS: 0.19

Mask R-CNN 測試

(py3cv4) nano@nano-desktop:~/Data/opencv/opencv-dnn-gpu-examples$ python3 opencv-mask-rcnn-cuda/mask_rcnn_segmentation.py --mask-rcnn opencv-mask-rcnn-cuda/mask-rcnn-coco/ --input example_videos/guitar.mp4 --display=0 --use-gpu=1

[INFO] loading Mask R-CNN from disk...

[INFO] setting preferable backend and target to CUDA...

[INFO] accessing video stream...

[INFO] elasped time: 445.18

[INFO] approx. FPS: 0.55

(py3cv4) nano@nano-desktop:~/Data/opencv/opencv-dnn-gpu-examples$ python3 opencv-mask-rcnn-cuda/mask_rcnn_segmentation.py --mask-rcnn opencv-mask-rcnn-cuda/mask-rcnn-coco/ --input example_videos/guitar.mp4 --display=0

[INFO] loading Mask R-CNN from disk...

[INFO] accessing video stream...

[INFO] elasped time: 4512.29

[INFO] approx. FPS: 0.05

CUDA 安裝失敗

CUDA 安裝失敗，通常是由於 Visual Studio Integration 失敗
所以透過自訂安裝，跳過不安裝 Visual Studio Integration, 可以安裝成功
Installer Type 要選擇 exe(local)

而 Visual Studio Integration 的安裝方式如下：
1. 使得可以編譯 CUDA 程式
注意安裝 CUDA 時的路徑，拷貝出 CUDAVisualStudioIntegration 目錄夾
將 D:\CUDAVisualStudioIntegration\extras\visual_studio_integration\MSBuildExtensions
目錄下所有檔案拷貝至
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\BuildCustomizations
2. 使得 Visual Studio 可以新建 CUDA 專案
將目錄
D:\CUDAVisualStudioIntegration\extras\visual_studio_integration\CudaProjectVsWizards
拷貝至
C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions
3. 安裝
D:\CUDAVisualStudioIntegration\NVIDIA_Nsight_Visual_Studio_Edition_Win64_5.4.0.17229.msi