Ubuntu 18.04 重灌

因為 Ubuntu 18.04 只能安裝 DeepStream 6.0.1

在 Ubuntu 20.04 才能安裝 DeepStream 6.1

所以在 Ubuntu 18.04 上安裝 Docker Ubuntu 20.04

版本對應如下

===========================

DS 6.1

Ubuntu 20.04

GCC 9.4.0

CUDA 11.6.1

cuDNN 8.4.0.27

TRT 8.2.5.1

Display Driver:R510.47.03

GStreamer 1.16.2

OpenCV 4.2.0

deepstream:6.1

===========================

DS 6.0.1

Ubuntu 18.04

GCC 7.3.0

CUDA 11.4.1

cuDNN 8.2+

TRT 8.0.1

Display Driver:R470.63.01

GStreamer 1.14.5

OpenCV 3.4.0

deepstream:6.0.1

===========================

安裝作業系統

BIOS 選擇開機

Install Ubuntu

Installation type 選 Something else

Create partition/Mount point 選擇 /

Settings/Details/Users

Unlock

Automatic Login: ON

更新系統，安裝一些常用套件

$ sudo apt-get update

$ sudo apt-get upgrade

$ sudo apt-get install ssh

$ sudo apt-get install python3-pip

mount nfs

$ sudo apt-get install nfs-common

$ sudo mount -t nfs ip:/share_folder /mount_folder

$ sudo vi /etc/fstab

#中間的空格要使用 tab

ip:/share_folder /mnt/mount_folder nfs defaults,bg 0 0

$ sudo mount -a

vnc

$ sudo apt-get install x11vnc

$ sudo x11vnc -storepasswd

$ sudo chown user.group ~/.vnc/passwd

安裝顯示卡驅動，CUDA 和 CUDNN

$ sudo ubuntu-drivers devices

$ sudo apt-get install nvidia-driver-510

https://developer.nvidia.com/cuda-downloads

選擇 Archive of Previous CUDA Releases

選擇 CUDA Toolkit 11.4.1

選擇 deb(local), deb(network) 不可使用了

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin

$ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600

$ wget https://developer.download.nvidia.com/compute/cuda/11.4.1/local_installers/cuda-repo-ubuntu1804-11-4-local_11.4.1-470.57.02-1_amd64.deb

$ sudo dpkg -i cuda-repo-ubuntu1804-11-4-local_11.4.1-470.57.02-1_amd64.deb

$ sudo apt-key add /var/cuda-repo-ubuntu1804-11-4-local/7fa2af80.pub

$ sudo apt-get update

$ sudo apt-get -y install cuda-11-4

之後再安裝一遍 CUDA Toolkit 11.3.1(因為 TensorRT 8.0.1 需要 CUDA-11-3)

$ sudo apt-get -y install cuda-11-3

之後再安裝一遍 CUDA Toolkit 11.6.1(因為 DeepStream 6.1 需要 CUDA-11-6)

$ sudo apt-get -y install cuda-11-6

$ update-alternatives --display cuda

$ update-alternatives --config cuda

參考 https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

到 2.2. Downloading cuDNN for Linux

下載 Local Install for Ubuntu18.04 x86_64(Deb)

$ sudo apt-get install zlib1g

$ sudo dpkg -i cudnn-local-repo-ubuntu1804-8.4.1.50_1.0-1_amd64.deb

$ sudo cp /var/cudnn-local-repo-ubuntu1804-8.4.1.50/cudnn-local-BA71F057-keyring.gpg /usr/share/keyrings/

$ sudo apt-get update

$ apt list -a libcudnn8

$ sudo apt-get install libcudnn8=8.4.1.50-1+cuda11.6

$ sudo apt-get install libcudnn8-dev=8.4.1.50-1+cuda11.6

$ sudo apt-get install libcudnn8-samples=8.4.1.50-1+cuda11.6

安裝 TensorRT 8.0.1

因為使用 Docker 安裝 DeepStream 6.1, 所以不用安裝 TensorRT 8.2.5.1

參考 https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html

在 TensorRT Archives 切換安裝文件版本

選擇 8.0.1 的 TensorRT Installation Guide

在 3. Downloading TensorRT

連結 https://developer.nvidia.com/tensorrt 按 GET STARTED

連結 https://developer.nvidia.com/tensorrt-getting-started 按 DOWNLOAD NOW

選擇 TensorRT 8

選擇 TensorRT 8.0 GA

選擇 TensorRT 8.0.1 GA for Ubuntu 18.04 and CUDA 11.3 DEB local repo package

$ sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626_1-1_amd64.deb

$ sudo apt-key add /var/nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626/7fa2af80.pub

$ sudo apt-get update

$ sudo apt --fix-broken install

$ sudo apt-get upgrade

$ sudo apt-get install tensorrt

安裝 DeepStream

因為不選擇最新的版本

參考 https://docs.nvidia.com/metropolis/deepstream-archive.html

選擇版本 6.0.1 DeepStream SDK Development Guid

到 dGPU Setup for Ubuntu

$ sudo apt install libssl1.0.0

$ sudo apt install libgstreamer1.0-0

$ sudo apt install gstreamer1.0-tools

$ sudo apt install gstreamer1.0-plugins-good

$ sudo apt install gstreamer1.0-plugins-bad

$ sudo apt install gstreamer1.0-plugins-ugly

$ sudo apt install gstreamer1.0-libav

$ sudo apt install libgstrtspserver-1.0-0

$ sudo apt install libjansson4

$ sudo apt install gcc

$ sudo apt install make

$ sudo apt install git

$ sudo apt install python3

$ cd /usr/bin

$ sudo ln -s python3 python

$ git clone https://github.com/edenhill/librdkafka.git

$ cd librdkafka

$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a

$ ./configure

$ make

$ sudo make install

$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-6.0/lib

$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-6.0/lib

下載 Deepstream 6.0 dGPU Debian package

https://developer.nvidia.com/deepstream-6.0_6.0.1-1_amd64deb

$ sudo apt-get install ./deepstream-6.0_6.0.1-1_amd64.deb

$ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin

$ cd /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/

$ deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt

安裝  Nvidia Docker

參考 https://docs.docker.com/engine/install/ubuntu/

$ sudo apt-get update

$ sudo apt-get install ca-certificates

$ sudo apt-get install curl

$ sudo apt-get install gnupg

$ sudo apt-get install lsb-release

$ sudo mkdir -p /etc/apt/keyrings

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

$ echo \

  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \

  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

$ sudo apt-get update

$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin

$ sudo docker run --rm hello-world

參考 https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker

$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \

      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \

      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \

            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \

            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

$ sudo apt-get update

$ sudo apt-get install -y nvidia-docker2

$ sudo systemctl restart docker

$ sudo groupadd docker

$ sudo usermod -a -G docker $USER

$ sudo reboot

安裝 NGC CLI

參考 https://ngc.nvidia.com/setup/installers/cli

$ wget --content-disposition https://ngc.nvidia.com/downloads/ngccli_linux.zip && \

  unzip ngccli_linux.zip && \

  chmod u+x ngc-cli/ngc

$ find ngc-cli/ -type f -exec md5sum {} + | LC_ALL=C sort | md5sum -c ngc-cli.md5

$ echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile && source ~/.bash_profile

$ ngc config set

# 直接 enter 即可

$ docker login nvcr.io

Username: $oauthtoken

Password: <Your API Key>

使用 Docker 安裝 TensorRT OSS

開啟 https://github.com/nvidia/TensorRT

切換至 Tags 8.0.1

$ git clone -b master https://github.com/nvidia/TensorRT TensorRT_OSS-8.0.1

$ cd TensorRT_OSS-8.0.1/

$ git describe --tags

8.2.0-EA-2-g96e2397

$ git tag -l

$ git branch -r

$ git checkout 8.0.1

$ git log -1

$ git describe --tags

8.0.1

$ git submodule update --init --recursive

$ vi docker/ubuntu-18.04.Dockerfile

修改下列一行

RUN cd /usr/local/bin && wget https://ngc.nvidia.com/downloads/ngccli_cat_linux.zip && \

  unzip ngccli_cat_linux.zip && chmod u+x ngc-cli/ngc && \

  rm ngccli_cat_linux.zip ngc-cli.md5 && echo "no-apikey\nascii\n" | ngc-cli/ngc config set

$ cat docker/ubuntu-18.04.Dockerfile | grep CUDA_VERSION

ARG CUDA_VERSION=11.3.1

$ ./docker/build.sh --file docker/ubuntu-18.04.Dockerfile --tag tensorrt-ubuntu18.04-cuda11.3 --cuda 11.3.1

$ ./docker/launch.sh --tag tensorrt-ubuntu18.04-cuda11.3 --gpus all

/workspace$ cd $TRT_OSSPATH

/workspace/TensorRT$ mkdir -p build && cd build

/workspace/TensorRT/build$ cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out

/workspace/TensorRT/build$ make -j$(nproc)

/workspace/TensorRT/build$ exit

$ mkdir backup

$ sudo mv /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.8.0.1 backup/

$ sudo cp build/out/libnvinfer_plugin.so.8.0.1 /usr/lib/x86_64-linux-gnu/

另外還要再安裝 TensorRT 8.2.1 給 Docker Ubuntu 20.04

將 8.0.1 改成 8.2.1, 18.04 改成 20.04, 11.3.1 改成 11.4.2, 但不需安裝

用 Docker 開發 DeepStream 6.0.1

參考 https://docs.nvidia.com/metropolis/deepstream/6.0.1/dev-guide/text/DS_docker_containers.html

$ docker pull nvcr.io/nvidia/deepstream:6.0.1-devel

$ xhost +

access control disabled, clients can connect from any host

$ sudo docker run --gpus all -it --rm --net=host \

  -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime \

  -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel

# update-alternatives --display cuda

# cat /etc/os-release

# cd samples/configs/deepstream-app

# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

# cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-app

# export CUDA_VER=11.4

# make

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# gst-launch-1.0 uridecodebin uri=$URI ! nvvideoconvert ! nveglglessink

# exit

用 Docker 開發 DeepStream 6.1

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_docker_containers.html

$ docker pull nvcr.io/nvidia/deepstream:6.1-devel

$ xhost +

access control disabled, clients can connect from any host

$ sudo docker run --gpus all -it --rm --net=host \

  -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime \

  -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.1 nvcr.io/nvidia/deepstream:6.1-devel

# cd samples/configs/deepstream-app

# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

# cd /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-app

# export CUDA_VER=11.6

# make

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.1/lib/gst-plugins/

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# gst-launch-1.0 uridecodebin uri=$URI ! nvvideoconvert ! nveglglessink

# exit

用 Docker DeepStream 6.0.1 測試 Integrate TAO model with DeepStream SDK

$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git deepstream_tao_apps-tao3.0_ds6.0.1

$ sudo apt install gitk

$ cd deepstream_tao_apps-tao3.0_ds6.0.1/

$ git branch -r

$ git checkout release/tao3.0_ds6.0.1

$ git log -1

$ sudo docker run --gpus all -it --rm --net=host \

> -v /etc/localtime:/etc/localtime \

> -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \

> -v /home/mark/Data/TensorRT/TensorRT_OSS-8.0.1/:/home/TensorRT \

> -v /home/mark/Data/DeepStream/deepstream_tap_apps/deepstream_tao_apps-tao3.0_ds6.0.1/:/home/deepstream_tao_apps \

> -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel

# cd /home/deepstream_tao_apps/

# ./download_models.sh 

# export CUDA_VER=11.4

# make

# cp /home/TensorRT/build/out/libnvinfer_plugin.so.8.0.1 /usr/lib/x86_64-linux-gnu/

# ./apps/tao_detection/ds-tao-detection -c configs/frcnn_tao/pgie_frcnn_tao_config.txt -i /opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.h264 -d

# cd /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-nvdsvideotemplate/

# make

# cp libnvdsgst_videotemplate.so /opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/

# rm -rf ~/.cache/gstreamer-1.0/

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.0/lib/cvcore_libs/

# cd /home/deepstream_tao_apps/apps/tao_others/

# make

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# cd deepstream-bodypose2d-app/

# ./deepstream-bodypose2d-app 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./body2dout

# cd ../deepstream-emotion-app/

# ./deepstream-emotion-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-faciallandmark-app/

# ./deepstream-faciallandmark-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-gaze-app/

# ./deepstream-gaze-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./gazenet

# cd ../deepstream-gesture-app/

# ./deepstream-gesture-app 3 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./gesture

# cd ../deepstream-heartrate-app/

# ./deepstream-heartrate-app 3 $URI ./heartrate

# exit

用 Docker DeepStream 6.1 測試 Integrate TAO model with DeepStream SDK

$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git deepstream_tao_apps-tao3.0_ds6.1ga

$ sudo apt install gitk

$ cd deepstream_tao_apps-tao3.0_ds6.1ga/

$ git branch -r

$ git checkout release/tao3.0_ds6.1ga

$ git log -1

$ sudo docker run --gpus all -it --rm --net=host \

> -v /etc/localtime:/etc/localtime \

> -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \

> -v /home/mark/Data/TensorRT/TensorRT_OSS-8.2.1/:/home/TensorRT \

> -v /home/mark/Data/DeepStream/deepstream_tap_apps/deepstream_tao_apps-tao3.0_ds6.1ga/:/home/deepstream_tao_apps \

> -w /opt/nvidia/deepstream/deepstream-6.1 nvcr.io/nvidia/deepstream:6.1-devel

# cp /home/TensorRT/build/out/libnvinfer_plugin.so.8.2.1 /usr/lib/x86_64-linux-gnu/

# cd /usr/lib/x86_64-linux-gnu/

# rm libnvinfer_plugin.so.8

# ln -s libnvinfer_plugin.so.8.2.1 libnvinfer_plugin.so.8

# cd /home/deepstream_tao_apps/

# ./download_models.sh 

# export CUDA_VER=11.6

# make

# ./apps/tao_detection/ds-tao-detection -c configs/frcnn_tao/pgie_frcnn_tao_config_dgpu.txt -i /opt/nvidia/deepstream/deepstream-6.1/samples/streams/sample_720p.h264 -d

# cd /opt/nvidia/deepstream/deepstream-6.1/sources/gst-plugins/gst-nvdsvideotemplate/

# make

# cp libnvdsgst_videotemplate.so /opt/nvidia/deepstream/deepstream-6.1/lib/gst-plugins/

# rm -rf ~/.cache/gstreamer-1.0/

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.1/lib/cvcore_libs/

# cd /home/deepstream_tao_apps/apps/tao_others/

# make

# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp

# cd deepstream-bodypose2d-app/

# ./deepstream-bodypose2d-app 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt 0 0 $URI ./body2dout

# cd ../deepstream-emotion-app/

# ./deepstream-emotion-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-faciallandmark-app/

# ./deepstream-faciallandmark-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks

# cd ../deepstream-gaze-app/

# ./deepstream-gaze-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./gazenet

# cd ../deepstream-gesture-app/

# ./deepstream-gesture-app 3 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./gesture

# cd ../deepstream-heartrate-app/

# ./deepstream-heartrate-app 3 $URI ./heartrate

# exit

安裝其他元件

參考 Ubuntu 18.04 重灌 上的 安裝 CMake 和 安裝 OpenCV