因為 Ubuntu 18.04 只能安裝 DeepStream 6.0.1
在 Ubuntu 20.04 才能安裝 DeepStream 6.1
所以在 Ubuntu 18.04 上安裝 Docker Ubuntu 20.04
版本對應如下
===========================
DS 6.1
Ubuntu 20.04
GCC 9.4.0
CUDA 11.6.1
cuDNN 8.4.0.27
TRT 8.2.5.1
Display Driver:R510.47.03
GStreamer 1.16.2
OpenCV 4.2.0
deepstream:6.1
===========================
DS 6.0.1
Ubuntu 18.04
GCC 7.3.0
CUDA 11.4.1
cuDNN 8.2+
TRT 8.0.1
Display Driver:R470.63.01
GStreamer 1.14.5
OpenCV 3.4.0
deepstream:6.0.1
===========================
安裝作業系統
BIOS 選擇開機
Install Ubuntu
Installation type 選 Something else
Create partition/Mount point 選擇 /
Settings/Details/Users
Unlock
Automatic Login: ON
更新系統，安裝一些常用套件
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install ssh
$ sudo apt-get install python3-pip
mount nfs
$ sudo apt-get install nfs-common
$ sudo mount -t nfs ip:/share_folder /mount_folder
$ sudo vi /etc/fstab
#中間的空格要使用 tab
ip:/share_folder /mnt/mount_folder nfs defaults,bg 0 0
$ sudo mount -a
vnc
$ sudo apt-get install x11vnc
$ sudo x11vnc -storepasswd
$ sudo chown user.group ~/.vnc/passwd
安裝顯示卡驅動，CUDA 和 CUDNN
$ sudo ubuntu-drivers devices
$ sudo apt-get install nvidia-driver-510
https://developer.nvidia.com/cuda-downloads
選擇 Archive of Previous CUDA Releases
選擇 CUDA Toolkit 11.4.1
選擇 deb(local), deb(network) 不可使用了
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
$ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
$ wget https://developer.download.nvidia.com/compute/cuda/11.4.1/local_installers/cuda-repo-ubuntu1804-11-4-local_11.4.1-470.57.02-1_amd64.deb
$ sudo dpkg -i cuda-repo-ubuntu1804-11-4-local_11.4.1-470.57.02-1_amd64.deb
$ sudo apt-key add /var/cuda-repo-ubuntu1804-11-4-local/7fa2af80.pub
$ sudo apt-get update
$ sudo apt-get -y install cuda-11-4
之後再安裝一遍 CUDA Toolkit 11.3.1(因為 TensorRT 8.0.1 需要 CUDA-11-3)
$ sudo apt-get -y install cuda-11-3
之後再安裝一遍 CUDA Toolkit 11.6.1(因為 DeepStream 6.1 需要 CUDA-11-6)
$ sudo apt-get -y install cuda-11-6
$ update-alternatives --display cuda
$ update-alternatives --config cuda
參考 https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html
下載 Local Install for Ubuntu18.04 x86_64(Deb)
$ sudo apt-get install zlib1g
$ sudo dpkg -i cudnn-local-repo-ubuntu1804-8.4.1.50_1.0-1_amd64.deb
$ sudo cp /var/cudnn-local-repo-ubuntu1804-8.4.1.50/cudnn-local-BA71F057-keyring.gpg /usr/share/keyrings/
$ sudo apt-get update
$ apt list -a libcudnn8
$ sudo apt-get install libcudnn8=8.4.1.50-1+cuda11.6
$ sudo apt-get install libcudnn8-dev=8.4.1.50-1+cuda11.6
$ sudo apt-get install libcudnn8-samples=8.4.1.50-1+cuda11.6
安裝 TensorRT 8.0.1
因為使用 Docker 安裝 DeepStream 6.1, 所以不用安裝 TensorRT 8.2.5.1
參考 https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html
在 3. Downloading TensorRT
連結 https://developer.nvidia.com/tensorrt 按 GET STARTED
連結 https://developer.nvidia.com/tensorrt-getting-started 按 DOWNLOAD NOW
選擇 TensorRT 8
選擇 TensorRT 8.0 GA
選擇 TensorRT 8.0.1 GA for Ubuntu 18.04 and CUDA 11.3 DEB local repo package
$ sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626_1-1_amd64.deb
$ sudo apt-key add /var/nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626/7fa2af80.pub
$ sudo apt-get update
$ sudo apt --fix-broken install
$ sudo apt-get upgrade
$ sudo apt-get install tensorrt
安裝 DeepStream
因為不選擇最新的版本
參考 https://docs.nvidia.com/metropolis/deepstream-archive.html
$ sudo apt install libssl1.0.0
$ sudo apt install libgstreamer1.0-0
$ sudo apt install gstreamer1.0-tools
$ sudo apt install gstreamer1.0-plugins-good
$ sudo apt install gstreamer1.0-plugins-bad
$ sudo apt install gstreamer1.0-plugins-ugly
$ sudo apt install gstreamer1.0-libav
$ sudo apt install libgstrtspserver-1.0-0
$ sudo apt install libjansson4
$ sudo apt install gcc
$ sudo apt install make
$ sudo apt install git
$ sudo apt install python3
$ cd /usr/bin
$ sudo ln -s python3 python
$ git clone https://github.com/edenhill/librdkafka.git
$ cd librdkafka
$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a
$ ./configure
$ make
$ sudo make install
$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-6.0/lib
$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-6.0/lib
下載 Deepstream 6.0 dGPU Debian package
https://developer.nvidia.com/deepstream-6.0_6.0.1-1_amd64deb
$ sudo apt-get install ./deepstream-6.0_6.0.1-1_amd64.deb
$ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin
$ cd /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/
$ deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt
安裝 Nvidia Docker
參考 https://docs.docker.com/engine/install/ubuntu/
$ sudo apt-get update
$ sudo apt-get install ca-certificates
$ sudo apt-get install curl
$ sudo apt-get install gnupg
$ sudo apt-get install lsb-release
$ sudo mkdir -p /etc/apt/keyrings
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
$ echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
$ sudo docker run --rm hello-world
參考 https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
&& curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
$ sudo apt-get update
$ sudo apt-get install -y nvidia-docker2
$ sudo systemctl restart docker
$ sudo groupadd docker
$ sudo usermod -a -G docker $USER
$ sudo reboot
安裝 NGC CLI
參考 https://ngc.nvidia.com/setup/installers/cli
$ wget --content-disposition https://ngc.nvidia.com/downloads/ngccli_linux.zip && \
unzip ngccli_linux.zip && \
chmod u+x ngc-cli/ngc
$ find ngc-cli/ -type f -exec md5sum {} + | LC_ALL=C sort | md5sum -c ngc-cli.md5
$ echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile && source ~/.bash_profile
$ ngc config set
# 直接 enter 即可
$ docker login nvcr.io
Username: $oauthtoken
Password: <Your API Key>
使用 Docker 安裝 TensorRT OSS
開啟 https://github.com/nvidia/TensorRT
切換至 Tags 8.0.1
$ git clone -b master https://github.com/nvidia/TensorRT TensorRT_OSS-8.0.1
$ cd TensorRT_OSS-8.0.1/
$ git describe --tags
8.2.0-EA-2-g96e2397
$ git tag -l
$ git branch -r
$ git checkout 8.0.1
$ git log -1
$ git describe --tags
8.0.1
$ git submodule update --init --recursive
$ vi docker/ubuntu-18.04.Dockerfile
修改下列一行
RUN cd /usr/local/bin && wget https://ngc.nvidia.com/downloads/ngccli_cat_linux.zip && \
unzip ngccli_cat_linux.zip && chmod u+x ngc-cli/ngc && \
rm ngccli_cat_linux.zip ngc-cli.md5 && echo "no-apikey\nascii\n" | ngc-cli/ngc config set
$ cat docker/ubuntu-18.04.Dockerfile | grep CUDA_VERSION
ARG CUDA_VERSION=11.3.1
$ ./docker/build.sh --file docker/ubuntu-18.04.Dockerfile --tag tensorrt-ubuntu18.04-cuda11.3 --cuda 11.3.1
$ ./docker/launch.sh --tag tensorrt-ubuntu18.04-cuda11.3 --gpus all
/workspace$ cd $TRT_OSSPATH
/workspace/TensorRT$ mkdir -p build && cd build
/workspace/TensorRT/build$ cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
/workspace/TensorRT/build$ make -j$(nproc)
/workspace/TensorRT/build$ exit
$ mkdir backup
$ sudo mv /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.8.0.1 backup/
$ sudo cp build/out/libnvinfer_plugin.so.8.0.1 /usr/lib/x86_64-linux-gnu/
另外還要再安裝 TensorRT 8.2.1 給 Docker Ubuntu 20.04
將 8.0.1 改成 8.2.1, 18.04 改成 20.04, 11.3.1 改成 11.4.2, 但不需安裝
用 Docker 開發 DeepStream 6.0.1
參考 https://docs.nvidia.com/metropolis/deepstream/6.0.1/dev-guide/text/DS_docker_containers.html
$ docker pull nvcr.io/nvidia/deepstream:6.0.1-devel
$ xhost +
access control disabled, clients can connect from any host
$ sudo docker run --gpus all -it --rm --net=host \
-v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime \
-e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel
# update-alternatives --display cuda
# cat /etc/os-release
# cd samples/configs/deepstream-app
# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt
# cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-app
# export CUDA_VER=11.4
# make
# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/
# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp
# gst-launch-1.0 uridecodebin uri=$URI ! nvvideoconvert ! nveglglessink
# exit
用 Docker 開發 DeepStream 6.1
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_docker_containers.html
$ docker pull nvcr.io/nvidia/deepstream:6.1-devel
$ xhost +
access control disabled, clients can connect from any host
$ sudo docker run --gpus all -it --rm --net=host \
-v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime \
-e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.1 nvcr.io/nvidia/deepstream:6.1-devel
# cd samples/configs/deepstream-app
# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt
# cd /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-app
# export CUDA_VER=11.6
# make
# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.1/lib/gst-plugins/
# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp
# gst-launch-1.0 uridecodebin uri=$URI ! nvvideoconvert ! nveglglessink
# exit
用 Docker DeepStream 6.0.1 測試 Integrate TAO model with DeepStream SDK
$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git deepstream_tao_apps-tao3.0_ds6.0.1
$ sudo apt install gitk
$ cd deepstream_tao_apps-tao3.0_ds6.0.1/
$ git branch -r
$ git checkout release/tao3.0_ds6.0.1
$ git log -1
$ sudo docker run --gpus all -it --rm --net=host \
> -v /etc/localtime:/etc/localtime \
> -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \
> -v /home/mark/Data/TensorRT/TensorRT_OSS-8.0.1/:/home/TensorRT \
> -v /home/mark/Data/DeepStream/deepstream_tap_apps/deepstream_tao_apps-tao3.0_ds6.0.1/:/home/deepstream_tao_apps \
> -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0.1-devel
# cd /home/deepstream_tao_apps/
# ./download_models.sh
# export CUDA_VER=11.4
# make
# cp /home/TensorRT/build/out/libnvinfer_plugin.so.8.0.1 /usr/lib/x86_64-linux-gnu/
# ./apps/tao_detection/ds-tao-detection -c configs/frcnn_tao/pgie_frcnn_tao_config.txt -i /opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.h264 -d
# cd /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-nvdsvideotemplate/
# make
# cp libnvdsgst_videotemplate.so /opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/
# rm -rf ~/.cache/gstreamer-1.0/
# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.0/lib/cvcore_libs/
# cd /home/deepstream_tao_apps/apps/tao_others/
# make
# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp
# cd deepstream-bodypose2d-app/
# ./deepstream-bodypose2d-app 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./body2dout
# cd ../deepstream-emotion-app/
# ./deepstream-emotion-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks
# cd ../deepstream-faciallandmark-app/
# ./deepstream-faciallandmark-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks
# cd ../deepstream-gaze-app/
# ./deepstream-gaze-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./gazenet
# cd ../deepstream-gesture-app/
# ./deepstream-gesture-app 3 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./gesture
# cd ../deepstream-heartrate-app/
# ./deepstream-heartrate-app 3 $URI ./heartrate
# exit
用 Docker DeepStream 6.1 測試 Integrate TAO model with DeepStream SDK
$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git deepstream_tao_apps-tao3.0_ds6.1ga
$ sudo apt install gitk
$ cd deepstream_tao_apps-tao3.0_ds6.1ga/
$ git branch -r
$ git checkout release/tao3.0_ds6.1ga
$ git log -1
$ sudo docker run --gpus all -it --rm --net=host \
> -v /etc/localtime:/etc/localtime \
> -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \
> -v /home/mark/Data/TensorRT/TensorRT_OSS-8.2.1/:/home/TensorRT \
> -v /home/mark/Data/DeepStream/deepstream_tap_apps/deepstream_tao_apps-tao3.0_ds6.1ga/:/home/deepstream_tao_apps \
> -w /opt/nvidia/deepstream/deepstream-6.1 nvcr.io/nvidia/deepstream:6.1-devel
# cp /home/TensorRT/build/out/libnvinfer_plugin.so.8.2.1 /usr/lib/x86_64-linux-gnu/
# cd /usr/lib/x86_64-linux-gnu/
# rm libnvinfer_plugin.so.8
# ln -s libnvinfer_plugin.so.8.2.1 libnvinfer_plugin.so.8
# cd /home/deepstream_tao_apps/
# ./download_models.sh
# export CUDA_VER=11.6
# make
# ./apps/tao_detection/ds-tao-detection -c configs/frcnn_tao/pgie_frcnn_tao_config_dgpu.txt -i /opt/nvidia/deepstream/deepstream-6.1/samples/streams/sample_720p.h264 -d
# cd /opt/nvidia/deepstream/deepstream-6.1/sources/gst-plugins/gst-nvdsvideotemplate/
# make
# cp libnvdsgst_videotemplate.so /opt/nvidia/deepstream/deepstream-6.1/lib/gst-plugins/
# rm -rf ~/.cache/gstreamer-1.0/
# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/nvidia/deepstream/deepstream-6.1/lib/cvcore_libs/
# cd /home/deepstream_tao_apps/apps/tao_others/
# make
# export URI=rtsp://user:passwd@192.168.0.108:554/live1s2.sdp
# cd deepstream-bodypose2d-app/
# ./deepstream-bodypose2d-app 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt 0 0 $URI ./body2dout
# cd ../deepstream-emotion-app/
# ./deepstream-emotion-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks
# cd ../deepstream-faciallandmark-app/
# ./deepstream-faciallandmark-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./landmarks
# cd ../deepstream-gaze-app/
# ./deepstream-gaze-app 3 ../../../configs/facial_tao/sample_faciallandmarks_config.txt $URI ./gazenet
# cd ../deepstream-gesture-app/
# ./deepstream-gesture-app 3 3 ../../../configs/bodypose2d_tao/sample_bodypose2d_model_config.txt $URI ./gesture
# cd ../deepstream-heartrate-app/
# ./deepstream-heartrate-app 3 $URI ./heartrate
# exit
安裝其他元件
