deepstream create_pipeline.create_common_elements 分析

參考之前 https://yingrenn.blogspot.com/2023/03/deepstream-createpipeline.html

create_common_elements {

    common_elements.segvisual_bin = create_segvisual_bin

    common_elements.secondary_gie_bin = create_secondary_gie_bin {

        bin.sink - tee - queue - bin.src

        bin->sub_bins[] = create_secondary_gie {

            secondary_gie = nvinfer | nvinferserver

            if (is_parent || has_children) {

                queue

            }

            if (no_children) {

                queue - secondary_gie - fakesink

            } else {

                queue - secondary_gie - tee

            }

        }

        if (parent_index == -1) {

            tee - sub_bins[i].queue

        } else if (sub_bins[parent_index].tee) {

            sub_bins[parent_index].tee - sub_bins[i].queue

        } else {

            sub_bins[parent_index].secondary_gie - sub_bins[i].secondary_gie

        }

    }

    common_elements.secondary_preprocess_bin = create_secondary_preprocess_bin

    common_elements.dsanalytics_bin = create_dsanalytics_bin

    common_elements.tracker_bin = create_tracking_bin

    common_elements.primary_gie_bin = create_primary_gie_bin

    common_elements.preprocess_bin = create_preprocess_bin

    common_elements.msg_conv = gst_element_factory_make

    common_elements.tee = gst_element_factory_make

    

    sink_elem - preprocess_bin - primary_gie_bin - tracker_bin - dsanalytics_bin - secondary_preprocess_bin - secondary_gie_bin - segvisual_bin - msg_conv - tee - src_elem

}

  

Fine-tuning Whisper in a Google Colab

參考 https://research.google.com/colaboratory/local-runtimes.html

使得 colab 可以用 local 的 cpu 和 gpu

文件說可以使用 docker 或 jupyter

但只有 jupyter 成功

建立 huggingface 帳號，並且登入

開啟 https://huggingface.co/settings/tokens

按下 New token

選擇 Role(有 read 和 write)

按下 copy

在執行下列命令時，貼上 token

$ huggingface-cli login

    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|

    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|

    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|

    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|

    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|

    

    To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .

Token: 

Add token as git credential? (Y/n) Y

Token is valid (permission: read).

$ huggingface-cli login

    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|

    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|

    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|

    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|

    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|

    

    A token is already saved on your machine. Run `huggingface-cli whoami` to get more information or `huggingface-cli logout` if you want to log out.

    Setting a new token will erase the existing one.

    To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .

Token: 

Add token as git credential? (Y/n) Y

Token is valid (permission: write).

Ubuntu 之 錄音 與 撥放, 使用 arecord aplay ffmpeg

列出錄音設備

$ arecord -l

**** List of CAPTURE Hardware Devices ****

card 0: PCH [HDA Intel PCH], device 0: ALCS1200A Analog [ALCS1200A Analog]

  Subdevices: 1/1

  Subdevice #0: subdevice #0

card 0: PCH [HDA Intel PCH], device 2: ALCS1200A Alt Analog [ALCS1200A Alt Analog]

  Subdevices: 1/1

  Subdevice #0: subdevice #0

指定 card:0, device 0, 使用16000採樣，錄音10秒

$ arecord -Dhw:0,0 -d 10 -f cd -r 16000 -c 2 -t wav test.wav

Recording WAVE 'test.wav' : Signed 16 bit Little Endian, Rate 16000 Hz, Stereo

Warning: rate is not accurate (requested = 16000Hz, got = 44100Hz)

         please, try the plug plugin 

$ arecord -D mono --device=hw:0,0 -d 10 -f cd -r 16000 -c 2 -t wav test.wav

Recording WAVE 'test.wav' : Signed 16 bit Little Endian, Rate 16000 Hz, Stereo

Warning: rate is not accurate (requested = 16000Hz, got = 44100Hz)

         please, try the plug plugin 

列出播放設備

$ aplay -l

**** List of PLAYBACK Hardware Devices ****

card 0: PCH [HDA Intel PCH], device 0: ALCS1200A Analog [ALCS1200A Analog]

  Subdevices: 1/1

  Subdevice #0: subdevice #0

card 0: PCH [HDA Intel PCH], device 1: ALCS1200A Digital [ALCS1200A Digital]

  Subdevices: 1/1

  Subdevice #0: subdevice #0

card 0: PCH [HDA Intel PCH], device 3: HDMI 0 [HDMI 0]

  Subdevices: 1/1

  Subdevice #0: subdevice #0

從錄音設備直接撥出

$ arecord -Dhw:0,0 -d 10 -f cd -r 16000 | aplay -Dhw:0,0 -r 16000

Recording WAVE 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Stereo

Warning: rate is not accurate (requested = 16000Hz, got = 44100Hz)

         please, try the plug plugin 

Playing WAVE 'stdin' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo

安裝 ffmpeg

$ sudo add-apt-repository ppa:savoury1/ffmpeg4  

$ sudo apt-cache policy ffmpeg  

$ sudo apt-get install ffmpeg  

$ ffmpeg -version  

$ sudo add-apt-repository --remove ppa:savoury1/ffmpeg4  

參考 https://ffmpeg.org/ffmpeg-devices.html

列出裝置

$ ffmpeg -devices

Devices:

 D. = Demuxing supported

 .E = Muxing supported

 --

 DE alsa            ALSA audio output

  E caca            caca (color ASCII art) output device

 DE fbdev           Linux framebuffer

 D  iec61883        libiec61883 (new DV1394) A/V input device

 D  jack            JACK Audio Connection Kit

 D  kmsgrab         KMS screen capture

 D  lavfi           Libavfilter virtual input device

$ cat /proc/asound/cards

 0 [PCH            ]: HDA-Intel - HDA Intel PCH

                      HDA Intel PCH at 0xa7230000 irq 148

 1 [NVidia         ]: HDA-Intel - HDA NVidia

                      HDA NVidia at 0xa5080000 irq 17

錄音

$ ffmpeg -f alsa -i hw:0 test.wav

安裝 Ubuntu 20.04

$ sudo apt-get update

$ sudo apt-get upgrade

$ sudo apt-get install ssh

$ sudo vi /etc/fstab

#中間的空格要使用 tab

ip:/share_folder /mnt/mount_folder nfs defaults,bg 0 0

$ cd /mnt

$ sudo mkdir QNAP_A QNAP_B

$ sudo mount -a

$ mkdir -p ~/.config/autostart

$ cp /usr/share/applications/vino-server.desktop ~/.config/autostart/

$ gsettings set org.gnome.Vino prompt-enabled false

$ gsettings set org.gnome.Vino require-encryption false

$ gsettings set org.gnome.Vino authentication-methods "['vnc']"

$ gsettings set org.gnome.Vino vnc-password $(echo -n 'ChangeToYourPasswd'|base64)

$ sudo vi /etc/gdm3/custom.conf

WaylandEnable=false

AutomaticLoginEnable = true

AutomaticLogin = UserLoginName

$ vi vino.sh

DISP=`ps -u $(id -u) -o pid= | \

    while read pid; do

        cat /proc/$pid/environ 2>/dev/null | tr '\0' '\n' | grep '^DISPLAY=:'

    done | grep -o ':[0-9]*' | sort -u`

echo $DISP

/usr/lib/vino/vino-server --display=$DISP

$ chmod +x vino.sh

依據 使用最新版本的 driver

CUDA Toolkit and Corresponding Driver Versions

https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html

dGPU Setup for Ubuntu

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Quickstart.html

Ubuntu 20.04

GStreamer 1.16.3

NVIDIA driver 525.125.06

CUDA 12.1

TensorRT 8.5.3.1

$ sudo ubuntu-drivers devices

$ sudo apt-get install nvidia-driver-535

$ sudo reboot

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.1-1_all.deb

$ sudo dpkg -i cuda-keyring_1.1-1_all.deb

$ sudo apt-get update

$ sudo apt-get -y install cuda-12-2

$ sudo apt-get -y install cuda-12-1

安裝 cuDNN

參考 https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

到 2.2. Downloading cuDNN for Linux(https://developer.nvidia.com/cudnn)

下載 Local Install for Ubuntu18.04 x86_64(Deb)

$ sudo apt-get install zlib1g

$ sudo dpkg -i cudnn-local-repo-ubuntu2004-8.9.5.29_1.0-1_amd64.deb

sudo cp /var/cudnn-local-repo-ubuntu2004-8.9.5.29/cudnn-local-98C06E99-keyring.gpg /usr/share/keyrings/

$ sudo apt-get update

$ apt list -a libcudnn8

$ sudo apt-get install libcudnn8=8.9.5.29-1+cuda12.2

$ sudo apt-get install libcudnn8-dev=8.9.5.29-1+cuda12.2

$ sudo apt-get install libcudnn8-samples=8.9.5.29-1+cuda12.2

$ update-alternatives --display libcudnn

$ cp -r /usr/src/cudnn_samples_v8/ .

$ cd cudnn_samples_v8/mnistCUDNN/

$ sudo apt-get install libfreeimage3 libfreeimage-dev

$ make clean && make

$ ./mnistCUDNN

...

Test passed!

安裝 TensorRT 8.6.1

https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-861/install-guide/index.html

$ sudo apt-get install python3-pip

$ sudo apt-get install python3.8.venv

$ python3 -m venv envs/tensorrt

$ source envs/tensorrt/bin/activate

$ pip3 install --upgrade pip

$ python3 -m pip install --extra-index-url https://pypi.nvidia.com tensorrt_libs

$ python3 -m pip install --extra-index-url https://pypi.nvidia.com tensorrt_bindings

$ python3 -m pip install --upgrade tensorrt

$ python3 -m pip install --upgrade tensorrt_lean

$ python3 -m pip install --upgrade tensorrt_dispatch

測試  TensorRT Python

$ python3

>>> import tensorrt

>>> print(tensorrt.__version__)

>>> assert tensorrt.Builder(tensorrt.Logger())

>>> import tensorrt_lean as trt

>>> print(trt.__version__)

>>> assert trt.Builder(trt.Logger())

>>> import tensorrt_dispatch as trt

>>> print(trt.__version__)

>>> assert trt.Builder(trt.Logger())

連結 https://developer.nvidia.com/tensorrt 按 GET STARTED

連結 https://developer.nvidia.com/tensorrt-getting-started 按 DOWNLOAD NOW

選擇 TensorRT 8

選擇 TensorRT 8.6 GA

TensorRT 8.6 GA for Ubuntu 20.04 and CUDA 12.0 and 12.1 DEB local repo Package

$ sudo dpkg -i nv-tensorrt-local-repo-ubuntu2004-8.6.1-cuda-12.0_1.0-1_amd64.deb

$ sudo cp /var/nv-tensorrt-local-repo-ubuntu2004-8.6.1-cuda-12.0/nv-tensorrt-local-9A1EDFBA-keyring.gpg /usr/share/keyrings/

$ sudo apt-get update

$ sudo apt-get install tensorrt

$ sudo apt-get install libnvinfer-lean8

$ sudo apt-get install libnvinfer-vc-plugin8

$ sudo apt-get install python3-libnvinfer-lean

$ sudo apt-get install python3-libnvinfer-dispatch

$ python3 -m pip install numpy

$ sudo apt-get install python3-libnvinfer-dev

$ python3 -m pip install protobuf

$ sudo apt-get install uff-converter-tf

$ python3 -m pip install numpy onnx

$ sudo apt-get install onnx-graphsurgeon

確認安裝

$ dpkg-query -W tensorrt

tensorrt        8.6.1.6-1+cuda12.0

安裝 DeepStream

$ sudo apt-get install libssl1.1

$ sudo apt-get install libgstreamer1.0-0

$ sudo apt-get install gstreamer1.0-tools

$ sudo apt-get install gstreamer1.0-plugins-good

$ sudo apt-get install gstreamer1.0-plugins-bad

$ sudo apt-get install gstreamer1.0-plugins-ugly

$ sudo apt-get install gstreamer1.0-libav

$ sudo apt-get install libgstreamer-plugins-base1.0-dev

$ sudo apt-get install libgstrtspserver-1.0-0

$ sudo apt-get install libjansson4

$ sudo apt-get install libyaml-cpp-dev

$ sudo apt-get install libjsoncpp-dev

$ sudo apt-get install protobuf-compiler

$ sudo apt-get install gcc

$ sudo apt-get install make

$ sudo apt-get install git

$ sudo apt-get install python3

$ git clone https://github.com/edenhill/librdkafka.git

$ cd librdkafka

$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a

$ ./configure

$ make

$ sudo make install

$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-6.3/lib

$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-6.3/lib

https://catalog.ngc.nvidia.com/orgs/nvidia/resources/deepstream

下載 deepstream-6.3_6.3.0-1_arm64.deb

$ wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/nvidia/deepstream/versions/6.3/files/deepstream-6.3_6.3.0-1_amd64.deb'

$ sudo apt-get install ./deepstream-6.3_6.3.0-1_amd64.deb

$ cd /opt/nvidia/deepstream/deepstream-6.3/samples/configs/deepstream-app

$ deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

安裝 Docker

https://docs.docker.com/engine/install/ubuntu/

$ sudo apt-get update

$ sudo apt-get install ca-certificates curl gnupg

$ sudo install -m 0755 -d /etc/apt/keyrings

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

$ sudo chmod a+r /etc/apt/keyrings/docker.gpg

$ echo \

  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \

  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \

  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

$ sudo apt-get update

$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

$ sudo docker run --rm hello-world

安裝 NVIDIA Container Toolkit

參考 https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \

  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \

    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \

    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list \

  && \

    sudo apt-get update

$ sudo apt-get install -y nvidia-container-toolkit

$ sudo nvidia-ctk runtime configure --runtime=docker

$ sudo systemctl restart docker

$ sudo groupadd docker

$ sudo usermod -a -G docker $USER

$ docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

安裝 NGC CLI

參考 https://ngc.nvidia.com/setup/installers/cli

$ wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/3.30.1/files/ngccli_linux.zip -O ngccli_linux.zip && unzip ngccli_linux.zip

$ find ngc-cli/ -type f -exec md5sum {} + | LC_ALL=C sort | md5sum -c ngc-cli.md5

$ sha256sum ngccli_linux.zip

$ chmod u+x ngc-cli/ngc

$ echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile && source ~/.bash_profile

$ ngc config set

# 直接 enter 即可

$ docker login nvcr.io

Username: $oauthtoken

Password: <Your API Key>

用 Docker 開發 DeepStream 6.3

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_docker_containers.html

$ sudo docker pull nvcr.io/nvidia/deepstream:6.3-gc-triton-devel

$ export DISPLAY=:0

$ xhost +

$ docker run -it --rm --net=host --gpus all -e DISPLAY=$DISPLAY --device /dev/snd -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/deepstream:6.3-gc-triton-devel

# cd samples/configs/deepstream-app

# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt 

# exit

$ sudo docker ps -a

$ sudo docker stop container_id

$ sudo docker rm container_id

$ sudo docker image list

$ sudo docker image rm image_id

Ubuntu 安裝 exiv2, 開發 exif 相關程式

參考 https://github.com/Exiv2/exiv2/tree/main

參考 https://github.com/Exiv2/exiv2/tree/main#PlatformLinux

download cmake from https://cmake.org/download/

$ tar xvfz cmake-3.27.6.tar.gz

$ cd cmake-3.27.6

$ sudo apt-get install libssl-dev

$ ./bootstrap

$ make -j4

$ sudo make install

$ git clone https://github.com/Exiv2/exiv2.git

$ cd exiv2

$ sudo apt-get install --yes build-essential ccache clang cmake git google-mock libbrotli-dev libcurl4-openssl-dev libexpat1-dev libgtest-dev libinih-dev libssh-dev libxml2-utils libz-dev python3 zlib1g-dev

$ cmake -S . -B build -G "Unix Makefiles"

$ cmake --build build

$ ctest --test-dir build --verbose

$ sudo cmake --install build

$ g++ -o exifprint exifprint.cpp -lexiv2

使用 sudo 不輸入密碼

增加可以 reboot 的 myuser

$ sudo deluser myuser

$ adduser myuser

$ sudo gpasswd -a myuser sudo

$ echo "myuser ALL = NOPASSWD: /usr/sbin/reboot" | sudo tee /etc/sudoers.d/60_myuser

$ sudo chmod 0440 /etc/sudoers.d/60_myuser

若不幸輸入錯字，會產生如下錯誤

>>> /etc/sudoers: syntax error near line 24 <<<

sudo: parse error in /etc/sudoers near line 24

sudo: no valid sudoers sources found, quitting

sudo: unable to initialize policy plugin

使用下列方法修復

$ pkexec visudo

What now? 會停在此處，別怕按下 Enter

Options are:

  (e)dit sudoers file again

  e(x)it without saving changes to sudoers file

  (Q)uit and save changes to sudoers file (DANGER!)

使用 ssh 不用輸入密碼

hostA$ ssh-keygen

hostA$ ssh-copy-id "user@hostB -p 22"

hostA$ ssh user@hostB "command.sh arg1 arg2"