jetson nano install tensorflow

$ sudo apt-get update

$ sudo apt-get install libhdf5-serial-dev hdf5-tools libhdf5-dev zlib1g-dev zip libjpeg8-dev liblapack-dev libblas-dev gfortran

$ sudo apt-get install python3-pip

$ sudo pip3 install -U pip testresources setuptools==49.6.0

$ sudo apt-get install virtualenv

$ make envs

$ cd envs

$ sudo pip3 install -U numpy==1.16.1 future==0.18.2 mock==3.0.5 h5py==2.10.0 keras_preprocessing==1.1.1 keras_applications==1.0.8 gast==0.2.2 futures protobuf pybind11

$ python3 -m virtualenv -p python3 tensorflow-2.3.1

$ source tensorflow-2.3.1/bin/activate

$ pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v44 tensorflow==2.3.1+nv20.11

$ python3 -m virtualenv -p python3 tensorflow-1.15.4

$ source tensorflow-1.15.4/bin/activate

$ pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v44 tensorflow==1.15.4+nv20.11

xpra 常用命令

啟動 xterm, select to copy, shift+insert or middle-click to paste

開啟選項 ctrl+middle-click

設定 buffer size -sl

顯示 scrollbar -sb

黑筆白字 -fg white -bg black

>Xpra.exe start ssh://user@ip/session --exit-with-children=yes --start-child="xterm -fa Monospace -fs 16 -xrm 'XTerm*selectToClipboard: true' -sl 1000 -sb -fg white -bg black"

>Xpra.exe start ssh://user@ip/session --exit-with-children=yes --start-child=/opt/google/chrome/chrome

>Xpra.exe start ssh://user@ip/session --exit-with-children=yes --start-child=/usr/bin//chromium-browser

>Xpra.exe stop ssh://user@ip/session

>Xpra.exe attach ssh://user@ip/session

>Xpra.exe detach ssh://user@ip/session

連上目前的桌面，如 vnc

>Xpra.exe shadow ssh://user@ip/1

>Xpra.exe shadow ssh://user@ip

$ xpra list

$ xpra start :session

$ DISPLAY=:session gedit

xpra in jsetson nano

參考 Jetson Nano平台通过Xpra运行图形程序

以下部分步驟，直接跳過避不開的錯誤

$ sudo apt-get install libx11-dev libxtst-dev libxcomposite-dev libxdamage-dev \

    libxkbfile-dev python-all-dev

$ sudo apt-get install libgtk-3-dev python3-dev python3-cairo-dev python-gi-dev cython3

$ sudo apt-get install xauth x11-xkb-utils

$ sudo apt-get install libx264-dev libvpx-dev yasm

$ sudo apt-get install libavformat-dev libavcodec-dev libswscale-dev

$ sudo apt-get install libturbojpeg-dev

$ sudo apt-get install libwebp-dev

$ sudo apt-get install uglifyjs brotli libjs-jquery libjs-jquery-ui gnome-backgrounds

$ sudo apt-get install python3-opengl python3-numpy python3-pil

$ sudo apt-get install python3-rencode python3-lz4 python3-dbus python3-cryptography \

    python3-netifaces python3-yaml

$ sudo apt-get install python3-setproctitle python3-xdg python3-pyinotify python3-opencv

$ sudo apt-get install libpam-dev quilt xserver-xorg-dev xutils-dev \

    xserver-xorg-video-dummy xvfb keyboard-configuration

$ sudo apt-get install python-kerberos python-gssapi

$ sudo apt-get install python-avahi

$ sudo apt-get install gstreamer1.0-pulseaudio gstreamer1.0-alsa \

    gstreamer1.0-plugins-base gstreamer1.0-plugins-good \

    gstreamer1.0-plugins-ugly python-gst-1.0

$ sudo apt-get install cups-filters cups-common cups-pdf python3-cups

$ sudo apt-get install openssh-client sshpass python3-paramiko

$ sudo apt-get install dh-systemd

$ wget https://www.xpra.org/src/xpra-4.0.4.tar.xz

$ tar -xf xpra-4.0.4.tar.xz

$ cd xpra-4.0.4

$ sudo ./setup.py install

$ vi setup.py

將第一行改成 python3

$ sudo ./setup.py install

$ sudo mkdir /usr/local/share/xpra/www/js/lib

$ sudo ./setup.py install

LXDE 桌面 lightdm, VNC Server, 中文輸入

$ sudo dpkg-reconfigure lightdm

$ sudo apt update

為避免 lightdm 黑畫面，無法登入，需要鎖住 systemd 不要升級

$ sudo apt-mark hold systemd

若有一天問題解決，解開封鎖

$ sudo apt-mark unhold systemd

$ sudo apt upgrade

$ sudo apt install x11vnc

$ sudo x11vnc -storepasswd

$ sudo vi /etc/systemd/system/x11vnc.service

# Description: Custom Service Unit file

# File: /etc/systemd/system/x11vnc.service

[Unit]

Description="x11vnc"

Requires=display-manager.service

After=display-manager.service

[Service]

ExecStart=/usr/bin/x11vnc -loop -nopw -xkb -repeat -noxrecord -noxfixes -noxdamage -forever -rfbport 5900 -display :0 -auth guess -rfbauth /home/nano/.vnc/passwd

ExecStop=/usr/bin/killall x11vnc

Restart=on-failure

RestartSec=2

[Install]

WantedBy=multi-user.target

$ sudo systemctl enable x11vnc.service

$ sudo systemctl start x11vnc.service

$ xrandr -q

$ sudo vi /etc/X11/xorg.conf

Section "Monitor"

    Identifier "DSI-0"

    Option "Ignore"

EndSection

Section "Screen"

    Identifier "Default Screen"

    Monitor "Configured Monitor"

    Device "Default Device"

    SubSection "Display"

        Depth 24

        Virtual 1280 800

    EndSubSection

EndSection

$ sudo apt-get install fcitx fcitx-chewing

Start up/Preferences/Language Support

Keyboard input method system: fcitx

Reboot

Start up/Preferences/Fcitx Configuration

按 + 並取消打勾 Only Show Current Language

加入 Chewing

Nvidia TLT 分析

參考 Deepstream nvinfer classification 分析

DetectPostprocessor::parseBoundingBox

networkInfo 640x368

outputCoverageLayerIndex

outputCoverageBuffer [4 40 23] 4 個 class 的 confidence

outputBBoxLayerIndex

outputBboxBuffer [16 40 23] numClasses*4, x1, y1, x2, y2

targetShape [40 23]

gridSize 40*23

strideX 16

strideY 16

gcCenters0 [40] (0.5 16.5 32.5...624.5)/35

gcCenters1 [23] (0.5 16.5 32.5...352.5)/35

numClasses 4

ClassifyPostprocessor::parseAttributesFromSoftmaxLayers

m_OutputLayerInfo[1]

m_OutputLayerInfo[1].inferDims[12 1 1] 12 個 class 的 probability

numClasses=12

參考 BBox Ground Truth Generator

cov: Batch_size, Num_classes, image_height/16, image_width/16

bbox: Batch_size, Num_classes * 4, image_height/16, image_width/16 (where 4 is the number of coordinates per cell)

Jupyter Notebook in virtualenv

$ source ~/envs/tensorflow-2.0/bin/activate

(tensorflow-2.0) $ pip3 install ipykernel

(tensorflow-2.0) $ ipython kernel install --user --name=tensorflow-2.0

(tensorflow-2.0) $ jupyter notebook

進入網頁

1. 在 New 一個 Notebook 時選擇 tensorflow-2.0

2. 開啟 ipynb 改變 virtualenv, 由 menu/Kernel/Change kernel/tensorflow-2.0

Deepstream nvinfer classification 分析

nvinfer 用於 classification 主要的參數

# 1: 用於物件偵測, 2:用於物件分類

process-mode=2

# 相隔多少 frames 之後要再推導一遍

secondary-reinfer-interval=0

# 同步 or 非同步模式

classifier-async-mode=1

nvinfer 有三個 threads

gst_nvinfer_submit_input_buffer

 主要 thread, 接收上一級傳來的 inbuf, 處理 resize, crop

 用 push input_queue 驅動 gst_nvinfer_input_queue_loop

gst_nvinfer_input_queue_loop

 處理 color format 轉換

 用 push process_queue 驅動 gst_nvinfer_generate_output

gst_nvinfer_generate_output

 用神經網路推導出 label

同步模式

gst_nvinfer_submit_input_buffer

  gst_nvinfer_process_objects (nvinfer, inbuf, in_surf);

    查看 object_id 是否已經在 object_history_map 中

    should_infer_object() 依據形狀大小(變化)，多久前推導過，判斷是否要推導

    不須推導放入 batch->objs_pending_meta_attach

    若 object_id 不在 object_history_map, 建立它

    若須推導建立 frame, 放入 batch->frames, 連結 object_history_map 和 frame.history

    將來可透過此取得 label

    須推導, 放入 batch->frames

    若收集到夠多的 frames

    convert_batch_and_push_to_input_thread()

      處理 resize, crop

      g_queue_push_tail (nvinfer->input_queue, batch);

  最後 push_buffer, g_queue_push_tail (nvinfer->input_queue, buf_push_batch);

gst_nvinfer_input_queue_loop

 處理 color format 轉換

 用 push process_queue 驅動 gst_nvinfer_generate_output

gst_nvinfer_generate_output

  if push_buffer gst_pad_push() 傳送 inbuf 往下一級

  推導出 label, 透過 frame.history 設定 object_history_map

  attach_metadata_segmentation()

  將之前 batch->objs_pending_meta_attach, attach_metadata_segmentation()

非同步模式

gst_nvinfer_submit_input_buffer

  gst_nvinfer_process_objects (nvinfer, inbuf, in_surf);

    查看 object_id 是否已經在 object_history_map 中

    should_infer_object() 依據形狀大小(變化)，多久前推導過，判斷是否要推導

    若已經有 label, attach_metadata_classifier()

    若 object_id 不在 object_history_map, 建立它

    若須推導建立 frame, 放入 batch->frames, 連結 object_history_map 和 frame.history

    將來可透過此取得 label

    若收集到夠多的 frames

    convert_batch_and_push_to_input_thread()

      處理 resize, crop

      g_queue_push_tail (nvinfer->input_queue, batch);

  gst_pad_push() 傳送 inbuf 往下一級

gst_nvinfer_input_queue_loop

 處理 color format 轉換

 用 push process_queue 驅動 gst_nvinfer_generate_output

gst_nvinfer_generate_output

  推導出 label, 透過 frame.history 設定 object_history_map

CPU GPU(CUDA) 異步並行分析

gst_nvinfer_submit_input_buffer

  cudaMemset2DAsync(convertStream)

gst_nvinfer_input_queue_loop

  cudaStreamWaitEvent(m_PreProcessStream, m_InputConsumedEvent)

  convertFcn(m_PreProcessStream)

  cudaStreamAddCallback(m_PreProcessStream)

  cudaEventRecord(m_PreProcessCompleteEvent, m_PreProcessStream)

  cudaStreamWaitEvent(m_InferStream, m_PreProcessCompleteEvent)

  enqueue(m_InferStream, m_InputConsumedEvent)

  cudaEventRecord(m_InferCompleteEvent, m_InferStream)

  cudaStreamWaitEvent(m_PostprocessStream, m_InferCompleteEvent)

  cudaMemcpyAsync(m_PostprocessStream)

  cudaEventRecord(m_OutputCopyDoneEvent, m_PostprocessStream)

gst_nvinfer_generate_output

  cudaEventSynchronize(m_OutputCopyDoneEvent)

DeepStream 之 gst-dsexample 研究

參考 Implementing a Custom GStreamer Plugin with OpenCV Integration Example

這個範例分成兩個版本

gstdsexample.cpp 和 gstdsexample_optimized.cpp

版本的區分要在 Makefile 的 USE_OPTIMIZED_DSEXAMPLE 設定

要存檔測試時，需打開下列開關，執行時加上 sudo

//#define DSEXAMPLE_DEBUG

在 Jetson 機器上執行時，要在 nvdsosd 和 nveglglessink 中間加上 nvegltransform

在 Tesla 機器上 nvvideoconvert 要加上 nvbuf-memory-type=nvbuf-mem-cuda-unified

gstdsexample.cpp 有 full-frame 和 blur-objects 參數

gstdsexample_optimized.cpp 有 full-frame 和 batch-size 參數

範例如下

$ gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4 \

! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! \

nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/config_infer_primary.txt ! \

nvvideoconvert nvbuf-memory-type=nvbuf-mem-cuda-unified ! 'video/x-raw(memory:NVMM), format=RGBA' ! \

dsexample full-frame=0 blur-objects=1 ! nvdsosd ! nveglglessink

---

gstdsexample.cpp 說明

gst_dsexample_transform_ip()

---

gstdsexample_optimized.cpp 說明

gst_dsexample_start()

建立兩組 queue

  dsexample->process_queue = g_queue_new ();

  dsexample->cvmat_queue = g_queue_new ();

並預先放入兩組 cvmat array 到 cvmat_queue 

  for (int i = 0; i < 2; i++) {

    cvmat = new cv::Mat[dsexample->max_batch_size];

    g_queue_push_tail (dsexample->cvmat_queue, cvmat);

  }

啟動 gst_dsexample_output_loop thread

  dsexample->process_thread =

      g_thread_new ("dsexample-process-thread", gst_dsexample_output_loop,

      dsexample);

gst_dsexample_submit_input_buffer()

啟動於每次資料由 sink 進入

一次的資料會包含多個 frame, 一的 frame 有多個 object

透過 scale_and_fill_data() 將資料放入 dsexample->batch_insurf

當 dsexample->batch_insurf 滿了(或是最後一次未滿) 透過

convert_batch_and_push_to_process_thread() 轉換到 dsexample->inter_buf

  batch->cvmat = (cv::Mat *) g_queue_pop_head (dsexample->cvmat_queue);

  由 dsexample->inter_buf 產生 batch->cvmat

  g_queue_push_tail (dsexample->process_queue, batch);

要離開 gst_dsexample_submit_input_buffer() 時，

多 push 一次輸出訊號(batch->push_buffer = TRUE)

    g_queue_push_tail (dsexample->process_queue, buf_push_batch);

gst_dsexample_output_loop()

輸出的 thread, 一直 loop, 直到結束

  batch = g_queue_pop_head (dsexample->process_queue));

  if (batch->push_buffer) {

    gst_pad_push (GST_BASE_TRANSFORM_SRC_PAD (dsexample), batch->inbuf);

  }

  g_queue_push_tail (dsexample->cvmat_queue, batch->cvmat);