網頁

2021年12月29日 星期三

Install nvdec and nvh264enc as GStreamer plugins

$ sudo apt-get install gtk-doc-tools
$ git clone -b 1.14.5 https://github.com/GStreamer/gst-plugins-bad.git

$ unzip Video_Codec_SDK_11.1.5.zip
$ cp Video_Codec_SDK_11.1.5/Interface/nvEncodeAPI.h gst-plugins-bad/sys/nvenc/
$ cp Video_Codec_SDK_11.1.5/Interface/cuviddec.h gst-plugins-bad/sys/nvdec/
$ cp Video_Codec_SDK_11.1.5/Interface/nvcuvid.h gst-plugins-bad/sys/nvdec/

$ cd gst-plugins-bad/
$ ./autogen.sh --with-cuda-prefix="/usr/local/cuda"
$ cd sys/nvenc/
$ make
$ sudo cp .libs/libgstnvenc.so /usr/lib/x86_64-linux-gnu/gstreamer-1.0/
$ cd ../nvdec
$ make
$ sudo cp .libs/libgstnvdec.so /usr/lib/x86_64-linux-gnu/gstreamer-1.0/
$ gst-inspect-1.0 nvh264enc
$ gst-inspect-1.0 nvdec


2021年12月20日 星期一

Ubuntu 加入 RamDisk

$ sudo mkdir /mnt/ramdisk
$ sudo chmod 1777 /mnt/ramdisk
$ sudo vi /etc/fstab
tmpfs /mnt/ramdisk tmpfs rw,size=1G 0 0

$ sudo reboot

set frpc as service in Ubuntu

vi /etc/systemd/system/frpc.service
[Unit]
Description=frp client
Wants=network-online.target
After=network.target network-online.target

[Service]
ExecStart=/path_to/frpc -c /path_to/frpc.ini

[Install]
WantedBy=multi-user.target

$ sudo systemctl daemon-reload
$ sudo systemctl enable frpc
$ sudo systemctl start frpc
$ sudo systemctl status frpc
$ sudo systemctl staop frpc

2021年12月17日 星期五

GStreamer 使用 rstp

URI="rtsp://user:password@192.168.1.123:554/cam/realmonitor?channel=1&subtype=0"

使用 playbin
$ gst-launch-1.0 -v playbin uri=$URI
$ gst-launch-1.0 -v playbin uri=$URI uridecodebin0::source::latency=3000::video-sink=autovideosink

使用 uridecodebin
$ gst-launch-1.0 uridecodebin uri=$URI name=d d. ! nvvideoconvert ! nveglglessink
$ gst-launch-1.0 uridecodebin uri=$URI name=d d. ! nvvideoconvert ! nveglglessink d. ! queue ! audioconvert ! audioresample ! autoaudiosink

使用 decodebin
$ gst-launch-1.0 rtspsrc location=$URI ! application/x-rtp, media=video ! decodebin ! nvvideoconvert ! nveglglessink
$ gst-launch-1.0 rtspsrc location=$URI ! application/x-rtp, media=video ! decodebin name=d d. ! queue ! nvvideoconvert ! nveglglessink d. ! queue ! audioconvert ! audioresample ! autoaudiosink

自行編寫
$ gst-launch-1.0 rtspsrc location=$URI ! application/x-rtp, media=video ! queue ! rtph265depay ! h265parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink
$ gst-launch-1.0 rtspsrc location=$URI name=d d. ! application/x-rtp, media=video ! queue ! rtph265depay ! h265parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink d. ! queue ! application/x-rtp, media=audio ! rtppcmadepay ! alawdec ! autoaudiosink

存檔
$ gst-launch-1.0 -e rtspsrc location=$URI ! application/x-rtp, media=video ! queue ! rtph265depay ! h265parse ! matroskamux ! filesink location=aa.mkv
存檔並顯示
$ gst-launch-1.0 -e rtspsrc location=$URI ! application/x-rtp, media=video ! queue ! rtph265depay ! h265parse ! tee name=t  t. ! queue ! avdec_h265 ! videoconvert ! autovideosink t. ! queue ! matroskamux ! filesink location=aa.mkv
存檔並顯示(因使用 nvv4l2decoder,所以要在 h265parse 前 tee)
$ gst-launch-1.0 -e rtspsrc location=$URI ! application/x-rtp, media=video ! queue ! rtph265depay ! tee name=t  t. ! queue ! h265parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink t. ! queue ! h265parse ! matroskamux ! filesink location=aa.mkv

常用
nveglglessink nvoverlaysink autovideosink xvimagesink fakesink
nvvideoconvert videoconvert
nvv4l2decoder avdec_h265

Jetson 使用 nveglglessink 前要加 nvegltransform

gst-inspect-1.0

2021年12月16日 星期四

Ubuntu 安裝 DeepStream

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install python3-pip
$ cd /usr/bin
$ sudo ln -s python3 python

一般顯卡參考 dGPU Setup for Ubuntu 段落
依據文件要求安裝 NVIDIA driver 470.63.01
但是據上次安裝經驗不要使用 NVIDIA-Linux-x86_64-470.63.01.run
$ ubuntu-drivers devices
$ sudo apt-get install nvidia-driver-470
$ sudo reboot

依據文件說明安裝 CUDA Toolkit 11.4 Update 1
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
$ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
$ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /"
$ sudo apt-get update
$ sudo apt-get -y install cuda

依據文件安裝 TensorRT 8.0.1
$ echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 /" | sudo tee /etc/apt/sources.list.d/cuda-repo.list
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
$ sudo apt-key add 7fa2af80.pub
$ sudo apt-get update
依據鏈結下載 TensorRT 8.0.1 GA for Ubuntu 18.04 and CUDA 11.3 DEB
$ sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626_1-1_amd64.deb
$ sudo apt-key add /var/nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626/7fa2af80.pub
$ sudo apt-get update
$ sudo apt-get install -y libnvinfer8=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvinfer-plugin8=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvparsers8=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvonnxparsers8=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvinfer-bin=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvinfer-dev=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvinfer-plugin-dev=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvparsers-dev=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvonnxparsers-dev=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvinfer-samples=8.0.1-1+cuda11.3 
$ sudo apt-get install -y libnvinfer-doc=8.0.1-1+cuda11.3

安裝 librdkafka
$ git clone https://github.com/edenhill/librdkafka.git
$ cd librdkafka/
$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a
$ .configure
$ make
$ sudo make install
$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-6.0/lib
$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-6.0/lib

$ sudo apt-get install ./deepstream-6.0_6.0.0-1_amd64.deb
$ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin
$ cd /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/
$ deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt

安裝 Docker
$ sudo apt-get update
$ sudo apt-get install -y ca-certificates
$ sudo apt-get install -y curl
$ sudo apt-get install -y gnupg
$ sudo apt-get install -y lsb-release
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
$ echo \
$   "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$   $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io
$ sudo docker run hello-world
$ sudo docker docker image list
$ sudo docker image rm hello-world

安裝 Nvidia Container Toolkit
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
$ curl -s -L https://nvidia.github.io/nvidia-container-runtime/experimental/$distribution/nvidia-container-runtime.list | sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list
$ sudo apt-get update
$ sudo apt-get install -y nvidia-docker2
$ sudo systemctl restart docker
$ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

在 Docker 內,執行 deepstream-app
參考 Docker Containers 下載 docker image
$ docker pull nvcr.io/nvidia/deepstream:6.0-samples
參考 Nvidia NGC DeepStream 執行程式
$ xhost +
$ sudo docker run --gpus '"'device=0'"' -it --rm -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0-samples
# cd samples/configs/deepstream-app
# deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt



Ubuntu 安裝 TeamViewer

到官方網站下載 Ubuntu 安裝 deb

$ sudo apt install gdebi-core
$ sudo gdebi teamviewer_15.25.5_amd64.deb
$ teamviewer

2021年12月15日 星期三

Ubuntu 不接螢幕,使用遠端桌面

$ sudo apt-get install xserver-xorg-video-dummy-hwe-18.04
$ sudo vi /usr/share/X11/xorg.conf.d/xorg.conf
Section "Device"    Identifier  "Configured Video Device"
    Driver      "dummy"
    VideoRam 256000
EndSection
 
 
Section "Monitor"
    Identifier  "Configured Monitor"
    HorizSync 5.0 - 1000.0
    VertRefresh 5.0 - 200.0
    ModeLine "1920x1080" 148.50 1920 2448 2492 2640 1080 1084 1089 1125 +Hsync +Vsync
EndSection
 
 
Section "Screen"
    Identifier  "Default Screen"
    Monitor     "Configured Monitor"
    Device      "Configured Video Device"
    DefaultDepth 24
    SubSection "Display"
    Depth 24
    Modes "1920x1080"
    EndSubSection
EndSection


因為要 login 才能有桌面,所以
start/settings/Details/Users
Automatic Login: ON

DeepStream 因為沒接實體螢幕
sink 不能使用 EglSink,只能使用 FakeSink or File


Ubuntu 安裝時出現錯誤

因為 nvidia 顯卡驅動問題出現
nouveau DRM: failed to create kernel channel, -22
等錯誤

進 GRUB menu 介面時,移到 Install Ubuntu,按 e 進入編輯環境
游標移到 linux 行,將行尾的 --- 改成 nomodeset
按 F10 開機

正常安裝後,系統顯示拔走 usb,按 Enter
按 Esc(or Shift),進入 GRUB menu
(若按太多下,會進入 grub>,輸入 normal,按 Enter 後馬上按 Esc,回到 GRUB menu)
移到 Ubuntu,按 e 進入編輯環境
尋找 quiet splash 後面加入 nomodeset
按 F10 開機

2021年12月13日 星期一

python onvif

https://github.com/quatanium/python-onvif
只適用 python2
https://github.com/FalkTannhaeuser/python-onvif-zeep
才能在 python3 使用


2021年11月25日 星期四

Ubuntu 中文輸入

Xavier
$ sudo apt install ibus-chewing

System Settings/Language Support
按 Install / Remove Langeages...
點選 Chinese (traditional)
按 Apply

$ sudo reboot

System Settings/Text Entry
按 + 選 Chinese (chewing)(IBus)

$ sudo reboot

auto mount nfs after boot

$ sudo vi /etc/fstab
ip:/folder /media/folder        auto    noauto,x-systemd.automount 0 0

Install JetPack 4.6

參考 https://yingrenn.blogspot.com/2020/05/install-jetpack-44.html

$ sudo apt update
$ sudo dpkg -i sdkmanager_1.6.1-8175_amd64.deb
(Reading database ... 346387 files and directories currently installed.)
Preparing to unpack sdkmanager_1.6.1-8175_amd64.deb ...
Unpacking sdkmanager (1.6.1-8175) over (1.6.1-8175) ...
dpkg: dependency problems prevent configuration of sdkmanager:
 sdkmanager depends on libgconf-2-4; however:
  Package libgconf-2-4 is not installed.
 sdkmanager depends on libcanberra-gtk-module; however:
  Package libcanberra-gtk-module is not installed.

dpkg: error processing package sdkmanager (--install):
 dependency problems - leaving unconfigured
Processing triggers for gnome-menus (3.13.3-11ubuntu1.1) ...
Processing triggers for desktop-file-utils (0.23-1ubuntu3.18.04.2) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
Errors were encountered while processing:
 sdkmanager

出錯,使用下列命令
$ sudo apt --fix-broken install

$ sdkmanager

2021年11月23日 星期二

nginx, tomcat and mariadb on QNAP

[/share/Dockers/mariadb] # cat 0_run.sh
docker run --detach --name mariadb \
--env MARIADB_ROOT_PASSWORD=mark1234 \
-v /share/Dockers/mariadb/test:/test \
-v /share/Dockers/mariadb/conf.d:/etc/mysql/conf.d \
mariadb:latest

/share/Dockers/mariadb/test 可放置一些 create database sql

[/share/Dockers/mariadb] # cat conf.d/char.cnf
[client]
# Default is Latin1, if you need UTF-8 set this (also in server section)
default-character-set = utf8mb4

[mysql]
# Default is Latin1, if you need UTF-8 set this (also in server section)
default-character-set = utf8mb4

[/share/Dockers/mariadb] # cat cs_get.sh
CONTAINER_ID=$(docker ps -qf "name=mariadb")
echo ${CONTAINER_ID}
CONTAINER_ID=$(cd /var/lib/docker/containers && ls -d ${CONTAINER_ID}*)
echo ${CONTAINER_ID}
QIP=127.0.0.1
QPORT=8080
curl -sq -XPOST -c cookies.txt -d '{"username": "admin", "password": "password"}' http://${QIP}:${QPORT}/container-station/api/v1/login
curl -sq -XGET -b cookies.txt http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/all

[/share/Dockers/mariadb] # cat cs_set.sh
CONTAINER_ID=$(docker ps -qf "name=mariadb")
echo ${CONTAINER_ID}
CONTAINER_ID=$(cd /var/lib/docker/containers && ls -d ${CONTAINER_ID}*)
echo ${CONTAINER_ID}
QIP=127.0.0.1
QPORT=8080
curl -sq -XPOST -c cookies.txt -d '{"username": "admin", "password": "password"}' http://${QIP}:${QPORT}/container-station/api/v1/login
curl -sq -XPUT -b cookies.txt http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/autostart/on
curl -sq -XPOST -b cookies.txt -d \
    '{
        "cputime": 500,
        "memory": "1024m"
    }' http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/resource/limit

因為此版本已經不建議使用 manager-gui
直接利用 webapps
[/share/Dockers/tomcat] # ls
0_run.sh*  build_docker/  cookies.txt  cs_set.sh*  webapps/
aa.sh*     conf/          cs_get.sh*   test.war
[/share/Dockers/tomcat] # cat 0_run.sh
rm -rf webapps/*
cp test.war webapps

docker run -d \
--link mariadb:mariadb \
-p 8888:8080 \
-v /share/Dockers/tomcat/webapps:/usr/local/tomcat/webapps \
-v /share/Dockers/tomcat/conf/context.xml:/usr/local/tomcat/conf/context.xml \
-v /share/Dockers/tomcat/conf/server.xml:/usr/local/tomcat/conf/server.xml \
--name tomcat tomcat:9.0.46

新增 Resource 部分
[/share/Dockers/tomcat] # cat conf/context.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
...
<Context>

    <!-- Default set of monitored resources. If one of these changes, the    -->
    <!-- web application will be reloaded.                                   -->
    <WatchedResource>WEB-INF/web.xml</WatchedResource>
    <WatchedResource>WEB-INF/tomcat-web.xml</WatchedResource>
    <WatchedResource>${catalina.base}/conf/web.xml</WatchedResource>

    <!-- Uncomment this to disable session persistence across Tomcat restarts -->
    <!--
    <Manager pathname="" />
    -->
    <Resource
        name="jdbc/MyDb"
        auth="Container"
        type="javax.sql.DataSource"
        maxActive="100"
        maxIdle="30"
        maxWait="500"
        driverClassName="org.mariadb.jdbc.Driver"
        url="jdbc:mariadb://mariadb:3306"
        username="mast"
        password="mast"
    />
</Context>

新增 maxDays 防止 log 太大
[/share/Dockers/tomcat] # cat conf/server.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
...
        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               rotatable="true" renameOnRotate="true" maxDays="3"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>


[/share/Dockers/tomcat] # cat cs_get.sh
CONTAINER_ID=$(docker ps -qf "name=tomcat")
echo ${CONTAINER_ID}
CONTAINER_ID=$(cd /var/lib/docker/containers && ls -d ${CONTAINER_ID}*)
echo ${CONTAINER_ID}
QIP=127.0.0.1
QPORT=8080
curl -sq -XPOST -c cookies.txt -d '{"username": "admin", "password": "password"}' http://${QIP}:${QPORT}/container-station/api/v1/login
curl -sq -XGET -b cookies.txt http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/all

[/share/Dockers/tomcat] # cat cs_set.sh
CONTAINER_ID=$(docker ps -qf "name=tomcat")
echo ${CONTAINER_ID}
CONTAINER_ID=$(cd /var/lib/docker/containers && ls -d ${CONTAINER_ID}*)
echo ${CONTAINER_ID}
QIP=127.0.0.1
QPORT=8080
curl -sq -XPOST -c cookies.txt -d '{"username": "admin", "password": "password"}' http://${QIP}:${QPORT}/container-station/api/v1/login
curl -sq -XPUT -b cookies.txt http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/autostart/on
curl -sq -XPOST -b cookies.txt -d \
    '{
        "cputime": 500,
        "memory": "1024m"
    }' http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/resource/limit

[/share/Dockers/nginx] # cat 0_run.sh
docker run -d \
--link tomcat:tomcat \
-p 8443:80 \
-v /share/Dockers/nginx/conf.d:/etc/nginx/conf.d \
-v /share/Dockers/nginx/conf:/etc/nginx/conf \
--name nginx nginx:1.14

[/share/Dockers/nginx] # ls conf
Certs/

[/share/Dockers/nginx] # cat conf.d/default.conf
upstream tomcat_server {
    server tomcat:8080 weight=1;
}
server {
    #listen       80;
    listen       80 ssl;
    ssl_certificate /etc/nginx/conf/Certs/server.pem;
    ssl_certificate_key /etc/nginx/conf/Certs/server.key;
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/nginx/conf/Certs/caChain.crt;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    #server_name  localhost;
    server_name  www.sunhousetechnology.com.tw;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
        #proxy_pass http://tomcat_server;
    }
    location ^~ /test/ {
        proxy_redirect off;
        proxy_set_header Host $host:$server_port;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        # for https
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://tomcat_server;
    }
    client_max_body_size 20M;

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}

[/share/Dockers/nginx] # cat cs_get.sh
CONTAINER_ID=$(docker ps -qf "name=nginx")
echo ${CONTAINER_ID}
CONTAINER_ID=$(cd /var/lib/docker/containers && ls -d ${CONTAINER_ID}*)
echo ${CONTAINER_ID}
QIP=127.0.0.1
QPORT=8080
curl -sq -XPOST -c cookies.txt -d '{"username": "admin", "password": "password"}' http://${QIP}:${QPORT}/container-station/api/v1/login
curl -sq -XGET -b cookies.txt http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/all

[/share/Dockers/nginx] # cat cs_set.sh
CONTAINER_ID=$(docker ps -qf "name=nginx")
echo ${CONTAINER_ID}
CONTAINER_ID=$(cd /var/lib/docker/containers && ls -d ${CONTAINER_ID}*)
echo ${CONTAINER_ID}
QIP=127.0.0.1
QPORT=8080
curl -sq -XPOST -c cookies.txt -d '{"username": "admin", "password": "password"}' http://${QIP}:${QPORT}/container-station/api/v1/login
curl -sq -XPUT -b cookies.txt http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/autostart/on
curl -sq -XPOST -b cookies.txt -d \
    '{
        "cputime": 500,
        "memory": "1024m"
    }' http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/resource/limit


2021年11月22日 星期一

gitea on QNAP (設定 Auto Start, CPU Memory 限制)

參考: http://qnap-dev.github.io/container-station-api/index.html
因為使用 QNAP 的 container station api, 不能使用 QNAP 的 2-step verification

[/share/Dockers/gitea] # cat 0_run.sh
#mkdir -p data/git
#mkdir -p data/gitea

docker run -d --name=gitea \
-p 10022:22 -p 3000:3000 \
-v /share/Dockers/gitea/data/gitea:/data/gitea \
-v /share/Dockers/gitea/data/git:/data/git \
gitea/gitea:latest

[/share/Dockers/gitea] # cat cs_get.sh
CONTAINER_ID=$(docker ps -qf "name=gitea")
echo ${CONTAINER_ID}
CONTAINER_ID=$(cd /var/lib/docker/containers && ls -d ${CONTAINER_ID}*)
echo ${CONTAINER_ID}
QIP=127.0.0.1
QPORT=8080
curl -sq -XPOST -c cookies.txt -d '{"username": "admin", "password": "password"}' http://${QIP}:${QPORT}/container-station/api/v1/login
curl -sq -XGET -b cookies.txt http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/all

[/share/Dockers/gitea] # cat cs_set.sh
CONTAINER_ID=$(docker ps -qf "name=gitea")
echo ${CONTAINER_ID}
CONTAINER_ID=$(cd /var/lib/docker/containers && ls -d ${CONTAINER_ID}*)
echo ${CONTAINER_ID}
QIP=127.0.0.1
QPORT=8080
curl -sq -XPOST -c cookies.txt -d '{"username": "admin", "password": "password"}' http://${QIP}:${QPORT}/container-station/api/v1/login
curl -sq -XPUT -b cookies.txt http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/autostart/on
curl -sq -XPOST -b cookies.txt -d \
    '{
        "cputime": 500,
        "memory": "1024m"
    }' http://${QIP}:${QPORT}/container-station/api/v1/container/docker/${CONTAINER_ID}/resource/limit

2021年11月17日 星期三

frps on QNAP

使用 ssh 登入 QNAP
# mkdir -p /share/Dockers/frp/
# vi /share/Dockers/frp/frps.ini
bind_port = 7000
dashboard_port = 7001
dashboard_user = user
dashboard_pwd = password

# curl -L https://github.com/fatedier/frp/releases/download/v0.31.2/frp_0.31.2_linux_amd64.tar.gz --output frp.tar.gz
# gzip -d frp.tar.gz
# tar frp.tar
# vi /share/Dockers/frp/Dockerfile
FROM ubuntu:18.04
MAINTAINER mark

COPY frp_0.31.2_linux_amd64/ /home/frp

ENTRYPOINT ["/bin/sh","-c","/home/frp/frps -c /home/frp/frps.ini"]

# vi /share/Dockers/frp/0_build.sh
docker build -t frps_0.31.2 . --no-cache

# vi /share/Dockers/frp/1_run.sh
#!/bin/bash
F_NAME=1_run_.sh
>$F_NAME
echo "docker run -d \\">>$F_NAME
echo "-p 7000:7000 -p 7001:7001 \\">>$F_NAME
while read line; do
  echo $line
  port=($(echo $line | tr ':' "\n"))
  echo "-p $port:$port \\">>$F_NAME
done < nat.txt
echo "--name frps frps_0.31.2:latest">>$F_NAME
chmod +x $F_NAME
./$F_NAME

# vi /share/Dockers/frp/nat.txt
7002:host 1
7003:host 2

# chmod +x /share/Dockers/frp/0_build.sh
# chmod +x /share/Dockers/frp/1_run.sh
# ./0_build.sh
# ./1_run.sh

QNAP/Container Station/Container
按 frps 連結
按 Settings
設定 Auto start, CPU Limit, Memory Limit

QNAP/Container Station/Images
刪除 frps_0.31.2:latest, 因為上個動作已經產生 frps_0.31.2:latest-data-1

gitea on QNAP

開啟 QNAP ssh
QNAP/Control Panel/Network & File Services/Telnet / SSH
Allow SSH connection
Apply

建立 docker 使用空間
QNAP/Control Panel/Privilege/Shared Folders
Create/Shared Folder
Folder Name: Dockers
Next/Next/Finish

建立 Gitea Docker
使用 ssh 登入 QNAP
Console Management - Main menu
>> q
>> y
# cd /share/Dockers
# mkdir -p /share/Dockers/gitea
# docker pull gitea/gitea:latest
# docker run -d --name=gitea -p 10022:22 -p 3000:3000 -v /share/Dockers/gitea:/data gitea/gitea:latest

設定 Gitea 帳號
網頁登入 http://qnap_ip:3000/
底部/管理員帳戶設定
輸入 管理員帳號 密碼 電子信箱
安裝 Gitea

進入 Gitea 建立 Repository
網頁登入 http://qnap_ip:3000/
右上/+/新增儲存庫
輸入 儲存庫名稱
建立儲存庫

由 Git 匯入 Gitea
進入 git host 的目錄
> git remote add gitea http://qnap_ip:3000/UserName/Repository.git
> git push gitea --all
Username for 'http://qnap_ip:3000':
Password for 'http://UserName@qnap_ip:3000':
> git push gitea --tags
Username for 'http://qnap_ip:3000':
Password for 'http://UserName@qnap_ip:3000':

Gitea 備份 兩種方法可選
(1.進入 docker)
QNAP/Container Station/Container
按 gitea 連結,開啟 console logs
按 >_ Terminal
/bin/sh
Connect
# su - git
$ gitea dump -c /data/gitea/conf/app.ini
產生 /data/git/gitea-dump-xxxxx.zip
位於 QNAP 的 /share/Dockers/gitea/git
(2.進入 QNAP)
使用 ssh 登入 QNAP
# docker exec -u git -it -w /data/git $(docker ps -qf "name=gitea") bash -c '/app/gitea/gitea dump -c /data/gitea/conf/app.ini'
產生 /share/Dockers/gitea/git/gitea-dump-xxxxx.zip
位於 docker 的 /data/git

Gitea 還原
QNAP/Container Station/Container
按 gitea 連結,開啟 console logs
按 Stop
使用 ssh 登入 QNAP
# mkdir -p /share/Dockers/backup
# mv /share/Dockers/gitea/git/gitea-dump-xxxxx.zip /share/Dockers/backup
# cd /share/Dockers/backup
# unzip gitea-dump-xxxxx.zip
# ls /share/Dockers/backup/data/
avatars/  gitea.db   jwt/     repo-archive/
conf/     indexers/  queues/  repo-avatars/
# ls /share/Dockers/gitea/gitea/
attachments/  conf/     indexers/  log/     repo-archive/  sessions/
avatars/      gitea.db  jwt/       queues/  repo-avatars/
移除 /share/Dockers/gitea/gitea/ 內相應 /share/Dockers/backup/data/ 的資料
# mv /share/Dockers/backup/data/* /share/Dockers/gitea/gitea/
# chown -R owner.group /share/Dockers/gitea/gitea/
# rm -rf /share/Dockers/gitea/git/repositories/*
# mv /share/Dockers/backup/repos/* /share/Dockers/gitea/git/repositories/
# chown -R owner.group /share/Dockers/gitea/git/repositories/
Gitea 文件上說明,還要搬 log, database
但 log 不見了, database 使用 sqlite3, 直接是一檔案, 搬 data/gitea.db 後不須重建
QNAP/Container Station/Container
按 gitea 連結,開啟 console logs
按 Start
# rm -rf /share/Dockers/backup

Gitea Migrations 遷移
在 Gitea 1.12 以前,可以遷移 程式碼 問題 等
但在之後,無法遷移程式碼
遷移預設無法在本地端執行,需在 /data/gitea/conf/app.ini
增加下列兩行
[migrations]
ALLOW_LOCALNETWORKS = true
要遷移 Wiki 里程碑 標籤 等,需要 Access Token
網頁登入 http://qnap_ip:3000/
右上/圖標/設定/應用程式
輸入 Token 名稱
產生 Token
網頁登入 http://qnap_ip:3000/
右上/+/遷移外部儲存庫

2021年9月22日 星期三

安裝 labelImg

參考 LabelImg GitHub

正常時
sudo apt-get install pyqt5-dev-tools
sudo pip3 install -r requirements/requirements-linux-python3.txt
make qt5py3
python3 labelImg.py

但是安裝 pyqt5-dev-tools 會造成版本衝突
所以到 https://download.qt.io/archive/qt 下載 qt-opensource-linux-x64-5.x.x.run
chmod +x qt-opensource-linux-x64-5.x.x.run
sudo ./qt-opensource-linux-x64-5.x.x.run
安裝到 /opt/Qt5.x.x
移除時使用 /opt/Qt5.x.x/MaintenanceTool
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/Qt5.10.1/5.10.1/gcc_64/lib

參考 OpenCV 安裝部分 qt5 不要安裝
sudo apt-get install qt5-default
sudo apt-get install qtcreator
cmake 增加
-D Qt5_DIR=/opt/Qt5.x.x/5.x.x/gcc_64/lib/cmake/Qt5 \


另外 安裝除錯時使用命令
apt list --installed
sudo apt-get install qt5-default
sudo apt-get purge -y qt5-default
pip install opencv-python
pip uninstall opencv-python
pip install -r requirements/requirements-linux-python3.txt
pip uninstall -r requirements/requirements-linux-python3.txt
安裝指定版本
pip install pyqt5==5.14.2
查詢可用版本
pip install pyqt5==
查詢 QT 的執行錯誤
export QT_DEBUG_PLUGINS=1


2021年6月30日 星期三

nvidia driver on ubuntu

$ lspci -k | grep -A 2 -i "VGA"
$ ubuntu-drivers devices
$ sudo ubuntu-drivers autoinstall

不建議下載 nvidia 網站的 driver


CUDNN 安裝

顯卡 驅動 CUDA CUDNN 版本關係
https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html

安裝
https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

2021年6月25日 星期五

ubuntu 換接顯示器 intel nvidia

圖形模式選擇
$ nvidia-settings
開啟 NVIDIA X Server Settings
選擇左邊 Tab 之 PRIME Profiles
右邊 Tab 出現
NVIDIA (Performance Mode)
NVIDIA On-Demand
Intel (Prower Saving Mode)
三種選項

命令模式選擇
$ sudo prime-select -?
$ sudo prime-select query
$ sudo prime-select on-deamand

開機直接進入圖形介面
$ sudo systemctl set-default graphical.target
開機直接進入文字介面
$ sudo systemctl set-default multi-user.target

Console 切換 Ctrl + Alt + Fn
Intel 上會顯示文字模式的 login:
Nvidia 上只有 Ctrl + Alt + F1 的圖形模式


2021年6月17日 星期四

LPD LPR on DeepStream

參考
https://developer.nvidia.com/blog/creating-a-real-time-license-plate-detection-and-recognition-app
https://github.com/NVIDIA-AI-IOT/deepstream_lpr_app
https://blogs.nvidia.com.tw/2021/05/12/creating-a-real-time-license-plate-detection-and-recognition-app/

依據 CUDA cuDNN TensorRT 的版本,下載 tlt-converter
https://developer.nvidia.com/tlt-get-started


完整程序
~/Data/DeepStream$ mkdir deepstream_lpr_app
~/Data/DeepStream$ cd deepstream_lpr_app/
~/Data/DeepStream/deepstream_lpr_app$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_lpr_app.git
~/Data/DeepStream/deepstream_lpr_app$ cd deepstream_lpr_app/
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app$ ./download_us.sh
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app$ ../../tlt-converter/cuda11.0_cudnn8.0_trt7.2/tlt-converter -k nvidia_tlt -p image_input,1x3x48x96,4x3x48x96,16x3x48x96 models/LP/LPR/us_lprnet_baseline18_deployable.etlt -t fp16 -e models/LP/LPR/lpr_us_onnx_b16.engine
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app$ make
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app$ cd deepstream-lpr-app/
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/deepstream-lpr-app$ cp dict_us.txt dict.txt
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/deepstream-lpr-app$ ./deepstream-lpr-app 1 2 0 /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.mp4 output.264
以上可以執行,並在第一次執行時產生 engine
也可以利用下列命令 tlt-converter 產生 engine
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/deepstream-lpr-app$ cd ../models/tlt_pretrained_models/trafficcamnet/
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/models/tlt_pretrained_models/trafficcamnet$ mv trafficnet_int8.txt trafficnet_int8.bin
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/models/tlt_pretrained_models/trafficcamnet$ ../../../../../tlt-converter/cuda11.0_cudnn8.0_trt7.2/tlt-converter -k tlt_encode -o output_bbox/BiasAdd,output_cov/Sigmoid -d 3,544,960 -t int8 -c trafficnet_int8.bin -e resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine resnet18_trafficcamnet_pruned.etlt
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/models/tlt_pretrained_models/trafficcamnet$ cd ../../LP/LPD/
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/models/LP/LPD$ ../../../../../tlt-converter/cuda11.0_cudnn8.0_trt7.2/tlt-converter -k nvidia_tlt -o output_bbox/BiasAdd,output_cov/Sigmoid -d 3,480,640 -t int8 -c usa_lpd_cal.bin -e usa_pruned.etlt_b16_gpu0_int8.engine usa_pruned.etlt
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/models/LP/LPD$ cd ../../../deepstream-lpr-app/
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/deepstream-lpr-app$ vi lpd_us_config.txt

不顯示
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/deepstream-lpr-app$ ./deepstream-lpr-app 1 2 0 /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.mp4 output.264
有顯示
~/Data/DeepStream/deepstream_lpr_app/deepstream_lpr_app/deepstream-lpr-app$ ./deepstream-lpr-app 1 3 0 /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.mp4 output.264


重新訓練模型
使用 tlt-launcher
python3 -m venv --system-site-packages tlt
source ~/envs/tlt/bin/activate
(tlt) $ pip3 install nvidia-pyindex
(tlt) $ pip3 install nvidia-tlt
(tlt) $ cat ~/.tlt_mounts.json
{
    "Mounts": [
        {
            "source": "/home/username/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments",
            "destination": "/workspace/tlt-experiments"
        },
        {
            "source": "/home/username/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr",
            "destination": "/workspace/openalpr"
        }

    ]
}
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr$ git clone https://github.com/openalpr/benchmarks benchmarks
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr$ mkdir tlt-experiments
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr$ mkdir openalpr
重新訓練 LPD 模型
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr$ cd openalpr/
下載 https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/release/tlt3.0/misc/dev_blog/LPDR/lpd/lpd_prepare_data.py
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ python3 lpd_prepare_data.py --input_dir ../benchmarks/endtoend/us --output_dir lpd --target_width 640 --target_height 480
下載 https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/release/tlt3.0/misc/dev_blog/LPDR/lpd/SPECS_tfrecord.txt
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ tlt detectnet_v2 dataset_convert -d /workspace/openalpr/SPECS_tfrecord.txt -o /workspace/openalpr/lpd_tfrecord/lpd
下載 https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/release/tlt3.0/misc/dev_blog/LPDR/lpd/SPECS_train.txt
下載預先訓練模型 (tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ ngc registry model download-version "nvidia/tlt_lpdnet:unpruned_v1.0"
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ mv tlt_lpdnet_vunpruned_v1.0/usa_unpruned.tlt .
訓練
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ tlt detectnet_v2 train -e /workspace/openalpr/SPECS_train.txt -r /workspace/openalpr/exp_unpruned -k nvidia_tlt
修剪
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ tlt detectnet_v2 prune -m /workspace/openalpr/exp_unpruned/weights/model.tlt -o /workspace/openalpr/exp_pruned/model_pruned_06.tlt -pth 0.6 -k nvidia_tlt
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ cp SPECS_train.txt SPECS_retrain.txt
修改 pretrained_model_file: "/workspace/openalpr/exp_pruned/model_pruned_06.tlt"
再次訓練
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ tlt detectnet_v2 train -e /workspace/openalpr/SPECS_retrain.txt -r /workspace/openalpr/exp_pruned -k nvidia_tlt
匯出
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ tlt detectnet_v2 export -m /workspace/openalpr/exp_unpruned/weights/model.tlt -o /workspace/openalpr/export/unpruned_model.etlt --cal_cache_file /workspace/openalpr/export/unpruned_calibration.bin -e /workspace/openalpr/SPECS_train.txt -k nvidia_tlt --cal_image_dir /workspace/openalpr/lpd/data/image --data_type int8 --batch_size 4 --batches 10 --engine_file /workspace/openalpr/export/unpruned_int8.trt
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ tlt detectnet_v2 export -m /workspace/openalpr/exp_pruned/weights/model.tlt -o /workspace/openalpr/export/pruned_model.etlt --cal_cache_file /workspace/openalpr/export/pruned_calibration.bin -e /workspace/openalpr/SPECS_train.txt -k nvidia_tlt --cal_image_dir /workspace/openalpr/lpd/data/image --data_type int8 --batch_size 4 --batches 10 --engine_file /workspace/openalpr/export/pruned_int8.trt
評估
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ tlt detectnet_v2 evaluate -m /workspace/openalpr/exp_unpruned/weights/model.tlt -k nvidia_tlt -e /workspace/openalpr/SPECS_train.txt
(tlt) mark@z390:~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/openalpr$ tlt detectnet_v2 evaluate -m /workspace/openalpr/exp_pruned/weights/model.tlt -k nvidia_tlt -e /workspace/openalpr/SPECS_train.txt
重新訓練 LPR 模型
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments$ mkdir lprnet
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments$ cd lprnet/
下載 https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/release/tlt3.0/misc/dev_blog/LPDR/lpr/preprocess_openalpr_benchmark.py
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments/lprnet$ python3 preprocess_openalpr_benchmark.py --input_dir=../../benchmarks/endtoend/us --output_dir=../data/openalpr
下載 https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/release/tlt3.0/misc/dev_blog/LPDR/lpr/tutorial_spec.txt
修改 tutorial_spec.txt
augmentation_config {
    output_width: 96
    output_height: 48
    output_channel: 3
    keep_original_prob: 0.3
    transform_prob: 0.5
    rotate_degree: 5
}
成為
augmentation_config {
    output_width: 96
    output_height: 48
    output_channel: 3
    keep_original_prob: 0.3
    max_rotate_degree: 5
    rotate_prob: 0.5
    blur_prob: 0.5
}

下載預先訓練模型
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments/lprnet$ ngc registry model download-version "nvidia/tlt_lprnet:trainable_v1.0"
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments/lprnet$ cp tlt_lprnet_vtrainable_v1.0/us_lprnet_baseline18_trainable.tlt .
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments/lprnet$ mkdir specs
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments/lprnet$ cp tlt_lprnet_vtrainable_v1.0/us_lp_characters.txt specs/
訓練模型
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments/lprnet$ tlt lprnet train -e /workspace/tlt-experiments/lprnet/tutorial_spec.txt -r /workspace/tlt-experiments/lprnet/ -k nvidia_tlt -m /workspace/tlt-experiments/lprnet/us_lprnet_baseline18_trainable.tlt --use_amp
匯出模型
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments/lprnet$ tlt lprnet export -m /workspace/tlt-experiments/lprnet/weights/lprnet_epoch-24.tlt -k nvidia_tlt -e /workspace/tlt-experiments/lprnet/tutorial_spec.txt
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments/lprnet$ ls weights/lprnet_epoch-24.etlt
評估
(tlt) ~/Data/DeepStream/deepstream_lpr_app/tlt-lpdr/tlt-experiments/lprnet$ tlt lprnet evaluate -m /workspace/tlt-experiments/lprnet/weights/lprnet_epoch-24.tlt -k nvidia_tlt -e /workspace/tlt-experiments/lprnet/tutorial_spec.txt

2021年6月8日 星期二

Ubuntu 18.04 的 launcher 或 sidebar 消失

發生時機,當保護螢幕啟動,由遠端登入後發現
工作列不見了

壓 Alt+F2, 會出現一個對話框, 輸入 r
$ gnome-shell -r
$ gnome-shell --replace
$ busctl --user call org.gnome.Shell /org/gnome/Shell org.gnome.Shell Eval s 'Meta.restart("Restarting…")'

Restart GNOME Shell
$ killall -HUP gnome-shell

2021年6月7日 星期一

Ubuntu 18.04 重灌

下載 iso 檔,利用 rufus 寫入 usb
搜尋工具/Disks 確認硬碟位置
備份資料含檔案屬性
$ sudo cp -a /path/source/. /path/dest

gitea 備份和還原
/var/lib/gitea
$ sudo cp /etc/gitea/app.ini .
$ sudo -u git gitea dump -c /etc/gitea/app.ini
$ unzip gitea-dump-xxxx.zip
$ cd gitea-dump-xxxx
$ mv data/conf/app.ini /etc/gitea/conf/app.ini
$ mv data/* /var/lib/gitea/data/
$ mv log/* /var/lib/gitea/log/
$ mv repos/* /var/lib/gitea/repositories/
$ chown -R gitea:gitea /etc/gitea/conf/app.ini /var/lib/gitea
$ mysql --default-character-set=utf8mb4 -u$USER -p$PASS $DATABASE <gitea-db.sql
$ service gitea restart

mariadb backup and restore
/var/lib/mysql
$ sudo mysql
> SHOW DATABASES;
$ sudo mysqldump --all-databases > all.sql
$ sudo mysql --one-database db_name < all.sql

$ sudo cp -a /opt/tomcat /backup/opt/tomcat
$ sudo cp -a /opt/nvidia /backup/opt/nvidia
$ sudo cp -a /etc/nginx /backup/etc/nginx
$ sudo cp -a /etc/systemd /backup/etc/systemd
$ sudo cp -a /etc/udev/rules.d /backup/udev

$ sudo vi /etc/default/grub
$ sudo update-grub

nvidia driver 更新
$ sudo apt-get install gcc
$ sudo apt-get install make
下載 https://www.nvidia.com/download/driverResults.aspx/168347/en-us 驅動
$ chmod 755 NVIDIA-Linux-x86_64-460.32.03.run
$ sudo ./ NVIDIA-Linux-x86_64-460.32.03.run
但會說 nvidia-drm 已經啟動
所以要卸載 舊驅動
$ sudo apt-get --purge remove "*cublas*" "*cufft*" "*curand*" \
 "*cusolver*" "*cusparse*" "*npp*" "*nvjpeg*" "cuda*" "nsight*"
$ sudo apt-get purge 'nvidia*'
$ sudo apt-get autoremove
$ sudo reboot
$ sudo ./ NVIDIA-Linux-x86_64-460.32.03.run
使用 --no-opengl-files 參數,以免使用DeepStream時,只開啟一下黑幕,出現下列錯誤
cuGraphicsGLRegisterBuffer failed with error(304) gst_eglglessink_cuda_init texture = 1
$ sudo ./ NVIDIA-Linux-x86_64-460.32.03.run --no-opengl-files --dkms --no-drm
因為 ubuntu 還是用了一個驅動,所以不能直接更新驅動
但可以設定停用此驅動,小心回答問題,在執行一遍
Would you like ton register the kernel module sources with DKMS? 回答 Yes
Install NVIDIA’s 32-bit compatibility libraries? 回答 No
$ nvidia-smi

安裝 CUDA, 不要使用 deb(network), 好像在版本安裝上會出錯
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
$ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
$ wget https://developer.download.nvidia.com/compute/cuda/11.1.1/local_installers/cuda-repo-ubuntu1804-11-1-local_11.1.1-455.32.00-1_amd64.deb
$ sudo dpkg -i cuda-repo-ubuntu1804-11-1-local_11.1.1-455.32.00-1_amd64.deb
$ sudo apt-key add /var/cuda-repo-ubuntu1804-11-1-local/7fa2af80.pub
$ sudo apt-get update
$ sudo apt-get -y install cuda
$ sudo apt-get -y install cuda-11-1
指定版本很重要,不然會裝最新的
$ nvidia-smi
Failed to initialize NVML: Driver/library version mismatch
重新開機即可
但顯示出來的 Driver 和 CUDA Version 都會改變

TensorRT 安裝
https://developer.nvidia.com/nvidia-tensorrt-download
由此進入選擇所需版本,並選擇 deb 版
由此選擇開啟文件
進入 TensorRT Installation Guide
https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-723/install-guide/index.html
跳至 4.1. Debian Installation
跟著步驟安裝

設定環境變數
PATH
LD_LIBRARY_PATH
pycuda 只能搭配 python 3.7
$ sudo apt-get install python3.7
$ sudo apt-get install python3.7-dev
$ python3.7 -m pip install 'pycuda>=2019.1.1'

Install Nvidia Docker
$ curl https://get.docker.com | sh && sudo systemctl --now enable docker
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
$ curl -s -L https://nvidia.github.io/nvidia-container-runtime/experimental/$distribution/nvidia-container-runtime.list | sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list
$ sudo apt-get update
$ sudo apt-get install -y nvidia-docker2
$ sudo systemctl restart docker
$ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
$ sudo groupadd docker
$ sudo usermod -a -G docker $USER
$ sudo reboot

Install TensorRT 7.2 OSS
參考 https://github.com/NVIDIA/TensorRT/tree/master
Install TensorRT 7.2 OSS
$ git clone -b master https://github.com/nvidia/TensorRT TensorRT_OSS-7.2.3.4
$ cd TensorRT_OSS-7.2.3.4/
$ git submodule update --init --recursive
$ cd /pathto/TensorRT-7.2.2.3/
$ export TRT_LIBPATH=`pwd`
$ cd /pathto/TensorRT_OSS-7.2.3.4/
$ ./docker/build.sh --file docker/ubuntu-18.04.Dockerfile --tag tensorrt-ubuntu-1804 --cuda 11.1
$ ./docker/launch.sh --tag tensorrt-ubuntu-1804 --gpus all
trtuser@c2936e108d43:/workspace$ cd $TRT_OSSPATH
trtuser@c2936e108d43:/workspace/TensorRT$ mkdir -p build && cd build
trtuser@c2936e108d43:/workspace/TensorRT/build$ cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
trtuser@c2936e108d43:/workspace/TensorRT/build$ make -j$(nproc)
trtuser@c2936e108d43:/workspace/TensorRT/build$ exit
$ cd ..
$ mkdir backup
$ mv TensorRT-7.2.2.3/targets/x86_64-linux-gnu/lib/libnvinfer_plugin.so.7.2.2 backup
$ cp TensorRT_OSS-7.2.3.4/build/out/libnvinfer_plugin.so.7.2.3 TensorRT-7.2.2.3/targets/x86_64-linux-gnu/lib/

install xpra
https://github.com/Xpra-org/xpra/blob/master/docs/Build/Debian.md
$ git clone https://github.com/Xpra-org/xpra.git
$ sudo ./setup.py install
Exception: ERROR: cannot find a valid pkg-config entry for nvjpeg-11.4 using PKG_CONFIG_PATH=(empty)
$ vi setup.py
/if nvjpeg_ENABLED:
在底下不遠處有兩處 for v in ("11.4", "11.3"...):
皆改成 for v in ("11.1"): 即可
另外 ld: connot find -lcuda
$ sudo ln -s /usr/local/cuda-11.1/lib64/stubs/libcuda.so /usr/lib

pip 更新出錯
$ pip install --upgrade --no-cache-dir pip
$ python3 -m pip install --upgrade --no-cache-dir pip -i https://pypi.python.org/simple

python 之 pip install --upgrade pip 出錯

Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 290, in run
    with self._build_session(options) as session:
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 69, in _build_session
    if options.cache_dir else None
  File "/usr/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not int

pip 更新出錯
$ pip install --upgrade --no-cache-dir pip
$ python3 -m pip install --upgrade --no-cache-dir pip -i https://pypi.python.org/simple

2021年6月3日 星期四

在 windows 10 利用 frp 執行 vnc

編輯 frps.ini
[common]
bind_port = 7000
token = password

編輯 frps.vbx
set ws=Wscript.CreateObject("Wscript.Shell")
ws.Run "C:\frp_0.31.2_windows_amd64\frps.exe -c C:\frp_0.31.2_windows_amd64\frps.ini", 0

gpedit.msc
本機電腦 原則/電腦設定/指令碼 - (啟動/關機)
啟動/內容
新增 ftps.vbs


編輯 frpc.ini
[common]
server_addr = remote.server.ip
server_port = 7000
token = password

[vnc]
type = tcp
local_ip = 127.0.0.1
local_port = 5800
remote_port = 5900
use_encryption = true
use_compression = true

編輯 frpc.vbx
set ws=Wscript.CreateObject("Wscript.Shell")
ws.Run "C:\frp_0.31.2_windows_amd64\frpc.exe -c C:\frp_0.31.2_windows_amd64\frpc.ini", 0

gpedit.msc
本機電腦 原則/電腦設定/指令碼 - (啟動/關機)
啟動/內容
新增 ftpc.vbs

2021年6月1日 星期二

Tesseract 在 Docker 上訓練 (二)

$ docker start t4cmp
$ docker exec -it t4cmp bash
# cd /home/workspace/tesseract
# TESSDATA_PREFIX=/root/tesstutorial/tesseract/tessdata
# vi src/training/tesstrain_util.sh
尋找 phase_E_extract_features()
在 run_command tesseract ${img_file} ${img_file%.*} 後面加入 --psm 7
在 ${box_config} ${config} & 去除 &
移除  jobs="$jobs $!" 和 wait $jobs
# vi src/training/language-specific.sh
加入 你要增加的語言

src/training/tesstrain.sh --fonts_dir /usr/share/fonts --lang plate \
--linedata_only --my_boxtiff_dir /home/tmp --noextract_font_properties \
--langdata_dir ../langdata --tessdata_dir ~/tesstutorial/tesseract/tessdata  \
--output_dir ~/tesstutorial/platetrain

lstmtraining --model_output ~/tesstutorial/impact_from_full/impact \
--continue_from ~/tesstutorial/impact_from_full/eng.lstm \
--traineddata ~/tesstutorial/tesseract/tessdata/eng.traineddata \
--train_listfile ~/tesstutorial/platetrain/plate.training_files.txt \
--max_iterations 400 

lstmtraining --stop_training \
--continue_from ~/tesstutorial/impact_from_full/impact_0.031000_2_400.checkpoint \
--traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \
--model_output ~/tesstutorial/impact_from_full/plate.traineddata

刪除無法訓練的圖
for i in `find /tmp/plate-2021-04-13.u4C -name "*.tif"`
do
LSTMF=${i//.tif/.lstmf}
if [ ! -f "$LSTMF" ]; then
  LSTMF=`basename "$LSTMF"`
  TIF=${LSTMF//.lstmf/.tif}
  BOX=${LSTMF//.lstmf/.box}
  rm /home/tmp/$TIF /home/tmp/$BOX
fi
done

2021年3月26日 星期五

Tesseract 在 Docker 上訓練

Tessseract 文件 中 Compiling and Installation 有敘述如何使用 Docker

安裝 Docker
 $ unzip tesseract-ocr-compilation-master.zip
 $ cd tesseract-ocr-compilation-master

為了方便除錯,建立共通的目錄
 $ vi scripts/3-run-new-container.sh
docker run -d -v debug_path:/home/tmp -p 4022:22 --name t4cmp tesseractshadow/tesseract4cmp
docker ps

執行步驟3 建立 t4cmp container 之後
在 Docker 中安裝 vi, 需在
$ docker exec -it t4cmp bash
# apt-get update
# apt-get install vim

使用 ssh login
# echo 'root:your_passwd' | chpasswd
# sed -i 's/#PermitRootLogin/PermitRootLogin/' /etc/ssh/sshd_config
# exit
$ docker stop t4cmp
$ docker start t4cmp
$ ssh root@localhost -p 4022

1. $ ./scripts/1-pull-container.sh 下載 docker image tesseractshadow/tesseract4cmp
2. $ ./scripts/2-remove-container.sh 移除 t4cmp container
3. $ ./scripts/3-run-new-container.sh 建立 t4cmp container
4. $ ./cripts/4-update-src.sh 更新 Leptionica 和 Tesseract Source
5. $ ./scripts/5-compile-src.sh 編譯 Leptionica 和 Tesseract 
6. $ ./scripts/6-test-ocr.sh 測試,目錄於 ocr-files
7. $ ./scripts/7-build-pkg.sh 建立安裝檔於 pkg 目錄

Tessseract 文件 中 Training for Tesseract 4 有敘述如何 訓練自己的字
$ docker exec -it t4cmp bash
# cd /home/workspace
# apt update
# apt install ttf-mscorefonts-installer
會停在下列
[More] 
Progress: [ 54%] [###############################...........................] 
按 Enter 直到出現如下
Do you accept the EULA license terms? [yes/no] 
輸入 yes
# apt install fonts-dejavu
# fc-cache -vf
# cd
# mkdir ~/tesstutorial
# cd ~/tesstutorial
# mkdir langdata
# cd langdata
# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/radical-stroke.txt
# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/common.punc
# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/font_properties
# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/Latin.unicharset
# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/Latin.xheights
# mkdir eng
# cd eng
# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.training_text
# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.punc
# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.numbers
# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.wordlist
# cd ~/tesstutorial
# git clone --depth 1 https://github.com/tesseract-ocr/tesseract.git
# cd tesseract/tessdata
# wget https://github.com/tesseract-ocr/tessdata/raw/master/eng.traineddata
# mkdir best
# cd best
# wget https://github.com/tesseract-ocr/tessdata_best/raw/master/eng.traineddata
# wget https://github.com/tesseract-ocr/tessdata_best/raw/master/chi_tra.traineddata

# cd ~/tesstutorial/tesseract/
# src/training/tesstrain.sh --fonts_dir /usr/share/fonts --lang eng --linedata_only \
  --noextract_font_properties --langdata_dir ../langdata \
  --tessdata_dir ./tessdata --output_dir ~/tesstutorial/engtrain
出現下列錯誤
ERROR: /tmp/eng-2021-03-26.coV/eng.Tex_Gyre_Bonum_Bold.exp0.box does not exist or is not readable
# vi src/training/language-specific.sh 
刪除 Tex Gyre* 字型
重新執行一遍,出現下列兩行,表示成功
Created starter traineddata for LSTM training of language 'eng'
Run 'lstmtraining' command to continue LSTM training for language 'eng'

建立另外一種字型,做為測試用
#src/training/tesstrain.sh --fonts_dir /usr/share/fonts --lang eng --linedata_only \
  --noextract_font_properties --langdata_dir ../langdata \
  --tessdata_dir ./tessdata \
  --fontlist "Impact Condensed" --output_dir ~/tesstutorial/engeval

從頭開始訓練
# mkdir -p ~/tesstutorial/engoutput
# /usr/local/bin/lstmtraining --debug_interval 0 \
  --traineddata ~/tesstutorial/engtrain/eng/eng.traineddata \
  --net_spec '[1,36,0,1 Ct3,3,16 Mp3,3 Lfys48 Lfx96 Lrx96 Lfx256 O1c111]' \
  --model_output ~/tesstutorial/engoutput/base --learning_rate 20e-4 \
  --train_listfile ~/tesstutorial/engtrain/eng.training_files.txt \
  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt \
  --max_iterations 5000 &>~/tesstutorial/engoutput/basetrain.log

因為在 docker 內,--debug_interval 為 0,要看詳細的訓練過程 --debug_interval 為 100
# make training
# make training-install
# make ScrollView.jar

O1c111 表示有111個字,{eng}.unicharset 檔內有 111行

# /usr/local/bin/lstmeval --model ~/tesstutorial/engoutput/base_checkpoint \
  --traineddata ~/tesstutorial/engtrain/eng/eng.traineddata \
  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt
自己從頭錯訓練,誤率太高
At iteration 0, stage 0, Eval Char error rate=107.795608, Word error rate=97.578246

# /usr/local/bin/lstmeval --model ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \
  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt
用別人訓練的,好太多
At iteration 0, stage 0, Eval Char error rate=3.095477, Word error rate=9.465216

從最佳的 model 導出可以訓練的 lstm model
# mkdir -p ~/tesstutorial/impact_from_full
# /usr/local/bin/combine_tessdata -e ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \
  ~/tesstutorial/impact_from_full/eng.lstm

微調訓練
# /usr/local/bin/lstmtraining --model_output ~/tesstutorial/impact_from_full/impact \
  --continue_from ~/tesstutorial/impact_from_full/eng.lstm \
  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \
  --train_listfile ~/tesstutorial/engeval/eng.training_files.txt \
  --max_iterations 400
#/usr/local/bin/lstmeval --model ~/tesstutorial/impact_from_full/impact_checkpoint \
  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \
  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt
測試的結果很好
At iteration 0, stage 0, Eval Char error rate=0.000000, Word error rate=0.000000

導出可用的 model
# /usr/local/bin/lstmtraining --stop_training \
  --continue_from ~/tesstutorial/impact_from_full/impact_0.271000_43_400.checkpoint \
  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \
  --model_output ~/tesstutorial/impact_from_full/eng.traineddata
# cp ~/tesstutorial/impact_from_full/eng.traineddata /usr/local/share/tessdata/eng_a.traineddata
# tesseract phototest.tif phototest -l eng_a -psm 1 --oem 1

準備自己的 tif 和 box, 兩者除附檔名外,檔名一樣且以 lang. 開頭 .exp0. 結尾
如 eng.AAA.exp0.tif, eng.AAA.exp0.box, 置於 /home/tmp
修改 tesstrain.sh, 去掉 phase_I_generate_image 8
# src/training/tesstrain_a.sh --fonts_dir /usr/share/fonts --lang eng --linedata_only \
  --my_boxtiff_dir /home/tmp \
  --noextract_font_properties --langdata_dir ../langdata \
  --tessdata_dir ./tessdata --output_dir ~/tesstutorial/platetrain

# /usr/local/bin/lstmtraining --model_output ~/tesstutorial/impact_from_full/impact \
  --continue_from ~/tesstutorial/impact_from_full/eng.lstm \
  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \
  --train_listfile ~/tesstutorial/platetrain/eng.training_files.txt \
  --max_iterations 400

2021年3月11日 星期四

pyinstaller

參考 使用說明

(tensorflow) D:\PlateOcr> pyinstaller --add-binary d:\your_path\opencv_ffmpeg340_64.dll;opencv_ffmpeg340_64.dll --hidden-import opencv-python --hidden-import cv2 --hidden-import another --paths D:\your_path_to_opencv_lib:D:\your_path_to_cv2.xxx.pyd -F PlateOcrEval.py

最終測試成功命令
(tensorflow) D:\PlateOcr> pyinstaller --hidden-import cv2 --paths D:\your_path_to_opencv_lib:D:\your_path_to_cv2.xxx.pyd -F PlateOcrEval.py

2021年2月25日 星期四

解決 跨網站 webapi 出現 blocked by CORS policy

因為跨網站使用 webapi 出現
Access to XMLHttpRequest at 'https://webapi.url' from origin 'https://web.url' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

因為使用 nginx 轉到 tomcat
不用修改 /etc/nginx/sites-available/config
要修改 tomcat 的 WEB-INF/web.xml 在 <web-app> 內加上如下內容
        <filter>
                <filter-name>CorsFilter</filter-name>
                <filter-class>org.apache.catalina.filters.CorsFilter</filter-class>
                <init-param>
                        <param-name>cors.allowed.origins</param-name>
                        <param-value>https://www.yoursite:8443,http://yoursite2.com</param-value>
                </init-param>
                <init-param>
                        <param-name>cors.allowed.methods</param-name>
                        <param-value>GET,POST,HEAD,OPTIONS,PUT</param-value>
                </init-param>
                <init-param>
                        <param-name>cors.allowed.headers</param-name>
                        <param-value>Content-Type,X-Requested-With,accept,Origin,Access-Control-Request-Method,Access-Control-Request-Headers</param-value>
                </init-param>
                <init-param>
                        <param-name>cors.exposed.headers</param-name>
                        <param-value>Access-Control-Allow-Origin,Access-Control-Allow-Credentials</param-value>
                </init-param>
                <init-param>
                        <param-name>cors.support.credentials</param-name>
                        <param-value>true</param-value>
                </init-param>
                <init-param>
                        <param-name>cors.preflight.maxage</param-name>
                        <param-value>10</param-value>
                </init-param>
        </filter>
        <filter-mapping>
                <filter-name>CorsFilter</filter-name>
                <url-pattern>/*</url-pattern>
        </filter-mapping>


2021年1月29日 星期五

install pycuda

sudo pip3 install --global-option=build_ext --global-option="-I/usr/local/cuda-10.2/targets/aarch64-linux/include/" --global-option="-L/usr/local/cuda-10.2/targets/aarch64-linux/lib/" pycuda

https://docs.donkeycar.com/guide/robot_sbc/tensorrt_jetson_nano/

2021年1月21日 星期四

Install Tensorflow 1.15 on Ubuntu 18.04

原本 pip3 install tensorflow-gpu=1.15 即可
但發現與 cuda 10.2 不合, 需要安裝 cuda 10.0,搭配 cudnn v7.6.5

安裝 CUDA
https://developer.nvidia.com/cuda-downloads
選 Archive of Previous CUDA Releases
選 CUDA Toolkit 10.0, Linux, x86_64, Ubuntu, 18.04, deb(local)
按下 Download
sudo dpkg -i cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64.deb
sudo apt-key add /var/cuda-repo-10-0-local-10.0.130-410.48/7fa2af80.pub
sudo apt-get update
查詢可安裝版本
apt-cache policy cuda
apt-cache madison cuda
安裝正確版本
sudo apt-get install cuda=10.0.130-1

安裝 CUDNN
https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html
https://developer.nvidia.com/cudnn
選擇 cuDNN v7.6.5 for CUDA 10.0
選擇 cuDNN Library for Linux (x86)
tar -xzvf cudnn-10.0-linux-x64-v7.6.5.32.tgz
sudo cp cuda/include/cudnn*.h /usr/local/cuda-10.0/include
sudo cp cuda/lib64/libcudnn* /usr/local/cuda-10.0/lib64
sudo chmod a+r /usr/local/cuda-10.0/include/cudnn*.h /usr/local-10.0/cuda/lib64/libcudnn*

sudo apt install python3-testresources
sudo apt install python-dev python-pip
mkdir envs; cd envs
python3 -m venv --system-site-packages tensorflow-1.15
source tensorflow-1.15/bin/activate
pip install --upgrade pip
pip install tensorflow-gpu==1.15

2021年1月11日 星期一

Ubuntu 18.04 重灌

依據 https://developer.nvidia.com/deepstream-getting-started
選擇各個版本

https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html

sudo apt-get install ssh
sudo apt install python3-pip
sudo ln -s /usr/bin/python3 /usr/bin/python

安裝 NVIDIA Driver
https://www.linuxbabe.com/ubuntu/install-nvidia-driver-ubuntu-18-04
sudo lshw -c display
sudo lshw -c video
sudo ubuntu-drivers devices
sudo ubuntu-drivers autoinstall
sudo reboot
sudo lshw -c display

安裝 CUDA Toolkit
https://developer.nvidia.com/cuda-downloads
請選擇 CUDA Toolkit 10.2
Linux/x86_64/Ubuntu/18.04/deb(local)
vi ~/.bashrc
export PATH=/usr/local/cuda-10.2:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64:$LD_LIBRARY_PATH

安裝 CUDNN
https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html
https://developer.nvidia.com/cudnn
選擇 cuDNN v8.0.5 for CUDA 10.2
選擇 cuDNN Library for Linux (x86)
tar -xzvf cudnn-10.2-linux-x64-v8.0.5.39.tgz
sudo cp cuda/include/cudnn*.h /usr/local/cuda/include
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
sudo chmod a+r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn*
sudo ldconfig
/sbin/ldconfig.real: /usr/local/cuda-10.2/targets/x86_64-linux/liblibcudnn.so.8 is not a symbolic link
cd /usr/local/cuda-10.2/targets/x86_64-linux/lib
sudo rm liblibcudnn.so.8 liblibcudnn.so
sudo ln -s libcudnn.so.8.0.5 libcudnn.so.8
sudo ln -s libcudnn.so.8 libcudnn.so
sudo ldconfig

安裝 TensorRT
下載 TensorRT binary
https://developer.nvidia.com/nvidia-tensorrt-7x-download
選擇 TensorRT 7.1.3.4 for Ubuntu 18.04 and CUDA 10.2 TAR package
version="7.1.3.4"
os="Ubuntu-18.04"
arch=$(uname -m)
cuda="cuda-10.2"
cudnn="cudnn8.0"
tar -xvzf TensorRT-7.1.3.4.Ubuntu-18.04.x86_64-gnu.cuda-10.2.cudnn8.0.tar.gz
vi ~/.bashrc
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/Data/TensorRT/TensorRT-7.1.3.4/lib
source ~/.bashrc
cd TensorRt-7.1.3.4/python
sudo pip3 install tensorrt-7.1.3.4-cp36-none-linux_x86_64.whl
cd ../uff
sudo pip3 install uff-0.6.9-py2.py3-none-any.whl
cd ../graphsurgeon/
sudo pip3 install graphsurgeon-0.4.5-py2.py3-none-any.whl

安裝 GStreamer
https://yingrenn.blogspot.com/2020/09/gstreamer.html

安裝 DeepStream
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Quickstart.html#dgpu-setup-for-ubuntu
sudo apt install \
 libssl1.0.0 \
 libgstreamer1.0-0 \
 gstreamer1.0-tools \
 gstreamer1.0-plugins-good \
 gstreamer1.0-plugins-bad \
 gstreamer1.0-plugins-ugly \
 gstreamer1.0-libav \
 libgstrtspserver-1.0-0 \
 libgstrtspserver-1.0-dev \
 libjansson4
git clone https://github.com/edenhill/librdkafka.git
cd librdkafka
git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a
./configure
make
sudo make install
sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.0/lib
sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.0/lib
下載 DeepStream
https://developer.nvidia.com/assets/Deepstream/5.0/ga/secure/deepstream_sdk_5.0.1_x86_64.tbz2
sudo tar -xvf deepstream_sdk_v5.0.1_x86_64.tbz2 -C /
cd /opt/nvidia/deepstream/deepstream-5.0/
sudo ./install.sh
sudo ldconfig

安裝 CMake v3.13
wget http://www.cmake.org/files/v3.13/cmake-3.13.5.tar.gz
tar xpvf cmake-3.13.5.tar.gz cmake-3.13.5
cd cmake-3.13.5
sudo apt-get install zlib1g-dev
sudo apt-get install curl
sudo apt-get install libcurl3
sudo apt-get install libcurl4 libcurl4-openssl-dev
./bootstrap --system-curl
make -j$(nproc)
sudo make install

安裝 Docker
https://docs.docker.com/engine/install/ubuntu/
sudo apt-get update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

安裝 NVIDIA Docker
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker
curl https://get.docker.com | sh \
  && sudo systemctl start docker \
  && sudo systemctl enable docker
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker
sudo groupadd docker
sudo usermod -a -G docker $USER
sudo reboot

若 sudo apt update 出現
W: Target CNF (stable/cnf/Commands-all) is configured multiple times in /etc/apt/sources.list:52 and /etc/apt/sources.list.d/docker.list:1
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4773BD5E130D1D45
sudo rm /etc/apt/sources.list.d/docker.list

安裝 TensorRT 7.1 OSS
https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html
https://github.com/NVIDIA/TensorRT/tree/master
從 master 切換到 release/7.1
下載 TensorRT OSS
TensorRT OSS: 包含 TensorRT plugins, Caffe 和 ONNX parsers 等
git clone -b master https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
參考之前的 TensorRT binary
cd TensorRT-7.1.3.4
export TRT_RELEASE=`pwd`
cd $TRT_SOURCE
./docker/build.sh --file docker/ubuntu.Dockerfile --tag tensorrt-ubuntu --os 18.04 --cuda 10.2
./docker/launch.sh --tag tensorrt-ubuntu --gpus all --release $TRT_RELEASE --source $TRT_SOURCE
cd $TRT_SOURCE
mkdir -p build && cd build
cmake .. -DTRT_LIB_DIR=$TRT_RELEASE/lib -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)
exit
mkdir backup
mv $TRT_RELEASE/targets/x86_64-linux-gnu/lib/libnvinfer_plugin.so.7.1.3 backup
cp $TRT_SOURCE/build/out/libnvinfer_plugin.so.7.2.1 $TRT_RELEASE/targets/x86_64-linux-gnu/lib/libnvinfer_plugin.so.7.1.3

安裝 OpenCV 4.4
git clone https://github.com/opencv/opencv.git opencv-4.4.0 -b 4.4.0 --depth 1
git clone https://github.com/opencv/opencv_contrib.git opencv_contrib-4.4.0 -b 4.4.0 --depth 1
sudo apt-get update
sudo add-apt-repository ppa:alex-p/tesseract-ocr
sudo apt install tesseract-ocr libtesseract-dev
sudo apt-get install libleptonica-dev
sudo apt-get install qt5-default
sudo apt-get install qtcreator
sudo apt-get install build-essential cmake unzip pkg-config
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libgtk-3-dev
sudo apt-get install libatlas-base-dev gfortran
sudo apt-get install libhdf5-dev
sudo apt-get install python3-dev
mkdir build; cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D WITH_TBB=ON \
-D BUILD_opencv_cudacodec=OFF \
-D ENABLE_FAST_MATH=1 \
-D CUDA_FAST_MATH=1 \
-D WITH_CUDA=ON \
-D WITH_CUBLAS=ON \
-D WITH_LIBV4L=ON \
-D BUILD_opencv_python3=ON \
-D BUILD_opencv_python2=OFF \
-D BUILD_opencv_java=OFF \
-D WITH_V4L=ON \
-D WITH_QT=ON \
-D WITH_OPENGL=ON \
-D WITH_GSTREAMER=ON \
-D WITH_GTK=ON \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D OPENCV_PC_FILE_NAME=opencv.pc \
-D OPENCV_ENABLE_NONFREE=OFF \
-D OPENCV_EXTRA_MODULES_PATH=/your_path_to/opencv/opencv_contrib-4.4.0/modules \
-D BUILD_EXAMPLES=ON \
-D WITH_CUDNN=ON \
-D CUDNN_VERSION="8.0.5" \
-D OPENCV_DNN_CUDA=ON \
-D CUDNN_INCLUDE_DIR=/usr/local/cuda/include \
-D CUDNN_LIBRARY=/usr/local/cuda/lib64/libcudnn.so.8.0.5 \
-D CUDA_ARCH_BIN=7.5 \
../opencv-4.4.0
#-D OPENCV_ENABLE_NONFREE=ON \
#-D OpenGL_GL_PREFERENCE=LEGACY \

make -j$(nproc)
sudo make install
sudo ldconfig
opencv_version -v

下載 Tesseract 的中文字 chi_tra.traineddata, chi_tra_vert.traineddate
https://github.com/tesseract-ocr/tessdata
放至 /usr/share/tesseract-ocr/4.00/tessdata

安裝 Xpra
https://www.xpra.org/trac/wiki/Building/Debian
sudo apt-get install libx11-dev libxtst-dev libxcomposite-dev libxdamage-dev \
 libxkbfile-dev python-all-dev
sudo apt-get install libgtk-3-dev python3-dev python3-cairo-dev python-gi-dev cython3
sudo apt-get install xauth x11-xkb-utils
sudo apt-get install libx264-dev libvpx-dev yasm
sudo apt-get install libnvidia-encode-440
sudo apt-get install libavformat-dev libavcodec-dev libswscale-dev
sudo apt-get install libturbojpeg-dev
sudo apt-get install libwebp-dev
sudo apt-get install uglifyjs brotli libjs-jquery libjs-jquery-ui gnome-backgrounds
sudo apt-get install python3-opengl python3-numpy python3-pil
sudo apt-get install python3-rencode python3-lz4 python3-dbus python3-cryptography \
 python3-netifaces python3-yaml
sudo apt-get install python3-setproctitle python3-xdg python3-pyinotify python3-opencv
sudo apt-get install libpam-dev quilt xserver-xorg-dev xutils-dev xvfb keyboard-configuration
sudo apt-get install python3-kerberos python3-gssapi
sudo apt-get install gstreamer1.0-pulseaudio gstreamer1.0-alsa \
 gstreamer1.0-plugins-base gstreamer1.0-plugins-good \
 gstreamer1.0-plugins-ugly
sudo apt-get install cups-filters cups-common cups-pdf python3-cups
sudo apt-get install openssh-client sshpass python3-paramiko
sudo apt-get install devscripts build-essential lintian debhelper
wget https://www.xpra.org/src/xpra-4.0.4.tar.xz
tar -xf xpra-4.0.6.tar.xz
cd xpra-4.0.6
vi setup.py
#!/usr/bin/env python3
sudo ./setup.py install

安裝 frp
https://yingrenn.blogspot.com/2020/03/frp.html
vi frps.ini
[common]
bind_port = 7000
dashboard_port = 7001
dashboard_user = user
dashboard_pwd = password

vi /etc/systemd/system/frps.service
[Unit]
Description=FRP Server Daemon

[Service]
Type=simple
ExecStartPre=-/usr/sbin/setcap cap_net_bind_service=+ep /home/mark/Data/frp/frp_0.34.3_linux_amd64/frps
ExecStart=/path_to_frp/frp_0.34.3_linux_amd64/frps -c /path_to_frp/frp_0.34.3_linux_amd64/frps.ini
Restart=always
RestartSec=20s
User=nobody
PermissionsStartOnly=true
LimitNOFILE=infinity

[Install]
WantedBy=multi-user.target

sudo systemctl start frps
sudo systemctl enable ftps

安裝 Gitea Tomcat Nginx
https://yingrenn.blogspot.com/2019/11/gitea-in-ubuntu.html
https://yingrenn.blogspot.com/search/label/tomcat

安裝 x11vnc
因為 Ubuntu 18.04 改 LightDM 到 GDM3, 所以要在 console login 後,才能動作
sudo apt install x11vnc
sudo x11vnc -storepasswd
sudo chown mark.mark ~/.vnc/passwd
sudo vi /etc/systemd/system/x11vnc.service
# Description: Custom Service Unit file
# File: /etc/systemd/system/x11vnc.service
[Unit]
Description="x11vnc"
Requires=display-manager.service
After=display-manager.service

[Service]
ExecStart=/usr/bin/x11vnc -loop -nopw -xkb -repeat -noxrecord -noxfixes -noxdamage -forever -rfbport 5900 -display :1 -auth /run/user/1000/gdm/Xauthority -rfbauth /home/mark/.vnc/passwd
ExecStop=/usr/bin/killall x11vnc
Restart=on-failure
RestartSec=2

[Install]
WantedBy=multi-user.target

安裝 VirtualBox
https://www.virtualbox.org/wiki/Downloads
到官網下載
virtualbox-6.1_6.1.16-140961~Ubuntu~bionic_amd64.deb
Oracle_VM_VirtualBox_Extension_Pack-6.1.16.vbox-extpack
sudo dpkg -i virtualbox-6.1_6.1.16-140961~Ubuntu~bionic_amd64.deb
sudo groupadd win10disk
sudo usermod -a -G win10disk youruser
sudo udevadm info /dev/sdX | grep UUID
E: ID_PART_TABLE_UUID=01234567-89ab-cdef-0123-456789abcde
vi /etc/udev/rules.d/99-win10disk.rules
ENV{ID_PART_TABLE_UUID}=="1234567-89ab-cdef-0123-456789abcde", GROUP="win10disk"
ls -l /dev/sdb
brw-rw---- 1 root win10disk 8, 16 Nov 4 23:33 /dev/sdb
VBoxManage internalcommands createrawvmdk -filename .VirtualBox/Crucial1T.vmdk -rawdisk /dev/sda
sudo usermod -a -G vboxusers youruser
拷貝舊的 VM, 修改 win10.vbox, win10.vbox-prev 內的 uuid
與 .VirtualBox/Crucial1T.vmdk 的 uuid 一致
開啟 VirtualBox
File/Preferences...
Extensions 按 +
選擇剛下載的 Oracle_VM_VirtualBox_Extension_Pack-6.1.16.vbox-extpack


只有登入畫面無法使用滑鼠和鍵盤,其餘正常
sudo apt-get instll xserver-xorg-input-all