Ubuntu 18.04 的 launcher 或 sidebar 消失

發生時機，當保護螢幕啟動，由遠端登入後發現
工作列不見了

壓 Alt+F2, 會出現一個對話框, 輸入 r

$ gnome-shell -r

$ gnome-shell --replace

$ busctl --user call org.gnome.Shell /org/gnome/Shell org.gnome.Shell Eval s 'Meta.restart("Restarting…")'

Restart GNOME Shell

$ killall -HUP gnome-shell

Ubuntu 18.04 重灌

下載 iso 檔，利用 rufus 寫入 usb
搜尋工具/Disks 確認硬碟位置

備份資料含檔案屬性

$ sudo cp -a /path/source/. /path/dest

gitea 備份和還原

/var/lib/gitea

$ sudo cp /etc/gitea/app.ini .

$ sudo -u git gitea dump -c /etc/gitea/app.ini

$ unzip gitea-dump-xxxx.zip

$ cd gitea-dump-xxxx

$ mv data/conf/app.ini /etc/gitea/conf/app.ini

$ mv data/* /var/lib/gitea/data/

$ mv log/* /var/lib/gitea/log/

$ mv repos/* /var/lib/gitea/repositories/

$ chown -R gitea:gitea /etc/gitea/conf/app.ini /var/lib/gitea

$ mysql --default-character-set=utf8mb4 -u$USER -p$PASS $DATABASE <gitea-db.sql

$ service gitea restart

mariadb backup and restore

/var/lib/mysql

$ sudo mysql

> SHOW DATABASES;

$ sudo mysqldump --all-databases > all.sql

$ sudo mysql --one-database db_name < all.sql

$ sudo cp -a /opt/tomcat /backup/opt/tomcat

$ sudo cp -a /opt/nvidia /backup/opt/nvidia

$ sudo cp -a /etc/nginx /backup/etc/nginx

$ sudo cp -a /etc/systemd /backup/etc/systemd

$ sudo cp -a /etc/udev/rules.d /backup/udev

$ sudo vi /etc/default/grub

$ sudo update-grub

nvidia driver 更新

$ sudo apt-get install gcc

$ sudo apt-get install make

下載 https://www.nvidia.com/download/driverResults.aspx/168347/en-us 驅動

$ chmod 755 NVIDIA-Linux-x86_64-460.32.03.run

$ sudo ./ NVIDIA-Linux-x86_64-460.32.03.run

但會說 nvidia-drm 已經啟動

所以要卸載 舊驅動

$ sudo apt-get --purge remove "*cublas*" "*cufft*" "*curand*" \

 "*cusolver*" "*cusparse*" "*npp*" "*nvjpeg*" "cuda*" "nsight*"

$ sudo apt-get purge 'nvidia*'

$ sudo apt-get autoremove

$ sudo reboot

$ sudo ./ NVIDIA-Linux-x86_64-460.32.03.run

使用 --no-opengl-files 參數，以免使用DeepStream時，只開啟一下黑幕，出現下列錯誤

cuGraphicsGLRegisterBuffer failed with error(304) gst_eglglessink_cuda_init texture = 1

$ sudo ./ NVIDIA-Linux-x86_64-460.32.03.run --no-opengl-files --dkms --no-drm

因為 ubuntu 還是用了一個驅動，所以不能直接更新驅動

但可以設定停用此驅動，小心回答問題，在執行一遍

Would you like ton register the kernel module sources with DKMS? 回答 Yes

Install NVIDIA’s 32-bit compatibility libraries? 回答 No

$ nvidia-smi

安裝 CUDA, 不要使用 deb(network), 好像在版本安裝上會出錯

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin

$ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600

$ wget https://developer.download.nvidia.com/compute/cuda/11.1.1/local_installers/cuda-repo-ubuntu1804-11-1-local_11.1.1-455.32.00-1_amd64.deb

$ sudo dpkg -i cuda-repo-ubuntu1804-11-1-local_11.1.1-455.32.00-1_amd64.deb

$ sudo apt-key add /var/cuda-repo-ubuntu1804-11-1-local/7fa2af80.pub

$ sudo apt-get update

$ sudo apt-get -y install cuda

$ sudo apt-get -y install cuda-11-1

指定版本很重要，不然會裝最新的

$ nvidia-smi

Failed to initialize NVML: Driver/library version mismatch

重新開機即可

但顯示出來的 Driver 和 CUDA Version 都會改變

TensorRT 安裝

https://developer.nvidia.com/nvidia-tensorrt-download

由此進入選擇所需版本，並選擇 deb 版

由此選擇開啟文件

進入 TensorRT Installation Guide

https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-723/install-guide/index.html

跳至 4.1. Debian Installation

跟著步驟安裝

設定環境變數

PATH

LD_LIBRARY_PATH

pycuda 只能搭配 python 3.7

$ sudo apt-get install python3.7

$ sudo apt-get install python3.7-dev

$ python3.7 -m pip install 'pycuda>=2019.1.1'

Install Nvidia Docker

$ curl https://get.docker.com | sh && sudo systemctl --now enable docker

$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \

   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \

   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

$ curl -s -L https://nvidia.github.io/nvidia-container-runtime/experimental/$distribution/nvidia-container-runtime.list | sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list

$ sudo apt-get update

$ sudo apt-get install -y nvidia-docker2

$ sudo systemctl restart docker

$ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

$ sudo groupadd docker

$ sudo usermod -a -G docker $USER

$ sudo reboot

Install TensorRT 7.2 OSS

參考 https://github.com/NVIDIA/TensorRT/tree/master

Install TensorRT 7.2 OSS

$ git clone -b master https://github.com/nvidia/TensorRT TensorRT_OSS-7.2.3.4

$ cd TensorRT_OSS-7.2.3.4/

$ git submodule update --init --recursive

$ cd /pathto/TensorRT-7.2.2.3/

$ export TRT_LIBPATH=`pwd`

$ cd /pathto/TensorRT_OSS-7.2.3.4/

$ ./docker/build.sh --file docker/ubuntu-18.04.Dockerfile --tag tensorrt-ubuntu-1804 --cuda 11.1

$ ./docker/launch.sh --tag tensorrt-ubuntu-1804 --gpus all

trtuser@c2936e108d43:/workspace$ cd $TRT_OSSPATH

trtuser@c2936e108d43:/workspace/TensorRT$ mkdir -p build && cd build

trtuser@c2936e108d43:/workspace/TensorRT/build$ cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out

trtuser@c2936e108d43:/workspace/TensorRT/build$ make -j$(nproc)

trtuser@c2936e108d43:/workspace/TensorRT/build$ exit

$ cd ..

$ mkdir backup

$ mv TensorRT-7.2.2.3/targets/x86_64-linux-gnu/lib/libnvinfer_plugin.so.7.2.2 backup

$ cp TensorRT_OSS-7.2.3.4/build/out/libnvinfer_plugin.so.7.2.3 TensorRT-7.2.2.3/targets/x86_64-linux-gnu/lib/

install xpra

https://github.com/Xpra-org/xpra/blob/master/docs/Build/Debian.md

$ git clone https://github.com/Xpra-org/xpra.git

$ sudo ./setup.py install

Exception: ERROR: cannot find a valid pkg-config entry for nvjpeg-11.4 using PKG_CONFIG_PATH=(empty)

$ vi setup.py

/if nvjpeg_ENABLED:

在底下不遠處有兩處 for v in ("11.4", "11.3"...):

皆改成 for v in ("11.1"): 即可

另外 ld: connot find -lcuda

$ sudo ln -s /usr/local/cuda-11.1/lib64/stubs/libcuda.so /usr/lib

pip 更新出錯

$ pip install --upgrade --no-cache-dir pip

$ python3 -m pip install --upgrade --no-cache-dir pip -i https://pypi.python.org/simple

python 之 pip install --upgrade pip 出錯

Exception:

Traceback (most recent call last):

  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main

    status = self.run(options, args)

  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 290, in run

    with self._build_session(options) as session:

  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 69, in _build_session

    if options.cache_dir else None

  File "/usr/lib/python3.6/posixpath.py", line 80, in join

    a = os.fspath(a)

TypeError: expected str, bytes or os.PathLike object, not int

pip 更新出錯
$ pip install --upgrade --no-cache-dir pip
$ python3 -m pip install --upgrade --no-cache-dir pip -i https://pypi.python.org/simple

在 windows 10 利用 frp 執行 vnc

編輯 frps.ini

[common]

bind_port = 7000

token = password

編輯 frps.vbx

set ws=Wscript.CreateObject("Wscript.Shell")

ws.Run "C:\frp_0.31.2_windows_amd64\frps.exe -c C:\frp_0.31.2_windows_amd64\frps.ini", 0

gpedit.msc

本機電腦 原則/電腦設定/指令碼 - (啟動/關機)

啟動/內容

新增 ftps.vbs

編輯 frpc.ini

[common]

server_addr = remote.server.ip

server_port = 7000

token = password

[vnc]

type = tcp

local_ip = 127.0.0.1

local_port = 5800

remote_port = 5900

use_encryption = true

use_compression = true

編輯 frpc.vbx

set ws=Wscript.CreateObject("Wscript.Shell")

ws.Run "C:\frp_0.31.2_windows_amd64\frpc.exe -c C:\frp_0.31.2_windows_amd64\frpc.ini", 0

gpedit.msc

本機電腦 原則/電腦設定/指令碼 - (啟動/關機)

啟動/內容

新增 ftpc.vbs

Tesseract 在 Docker 上訓練 (二)

$ docker start t4cmp

$ docker exec -it t4cmp bash

# cd /home/workspace/tesseract

# TESSDATA_PREFIX=/root/tesstutorial/tesseract/tessdata

# vi src/training/tesstrain_util.sh

尋找 phase_E_extract_features()

在 run_command tesseract ${img_file} ${img_file%.*} 後面加入 --psm 7

在 ${box_config} ${config} & 去除 &

移除  jobs="$jobs $!" 和 wait $jobs

# vi src/training/language-specific.sh

加入 你要增加的語言

src/training/tesstrain.sh --fonts_dir /usr/share/fonts --lang plate \

--linedata_only --my_boxtiff_dir /home/tmp --noextract_font_properties \

--langdata_dir ../langdata --tessdata_dir ~/tesstutorial/tesseract/tessdata  \

--output_dir ~/tesstutorial/platetrain

lstmtraining --model_output ~/tesstutorial/impact_from_full/impact \

--continue_from ~/tesstutorial/impact_from_full/eng.lstm \

--traineddata ~/tesstutorial/tesseract/tessdata/eng.traineddata \

--train_listfile ~/tesstutorial/platetrain/plate.training_files.txt \

--max_iterations 400 

lstmtraining --stop_training \

--continue_from ~/tesstutorial/impact_from_full/impact_0.031000_2_400.checkpoint \

--traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

--model_output ~/tesstutorial/impact_from_full/plate.traineddata

刪除無法訓練的圖

for i in `find /tmp/plate-2021-04-13.u4C -name "*.tif"`

do

LSTMF=${i//.tif/.lstmf}

if [ ! -f "$LSTMF" ]; then

  LSTMF=`basename "$LSTMF"`

  TIF=${LSTMF//.lstmf/.tif}

  BOX=${LSTMF//.lstmf/.box}

  rm /home/tmp/$TIF /home/tmp/$BOX

fi

done

Tesseract 在 Docker 上訓練

Tessseract 文件 中 Compiling and Installation 有敘述如何使用 Docker

Tesseract 4 OCR Compilation - Docker Container

安裝 Docker

下載 tesseract-ocr-compilation-master.zip

 $ unzip tesseract-ocr-compilation-master.zip

 $ cd tesseract-ocr-compilation-master

為了方便除錯，建立共通的目錄

 $ vi scripts/3-run-new-container.sh

docker run -d -v debug_path:/home/tmp -p 4022:22 --name t4cmp tesseractshadow/tesseract4cmp

docker ps

執行步驟3 建立 t4cmp container 之後

在 Docker 中安裝 vi, 需在

$ docker exec -it t4cmp bash

# apt-get update

# apt-get install vim

使用 ssh login

# echo 'root:your_passwd' | chpasswd

# sed -i 's/#PermitRootLogin/PermitRootLogin/' /etc/ssh/sshd_config

# exit

$ docker stop t4cmp

$ docker start t4cmp

$ ssh root@localhost -p 4022

1. $ ./scripts/1-pull-container.sh 下載 docker image tesseractshadow/tesseract4cmp

2. $ ./scripts/2-remove-container.sh 移除 t4cmp container

3. $ ./scripts/3-run-new-container.sh 建立 t4cmp container

4. $ ./cripts/4-update-src.sh 更新 Leptionica 和 Tesseract Source

5. $ ./scripts/5-compile-src.sh 編譯 Leptionica 和 Tesseract 

6. $ ./scripts/6-test-ocr.sh 測試，目錄於 ocr-files

7. $ ./scripts/7-build-pkg.sh 建立安裝檔於 pkg 目錄

Tessseract 文件 中 Training for Tesseract 4 有敘述如何 訓練自己的字

TrainingTesseract 4.00 - Detailed Guide by Ray Smith

$ docker exec -it t4cmp bash

# cd /home/workspace

# apt update

# apt install ttf-mscorefonts-installer

會停在下列

[More] 

Progress: [ 54%] [###############################...........................] 

按 Enter 直到出現如下

Do you accept the EULA license terms? [yes/no] 

輸入 yes

# apt install fonts-dejavu

# fc-cache -vf

# cd

# mkdir ~/tesstutorial

# cd ~/tesstutorial

# mkdir langdata

# cd langdata

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/radical-stroke.txt

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/common.punc

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/font_properties

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/Latin.unicharset

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/Latin.xheights

# mkdir eng

# cd eng

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.training_text

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.punc

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.numbers

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.wordlist

# cd ~/tesstutorial

# git clone --depth 1 https://github.com/tesseract-ocr/tesseract.git

# cd tesseract/tessdata

# wget https://github.com/tesseract-ocr/tessdata/raw/master/eng.traineddata

# mkdir best

# cd best

# wget https://github.com/tesseract-ocr/tessdata_best/raw/master/eng.traineddata

# wget https://github.com/tesseract-ocr/tessdata_best/raw/master/chi_tra.traineddata

# cd ~/tesstutorial/tesseract/

# src/training/tesstrain.sh --fonts_dir /usr/share/fonts --lang eng --linedata_only \

  --noextract_font_properties --langdata_dir ../langdata \

  --tessdata_dir ./tessdata --output_dir ~/tesstutorial/engtrain

出現下列錯誤

ERROR: /tmp/eng-2021-03-26.coV/eng.Tex_Gyre_Bonum_Bold.exp0.box does not exist or is not readable

# vi src/training/language-specific.sh 

刪除 Tex Gyre* 字型

重新執行一遍，出現下列兩行，表示成功

Created starter traineddata for LSTM training of language 'eng'

Run 'lstmtraining' command to continue LSTM training for language 'eng'

建立另外一種字型，做為測試用

#src/training/tesstrain.sh --fonts_dir /usr/share/fonts --lang eng --linedata_only \

  --noextract_font_properties --langdata_dir ../langdata \

  --tessdata_dir ./tessdata \

  --fontlist "Impact Condensed" --output_dir ~/tesstutorial/engeval

從頭開始訓練

# mkdir -p ~/tesstutorial/engoutput

# /usr/local/bin/lstmtraining --debug_interval 0 \

  --traineddata ~/tesstutorial/engtrain/eng/eng.traineddata \

  --net_spec '[1,36,0,1 Ct3,3,16 Mp3,3 Lfys48 Lfx96 Lrx96 Lfx256 O1c111]' \

  --model_output ~/tesstutorial/engoutput/base --learning_rate 20e-4 \

  --train_listfile ~/tesstutorial/engtrain/eng.training_files.txt \

  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt \

  --max_iterations 5000 &>~/tesstutorial/engoutput/basetrain.log

因為在 docker 內，--debug_interval 為 0，要看詳細的訓練過程 --debug_interval 為 100

# make training

# make training-install

# make ScrollView.jar

O1c111 表示有111個字，{eng}.unicharset 檔內有 111行

# /usr/local/bin/lstmeval --model ~/tesstutorial/engoutput/base_checkpoint \

  --traineddata ~/tesstutorial/engtrain/eng/eng.traineddata \

  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt

自己從頭錯訓練，誤率太高

At iteration 0, stage 0, Eval Char error rate=107.795608, Word error rate=97.578246

# /usr/local/bin/lstmeval --model ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt

用別人訓練的，好太多

At iteration 0, stage 0, Eval Char error rate=3.095477, Word error rate=9.465216

從最佳的 model 導出可以訓練的 lstm model

# mkdir -p ~/tesstutorial/impact_from_full

# /usr/local/bin/combine_tessdata -e ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  ~/tesstutorial/impact_from_full/eng.lstm

微調訓練

# /usr/local/bin/lstmtraining --model_output ~/tesstutorial/impact_from_full/impact \

  --continue_from ~/tesstutorial/impact_from_full/eng.lstm \

  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --train_listfile ~/tesstutorial/engeval/eng.training_files.txt \

  --max_iterations 400

#/usr/local/bin/lstmeval --model ~/tesstutorial/impact_from_full/impact_checkpoint \

  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt

測試的結果很好

At iteration 0, stage 0, Eval Char error rate=0.000000, Word error rate=0.000000

導出可用的 model

# /usr/local/bin/lstmtraining --stop_training \

  --continue_from ~/tesstutorial/impact_from_full/impact_0.271000_43_400.checkpoint \

  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --model_output ~/tesstutorial/impact_from_full/eng.traineddata

# cp ~/tesstutorial/impact_from_full/eng.traineddata /usr/local/share/tessdata/eng_a.traineddata

# tesseract phototest.tif phototest -l eng_a -psm 1 --oem 1

準備自己的 tif 和 box, 兩者除附檔名外，檔名一樣且以 lang. 開頭 .exp0. 結尾

如 eng.AAA.exp0.tif, eng.AAA.exp0.box, 置於 /home/tmp

修改 tesstrain.sh, 去掉 phase_I_generate_image 8

# src/training/tesstrain_a.sh --fonts_dir /usr/share/fonts --lang eng --linedata_only \

  --my_boxtiff_dir /home/tmp \

  --noextract_font_properties --langdata_dir ../langdata \

  --tessdata_dir ./tessdata --output_dir ~/tesstutorial/platetrain

# /usr/local/bin/lstmtraining --model_output ~/tesstutorial/impact_from_full/impact \

  --continue_from ~/tesstutorial/impact_from_full/eng.lstm \

  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --train_listfile ~/tesstutorial/platetrain/eng.training_files.txt \

  --max_iterations 400

pyinstaller

參考 使用說明

(tensorflow) D:\PlateOcr> pyinstaller --add-binary d:\your_path\opencv_ffmpeg340_64.dll;opencv_ffmpeg340_64.dll --hidden-import opencv-python --hidden-import cv2 --hidden-import another --paths D:\your_path_to_opencv_lib:D:\your_path_to_cv2.xxx.pyd -F PlateOcrEval.py

最終測試成功命令

(tensorflow) D:\PlateOcr> pyinstaller --hidden-import cv2 --paths D:\your_path_to_opencv_lib:D:\your_path_to_cv2.xxx.pyd -F PlateOcrEval.py