Tesseract 在 Docker 上訓練 (二)

$ docker start t4cmp

$ docker exec -it t4cmp bash

# cd /home/workspace/tesseract

# TESSDATA_PREFIX=/root/tesstutorial/tesseract/tessdata

# vi src/training/tesstrain_util.sh

尋找 phase_E_extract_features()

在 run_command tesseract ${img_file} ${img_file%.*} 後面加入 --psm 7

在 ${box_config} ${config} & 去除 &

移除  jobs="$jobs $!" 和 wait $jobs

# vi src/training/language-specific.sh

加入 你要增加的語言

src/training/tesstrain.sh --fonts_dir /usr/share/fonts --lang plate \

--linedata_only --my_boxtiff_dir /home/tmp --noextract_font_properties \

--langdata_dir ../langdata --tessdata_dir ~/tesstutorial/tesseract/tessdata  \

--output_dir ~/tesstutorial/platetrain

lstmtraining --model_output ~/tesstutorial/impact_from_full/impact \

--continue_from ~/tesstutorial/impact_from_full/eng.lstm \

--traineddata ~/tesstutorial/tesseract/tessdata/eng.traineddata \

--train_listfile ~/tesstutorial/platetrain/plate.training_files.txt \

--max_iterations 400 

lstmtraining --stop_training \

--continue_from ~/tesstutorial/impact_from_full/impact_0.031000_2_400.checkpoint \

--traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

--model_output ~/tesstutorial/impact_from_full/plate.traineddata

刪除無法訓練的圖

for i in `find /tmp/plate-2021-04-13.u4C -name "*.tif"`

do

LSTMF=${i//.tif/.lstmf}

if [ ! -f "$LSTMF" ]; then

  LSTMF=`basename "$LSTMF"`

  TIF=${LSTMF//.lstmf/.tif}

  BOX=${LSTMF//.lstmf/.box}

  rm /home/tmp/$TIF /home/tmp/$BOX

fi

done

Tesseract 在 Docker 上訓練

Tessseract 文件 中 Compiling and Installation 有敘述如何使用 Docker

Tesseract 4 OCR Compilation - Docker Container

安裝 Docker

下載 tesseract-ocr-compilation-master.zip

 $ unzip tesseract-ocr-compilation-master.zip

 $ cd tesseract-ocr-compilation-master

為了方便除錯，建立共通的目錄

 $ vi scripts/3-run-new-container.sh

docker run -d -v debug_path:/home/tmp -p 4022:22 --name t4cmp tesseractshadow/tesseract4cmp

docker ps

執行步驟3 建立 t4cmp container 之後

在 Docker 中安裝 vi, 需在

$ docker exec -it t4cmp bash

# apt-get update

# apt-get install vim

使用 ssh login

# echo 'root:your_passwd' | chpasswd

# sed -i 's/#PermitRootLogin/PermitRootLogin/' /etc/ssh/sshd_config

# exit

$ docker stop t4cmp

$ docker start t4cmp

$ ssh root@localhost -p 4022

1. $ ./scripts/1-pull-container.sh 下載 docker image tesseractshadow/tesseract4cmp

2. $ ./scripts/2-remove-container.sh 移除 t4cmp container

3. $ ./scripts/3-run-new-container.sh 建立 t4cmp container

4. $ ./cripts/4-update-src.sh 更新 Leptionica 和 Tesseract Source

5. $ ./scripts/5-compile-src.sh 編譯 Leptionica 和 Tesseract 

6. $ ./scripts/6-test-ocr.sh 測試，目錄於 ocr-files

7. $ ./scripts/7-build-pkg.sh 建立安裝檔於 pkg 目錄

Tessseract 文件 中 Training for Tesseract 4 有敘述如何 訓練自己的字

TrainingTesseract 4.00 - Detailed Guide by Ray Smith

$ docker exec -it t4cmp bash

# cd /home/workspace

# apt update

# apt install ttf-mscorefonts-installer

會停在下列

[More] 

Progress: [ 54%] [###############################...........................] 

按 Enter 直到出現如下

Do you accept the EULA license terms? [yes/no] 

輸入 yes

# apt install fonts-dejavu

# fc-cache -vf

# cd

# mkdir ~/tesstutorial

# cd ~/tesstutorial

# mkdir langdata

# cd langdata

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/radical-stroke.txt

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/common.punc

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/font_properties

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/Latin.unicharset

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata_lstm/master/Latin.xheights

# mkdir eng

# cd eng

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.training_text

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.punc

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.numbers

# wget https://raw.githubusercontent.com/tesseract-ocr/langdata/master/eng/eng.wordlist

# cd ~/tesstutorial

# git clone --depth 1 https://github.com/tesseract-ocr/tesseract.git

# cd tesseract/tessdata

# wget https://github.com/tesseract-ocr/tessdata/raw/master/eng.traineddata

# mkdir best

# cd best

# wget https://github.com/tesseract-ocr/tessdata_best/raw/master/eng.traineddata

# wget https://github.com/tesseract-ocr/tessdata_best/raw/master/chi_tra.traineddata

# cd ~/tesstutorial/tesseract/

# src/training/tesstrain.sh --fonts_dir /usr/share/fonts --lang eng --linedata_only \

  --noextract_font_properties --langdata_dir ../langdata \

  --tessdata_dir ./tessdata --output_dir ~/tesstutorial/engtrain

出現下列錯誤

ERROR: /tmp/eng-2021-03-26.coV/eng.Tex_Gyre_Bonum_Bold.exp0.box does not exist or is not readable

# vi src/training/language-specific.sh 

刪除 Tex Gyre* 字型

重新執行一遍，出現下列兩行，表示成功

Created starter traineddata for LSTM training of language 'eng'

Run 'lstmtraining' command to continue LSTM training for language 'eng'

建立另外一種字型，做為測試用

#src/training/tesstrain.sh --fonts_dir /usr/share/fonts --lang eng --linedata_only \

  --noextract_font_properties --langdata_dir ../langdata \

  --tessdata_dir ./tessdata \

  --fontlist "Impact Condensed" --output_dir ~/tesstutorial/engeval

從頭開始訓練

# mkdir -p ~/tesstutorial/engoutput

# /usr/local/bin/lstmtraining --debug_interval 0 \

  --traineddata ~/tesstutorial/engtrain/eng/eng.traineddata \

  --net_spec '[1,36,0,1 Ct3,3,16 Mp3,3 Lfys48 Lfx96 Lrx96 Lfx256 O1c111]' \

  --model_output ~/tesstutorial/engoutput/base --learning_rate 20e-4 \

  --train_listfile ~/tesstutorial/engtrain/eng.training_files.txt \

  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt \

  --max_iterations 5000 &>~/tesstutorial/engoutput/basetrain.log

因為在 docker 內，--debug_interval 為 0，要看詳細的訓練過程 --debug_interval 為 100

# make training

# make training-install

# make ScrollView.jar

O1c111 表示有111個字，{eng}.unicharset 檔內有 111行

# /usr/local/bin/lstmeval --model ~/tesstutorial/engoutput/base_checkpoint \

  --traineddata ~/tesstutorial/engtrain/eng/eng.traineddata \

  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt

自己從頭錯訓練，誤率太高

At iteration 0, stage 0, Eval Char error rate=107.795608, Word error rate=97.578246

# /usr/local/bin/lstmeval --model ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt

用別人訓練的，好太多

At iteration 0, stage 0, Eval Char error rate=3.095477, Word error rate=9.465216

從最佳的 model 導出可以訓練的 lstm model

# mkdir -p ~/tesstutorial/impact_from_full

# /usr/local/bin/combine_tessdata -e ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  ~/tesstutorial/impact_from_full/eng.lstm

微調訓練

# /usr/local/bin/lstmtraining --model_output ~/tesstutorial/impact_from_full/impact \

  --continue_from ~/tesstutorial/impact_from_full/eng.lstm \

  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --train_listfile ~/tesstutorial/engeval/eng.training_files.txt \

  --max_iterations 400

#/usr/local/bin/lstmeval --model ~/tesstutorial/impact_from_full/impact_checkpoint \

  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --eval_listfile ~/tesstutorial/engeval/eng.training_files.txt

測試的結果很好

At iteration 0, stage 0, Eval Char error rate=0.000000, Word error rate=0.000000

導出可用的 model

# /usr/local/bin/lstmtraining --stop_training \

  --continue_from ~/tesstutorial/impact_from_full/impact_0.271000_43_400.checkpoint \

  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --model_output ~/tesstutorial/impact_from_full/eng.traineddata

# cp ~/tesstutorial/impact_from_full/eng.traineddata /usr/local/share/tessdata/eng_a.traineddata

# tesseract phototest.tif phototest -l eng_a -psm 1 --oem 1

準備自己的 tif 和 box, 兩者除附檔名外，檔名一樣且以 lang. 開頭 .exp0. 結尾

如 eng.AAA.exp0.tif, eng.AAA.exp0.box, 置於 /home/tmp

修改 tesstrain.sh, 去掉 phase_I_generate_image 8

# src/training/tesstrain_a.sh --fonts_dir /usr/share/fonts --lang eng --linedata_only \

  --my_boxtiff_dir /home/tmp \

  --noextract_font_properties --langdata_dir ../langdata \

  --tessdata_dir ./tessdata --output_dir ~/tesstutorial/platetrain

# /usr/local/bin/lstmtraining --model_output ~/tesstutorial/impact_from_full/impact \

  --continue_from ~/tesstutorial/impact_from_full/eng.lstm \

  --traineddata ~/tesstutorial/tesseract/tessdata/best/eng.traineddata \

  --train_listfile ~/tesstutorial/platetrain/eng.training_files.txt \

  --max_iterations 400

Ubuntu 18.04 重灌

依據 https://developer.nvidia.com/deepstream-getting-started

選擇各個版本

https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html

sudo apt-get install ssh

sudo apt install python3-pip

sudo ln -s /usr/bin/python3 /usr/bin/python

安裝 NVIDIA Driver

https://www.linuxbabe.com/ubuntu/install-nvidia-driver-ubuntu-18-04

sudo lshw -c display

sudo lshw -c video

sudo ubuntu-drivers devices

sudo ubuntu-drivers autoinstall

sudo reboot

sudo lshw -c display

安裝 CUDA Toolkit

https://developer.nvidia.com/cuda-downloads

請選擇 CUDA Toolkit 10.2

Linux/x86_64/Ubuntu/18.04/deb(local)

vi ~/.bashrc

export PATH=/usr/local/cuda-10.2:$PATH

export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64:$LD_LIBRARY_PATH

安裝 CUDNN

https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

https://developer.nvidia.com/cudnn

選擇 cuDNN v8.0.5 for CUDA 10.2

選擇 cuDNN Library for Linux (x86)

tar -xzvf cudnn-10.2-linux-x64-v8.0.5.39.tgz

sudo cp cuda/include/cudnn*.h /usr/local/cuda/include

sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64

sudo chmod a+r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn*

sudo ldconfig

/sbin/ldconfig.real: /usr/local/cuda-10.2/targets/x86_64-linux/liblibcudnn.so.8 is not a symbolic link

cd /usr/local/cuda-10.2/targets/x86_64-linux/lib

sudo rm liblibcudnn.so.8 liblibcudnn.so

sudo ln -s libcudnn.so.8.0.5 libcudnn.so.8

sudo ln -s libcudnn.so.8 libcudnn.so

sudo ldconfig

安裝 TensorRT

下載 TensorRT binary

https://developer.nvidia.com/nvidia-tensorrt-7x-download

選擇 TensorRT 7.1.3.4 for Ubuntu 18.04 and CUDA 10.2 TAR package

version="7.1.3.4"

os="Ubuntu-18.04"

arch=$(uname -m)

cuda="cuda-10.2"

cudnn="cudnn8.0"

tar -xvzf TensorRT-7.1.3.4.Ubuntu-18.04.x86_64-gnu.cuda-10.2.cudnn8.0.tar.gz

vi ~/.bashrc

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/Data/TensorRT/TensorRT-7.1.3.4/lib

source ~/.bashrc

cd TensorRt-7.1.3.4/python

sudo pip3 install tensorrt-7.1.3.4-cp36-none-linux_x86_64.whl

cd ../uff

sudo pip3 install uff-0.6.9-py2.py3-none-any.whl

cd ../graphsurgeon/

sudo pip3 install graphsurgeon-0.4.5-py2.py3-none-any.whl

安裝 GStreamer

https://yingrenn.blogspot.com/2020/09/gstreamer.html

安裝 DeepStream

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Quickstart.html#dgpu-setup-for-ubuntu

sudo apt install \

 libssl1.0.0 \

 libgstreamer1.0-0 \

 gstreamer1.0-tools \

 gstreamer1.0-plugins-good \

 gstreamer1.0-plugins-bad \

 gstreamer1.0-plugins-ugly \

 gstreamer1.0-libav \

 libgstrtspserver-1.0-0 \

 libgstrtspserver-1.0-dev \

 libjansson4

git clone https://github.com/edenhill/librdkafka.git

cd librdkafka

git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a

./configure

make

sudo make install

sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.0/lib

sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.0/lib

下載 DeepStream

https://developer.nvidia.com/assets/Deepstream/5.0/ga/secure/deepstream_sdk_5.0.1_x86_64.tbz2

sudo tar -xvf deepstream_sdk_v5.0.1_x86_64.tbz2 -C /

cd /opt/nvidia/deepstream/deepstream-5.0/

sudo ./install.sh

sudo ldconfig

安裝 CMake v3.13

wget http://www.cmake.org/files/v3.13/cmake-3.13.5.tar.gz

tar xpvf cmake-3.13.5.tar.gz cmake-3.13.5

cd cmake-3.13.5

sudo apt-get install zlib1g-dev

sudo apt-get install curl

sudo apt-get install libcurl3

sudo apt-get install libcurl4 libcurl4-openssl-dev

./bootstrap --system-curl

make -j$(nproc)

sudo make install

安裝 Docker

https://docs.docker.com/engine/install/ubuntu/

sudo apt-get update

sudo apt-get install \

    apt-transport-https \

    ca-certificates \

    curl \

    gnupg-agent \

    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo apt-key fingerprint 0EBFCD88

sudo add-apt-repository \

   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \

   $(lsb_release -cs) \

   stable"

sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io

安裝 NVIDIA Docker

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker

curl https://get.docker.com | sh \

  && sudo systemctl start docker \

  && sudo systemctl enable docker

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \

   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \

   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update

sudo apt-get install -y nvidia-docker2

sudo systemctl restart docker

sudo groupadd docker

sudo usermod -a -G docker $USER

sudo reboot

若 sudo apt update 出現

W: Target CNF (stable/cnf/Commands-all) is configured multiple times in /etc/apt/sources.list:52 and /etc/apt/sources.list.d/docker.list:1

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4773BD5E130D1D45

sudo rm /etc/apt/sources.list.d/docker.list

安裝 TensorRT 7.1 OSS

https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html

https://github.com/NVIDIA/TensorRT/tree/master

從 master 切換到 release/7.1

下載 TensorRT OSS

TensorRT OSS: 包含 TensorRT plugins, Caffe 和 ONNX parsers 等

git clone -b master https://github.com/nvidia/TensorRT TensorRT

cd TensorRT

git submodule update --init --recursive

參考之前的 TensorRT binary

cd TensorRT-7.1.3.4

export TRT_RELEASE=`pwd`

cd $TRT_SOURCE

./docker/build.sh --file docker/ubuntu.Dockerfile --tag tensorrt-ubuntu --os 18.04 --cuda 10.2

./docker/launch.sh --tag tensorrt-ubuntu --gpus all --release $TRT_RELEASE --source $TRT_SOURCE

cd $TRT_SOURCE

mkdir -p build && cd build

cmake .. -DTRT_LIB_DIR=$TRT_RELEASE/lib -DTRT_OUT_DIR=`pwd`/out

make -j$(nproc)

exit

mkdir backup

mv $TRT_RELEASE/targets/x86_64-linux-gnu/lib/libnvinfer_plugin.so.7.1.3 backup

cp $TRT_SOURCE/build/out/libnvinfer_plugin.so.7.2.1 $TRT_RELEASE/targets/x86_64-linux-gnu/lib/libnvinfer_plugin.so.7.1.3

安裝 OpenCV 4.4

git clone https://github.com/opencv/opencv.git opencv-4.4.0 -b 4.4.0 --depth 1

git clone https://github.com/opencv/opencv_contrib.git opencv_contrib-4.4.0 -b 4.4.0 --depth 1

sudo apt-get update

sudo add-apt-repository ppa:alex-p/tesseract-ocr

sudo apt install tesseract-ocr libtesseract-dev

sudo apt-get install libleptonica-dev

sudo apt-get install qt5-default

sudo apt-get install qtcreator

sudo apt-get install build-essential cmake unzip pkg-config

sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev

sudo apt-get install libxvidcore-dev libx264-dev

sudo apt-get install libgtk-3-dev

sudo apt-get install libatlas-base-dev gfortran

sudo apt-get install libhdf5-dev

sudo apt-get install python3-dev

mkdir build; cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \

-D CMAKE_INSTALL_PREFIX=/usr/local \

-D INSTALL_PYTHON_EXAMPLES=ON \

-D INSTALL_C_EXAMPLES=OFF \

-D WITH_TBB=ON \

-D BUILD_opencv_cudacodec=OFF \

-D ENABLE_FAST_MATH=1 \

-D CUDA_FAST_MATH=1 \

-D WITH_CUDA=ON \

-D WITH_CUBLAS=ON \

-D WITH_LIBV4L=ON \

-D BUILD_opencv_python3=ON \

-D BUILD_opencv_python2=OFF \

-D BUILD_opencv_java=OFF \

-D WITH_V4L=ON \

-D WITH_QT=ON \

-D WITH_OPENGL=ON \

-D WITH_GSTREAMER=ON \

-D WITH_GTK=ON \

-D OPENCV_GENERATE_PKGCONFIG=ON \

-D OPENCV_PC_FILE_NAME=opencv.pc \

-D OPENCV_ENABLE_NONFREE=OFF \

-D OPENCV_EXTRA_MODULES_PATH=/your_path_to/opencv/opencv_contrib-4.4.0/modules \

-D BUILD_EXAMPLES=ON \

-D WITH_CUDNN=ON \

-D CUDNN_VERSION="8.0.5" \

-D OPENCV_DNN_CUDA=ON \

-D CUDNN_INCLUDE_DIR=/usr/local/cuda/include \

-D CUDNN_LIBRARY=/usr/local/cuda/lib64/libcudnn.so.8.0.5 \

-D CUDA_ARCH_BIN=7.5 \

../opencv-4.4.0

#-D OPENCV_ENABLE_NONFREE=ON \

#-D OpenGL_GL_PREFERENCE=LEGACY \

make -j$(nproc)

sudo make install

sudo ldconfig

opencv_version -v

下載 Tesseract 的中文字 chi_tra.traineddata, chi_tra_vert.traineddate

https://github.com/tesseract-ocr/tessdata

放至 /usr/share/tesseract-ocr/4.00/tessdata

安裝 Xpra

https://www.xpra.org/trac/wiki/Building/Debian

sudo apt-get install libx11-dev libxtst-dev libxcomposite-dev libxdamage-dev \

 libxkbfile-dev python-all-dev

sudo apt-get install libgtk-3-dev python3-dev python3-cairo-dev python-gi-dev cython3

sudo apt-get install xauth x11-xkb-utils

sudo apt-get install libx264-dev libvpx-dev yasm

sudo apt-get install libnvidia-encode-440

sudo apt-get install libavformat-dev libavcodec-dev libswscale-dev

sudo apt-get install libturbojpeg-dev

sudo apt-get install libwebp-dev

sudo apt-get install uglifyjs brotli libjs-jquery libjs-jquery-ui gnome-backgrounds

sudo apt-get install python3-opengl python3-numpy python3-pil

sudo apt-get install python3-rencode python3-lz4 python3-dbus python3-cryptography \

 python3-netifaces python3-yaml

sudo apt-get install python3-setproctitle python3-xdg python3-pyinotify python3-opencv

sudo apt-get install libpam-dev quilt xserver-xorg-dev xutils-dev xvfb keyboard-configuration

sudo apt-get install python3-kerberos python3-gssapi

sudo apt-get install gstreamer1.0-pulseaudio gstreamer1.0-alsa \

 gstreamer1.0-plugins-base gstreamer1.0-plugins-good \

 gstreamer1.0-plugins-ugly

sudo apt-get install cups-filters cups-common cups-pdf python3-cups

sudo apt-get install openssh-client sshpass python3-paramiko

sudo apt-get install devscripts build-essential lintian debhelper

wget https://www.xpra.org/src/xpra-4.0.4.tar.xz

tar -xf xpra-4.0.6.tar.xz

cd xpra-4.0.6

vi setup.py

#!/usr/bin/env python3

sudo ./setup.py install

安裝 frp

https://yingrenn.blogspot.com/2020/03/frp.html

vi frps.ini

[common]

bind_port = 7000

dashboard_port = 7001

dashboard_user = user

dashboard_pwd = password

vi /etc/systemd/system/frps.service

[Unit]

Description=FRP Server Daemon

[Service]

Type=simple

ExecStartPre=-/usr/sbin/setcap cap_net_bind_service=+ep /home/mark/Data/frp/frp_0.34.3_linux_amd64/frps

ExecStart=/path_to_frp/frp_0.34.3_linux_amd64/frps -c /path_to_frp/frp_0.34.3_linux_amd64/frps.ini

Restart=always

RestartSec=20s

User=nobody

PermissionsStartOnly=true

LimitNOFILE=infinity

[Install]

WantedBy=multi-user.target

sudo systemctl start frps

sudo systemctl enable ftps

安裝 Gitea Tomcat Nginx

https://yingrenn.blogspot.com/2019/11/gitea-in-ubuntu.html

https://yingrenn.blogspot.com/search/label/tomcat

安裝 x11vnc

因為 Ubuntu 18.04 改 LightDM 到 GDM3, 所以要在 console login 後，才能動作

sudo apt install x11vnc

sudo x11vnc -storepasswd

sudo chown mark.mark ~/.vnc/passwd

sudo vi /etc/systemd/system/x11vnc.service

# Description: Custom Service Unit file

# File: /etc/systemd/system/x11vnc.service

[Unit]

Description="x11vnc"

Requires=display-manager.service

After=display-manager.service

[Service]

ExecStart=/usr/bin/x11vnc -loop -nopw -xkb -repeat -noxrecord -noxfixes -noxdamage -forever -rfbport 5900 -display :1 -auth /run/user/1000/gdm/Xauthority -rfbauth /home/mark/.vnc/passwd

ExecStop=/usr/bin/killall x11vnc

Restart=on-failure

RestartSec=2

[Install]

WantedBy=multi-user.target

安裝 VirtualBox

https://www.virtualbox.org/wiki/Downloads

到官網下載

virtualbox-6.1_6.1.16-140961~Ubuntu~bionic_amd64.deb

Oracle_VM_VirtualBox_Extension_Pack-6.1.16.vbox-extpack

sudo dpkg -i virtualbox-6.1_6.1.16-140961~Ubuntu~bionic_amd64.deb

sudo groupadd win10disk

sudo usermod -a -G win10disk youruser

sudo udevadm info /dev/sdX | grep UUID

E: ID_PART_TABLE_UUID=01234567-89ab-cdef-0123-456789abcde

vi /etc/udev/rules.d/99-win10disk.rules

ENV{ID_PART_TABLE_UUID}=="1234567-89ab-cdef-0123-456789abcde", GROUP="win10disk"

ls -l /dev/sdb

brw-rw---- 1 root win10disk 8, 16 Nov 4 23:33 /dev/sdb

VBoxManage internalcommands createrawvmdk -filename .VirtualBox/Crucial1T.vmdk -rawdisk /dev/sda

sudo usermod -a -G vboxusers youruser

拷貝舊的 VM, 修改 win10.vbox, win10.vbox-prev 內的 uuid

與 .VirtualBox/Crucial1T.vmdk 的 uuid 一致

開啟 VirtualBox

File/Preferences...

Extensions 按 +

選擇剛下載的 Oracle_VM_VirtualBox_Extension_Pack-6.1.16.vbox-extpack

只有登入畫面無法使用滑鼠和鍵盤，其餘正常

sudo apt-get instll xserver-xorg-input-all

tsseract 訓練 在 Unbuntu 上

https://github.com/tesseract-ocr/tesseract/wiki/Compiling
https://github.com/tesseract-ocr/tesseract/wiki/Compiling-%E2%80%93-GitInstallation

sudo apt-get install g++
sudo apt-get install autoconf automake libtool
sudo apt-get install pkg-config
sudo apt-get install libpng-dev
sudo apt-get install libjpeg8-dev
sudo apt-get install libtiff5-dev
sudo apt-get install zlib1g-dev

sudo apt-get install libicu-dev
sudo apt-get install libpango1.0-dev
sudo apt-get install libcairo2-dev

sudo apt-get install libleptonica-dev

sudo apt install git

git clone https://github.com/tesseract-ocr/tesseract.git tesseract-ocr
git clone https://github.com/tesseract-ocr/langdata.git langdata
git clone https://github.com/tesseract-ocr/tessdata_best.git tessdata_best
git clone https://github.com/tesseract-ocr/tessdata_fast.git tessdata_fast
cd tesseract-ocr
./autogen.sh
./configure
make
sudo make install
sudo ldconfig
make training
sudo make training-install

https://github.com/tesseract-ocr/tesseract/wiki/Data-Files

sudo apt install curl
sudo apt install default-jre
sudo apt install openjdk-11-jre-headless
sudo apt install openjdk-8-jre-headless
sudo apt install default-jdk
sudo apt install openjdk-11-jdk-headless
sudo apt install openjdk-8-jdk-headless

cd java
make ScrollView.jar

https://github.com/tesseract-ocr/tesseract/wiki/ViewerDebugging

text2image --find_fonts \
--fonts_dir /usr/share/fonts \
--text ./langdata/plate/plate.
--min_coverage .9  \
--outputbase ./langdata/plate/plate \
|& grep raw \
 | sed -e 's/ :.*/@ \\/g' \
 | sed -e "s/^/  '/" \
 | sed -e "s/@/'/g" >./langdata/plate/fontslist.txt

text2image --font="aakar Medium" \
--fonts_dir /usr/share/fonts \
--text ./langdata/plate/plate.txt \
--min_coverage .9  \
--outputbase ./langdata/plate/plate.aakar_Medium.exp0

text2image --font="Abyssinica SIL" \
--fonts_dir /usr/share/fonts \
--text ./langdata/plate/plate.txt \
--min_coverage .9  \
--outputbase ./langdata/plate/plate.Abyssinica_SIL.exp0

mkdir training
cd training
mkdir -p langdata/eng
準備 langdata/eng/eng.traning_text
拷貝 langdata/radical-stroke.txt
mkdir -p tessdata/configs
拷貝 tessdata/eng.traineddata
拷貝 tessdata/configs/lstm.train

~/ocr/tesseract-ocr/src/training/tesstrain.sh --fonts_dir /usr/share/fonts \
 --lang eng --linedata_only \
 --noextract_font_properties --langdata_dir langdata \
 --tessdata_dir tessdata \
 --output_dir train

若字型不存在 (Could not find font named 'xxx xxx')
編輯 src/training/language-specific.sh 刪除不存在的字型

tif 檔產生到一半停住了, 原因是利用背景處理加速
let rem=counter%par_factor 為零時出錯
修改 src/training/tesstrain_utils.sh, 不要使用背景處理
有以下兩段, 刪除命令後面的 &
phase_I_generate_image()
  generate_font_image "${font}"
phase_E_extract_features()
  run_command tesseract ${img_file} ${img_file%.*} ${box_config} ${config} &
並刪除相關的 let, if, wait

combine_tessdata -e ../tessdata_best/eng.traineddata train/eng.lstm

lstmtraining --model_output model \
 --continue_from train/eng.lstm \
 --traineddata train/eng/eng.traineddata \
 --old_traineddata ../tessdata_best/eng.traineddata \
 --train_listfile train/eng.training_files.txt \
 --max_iterations 3600

lstmtraining --stop_training \
 --continue_from model_checkpoint \
 --traineddata train/eng/eng.traineddata \
 --old_traineddata ../tessdata_best/eng.traineddata \
 --model_output new.traineddata

若要直接使用 tif 和 box
修改 src/training/tesstrain_utils.sh, 固定 tif, box 檔案的目錄

    #TRAINING_DIR=${TMP_DIR}

    TRAINING_DIR=/tmp/images

修改 src/training/tesstrain.sh, 不要自動產生圖形

#phase_I_generate_image 8

Tesseract 訓練, 使用 jTessBoxEditor

這是舊版的訓練方式，Tesseract 4.0 開始了新的 LSTM(AI) 的方式

D:\>java -jar jTessBoxEditor.jar
TIFF/Box Generator
text2image 不要勾
Input 選擇 .txt 文字檔
Output 按 ... 設定輸出位置
英文輸入 eng, 中文輸入 chi_tra
選擇字型
按 Generate 產生 .box 和 .font_properties

tif文面命名格式[lang].[fontname].exp[num].tif
lang:plt
fontname:normal
num:0

手動由 .tif -> .box
D:\>ttesseract.exe plt.normal.exp0.tif plt.normal.exp0 --psm 7 -l eng batch.nochop makebox

手動生成 .font_properties, 內容為 <fontname> <italic> <bold> <fixed> <serif> <fraktur>
echo normal 0 1 1 0 1 >plt.font_properties

由 .box 文件 -> .tr
D:\>tesseract.exe plt.normal.exp0.tif plt.normal.exp0 box.train.stderr
由 .box 文件 -> unicharset
D:\>unicharset_extractor.exe plt.normal.exp0.box
由 font_properties, unicharset, .tr -> shapetable
D:\>shapeclustering.exe -F plt.font_properties -U unicharset plt.normal.exp0.tr
由 font_properties, unicharset, .tr ->lang.unicharset, inttemp, pffmtable
D:\>mftraining.exe -F plt.font_properties -U unicharset -O plt.unicharset plt.normal.exp0.tr
由.tr 文件 -> normproto
D:\>cntraining.exe plt.normal.exp0.tr
重命名 以 lang. 開頭重命名 inttemp, normproto, pffmtable, shapetable
D:\>move inttemp plt.inttemp
D:\>move normproto plt.normproto
D:\>move pffmtable plt.pffmtable
D:\>move shapetable plt.shapetable
合併生成 lang.traineddata文件，在這裏爲 plt.traineddata
D:\>combine_tessdata.exe plt.

EAST Tesseract 效能測試

EAST: An Efficient and Accurate Scene Text Detector

參考 OpenCV OCR and text recognition with Tesseract

發現 opencv 使用硬體加速是
net = cv2.dnn.readNet(args["east"])
net.setPreferableTarget(cv2.dnn.DNN_TARGET_OPENCL);

而 OPENCL 並不是 NVDIA 的 CUDA 是 Intel(GPU)

參考 A tensorflow implementation of EAST text detector 的 eval.py

使用了 tensorflow 的 gpu(CUDA)

效能的卻比較好

640x480 從 400ms 到 340ms

但 Tsseract 沒有加速，只能加速 EAST

OpenCV Tesseract sample

samples/text/(sample)webcam_demo
Configuration Properties/Debugging/Environment
PATH=C:\Program Files\tesseract\bin;D:\TensorFlow\OCR\tesseract\win64\bin\Debug;D:\Anaconda3\Library\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin;%PATH%
QT_PLUGIN_PATH=D:\Anaconda3\Library\plugins

修改 webcam_demo.cpp 指定下列檔案的路徑

D:/OpenCV_4/build34/testdata/contrib/text/trained_classifierNM1.xml

D:/OpenCV_4/build34/testdata/contrib/text/trained_classifierNM2.xml

D:/OpenCV_4/build34/testdata/contrib/text/OCRHMM_knn_model_data.xml.gz

Build OpenCV v4

目前最新的版本為 4.0.0, 穩定版本 3.4.3
查詢 opencv 版本
opencv_src\modules\core\include\opencv2\core\version.hpp

查詢 python 版本

(base) D:\OpenCV_4>python --version

Python 3.6.4 :: Anaconda, Inc.

CUDA 使用 10 版，9 or 9.1 都不行

下載 OpenCV 最新版本

(base) D:\OpenCV_4>git clone https://github.com/opencv/opencv opencv

下載指定版本

(base) D:\OpenCV_4>git clone https://github.com/opencv/opencv --branch 3.4 opencv34

更新之前下載的程式 到 遠端的最新版本

(base) D:\OpenCV_4\opencv>git pull origin master

Eigen 是一個 C++ 程式庫，支持線性代數，矩陣，向量運算

http://eigen.tuxfamily.org

解壓縮後移至 D:\OpenCV_4\dep

Intel TBB Library 不需安裝

用 Intel IPP 即可

MikTex 排版軟體

http://miktex.org/

(base) D:\OpenCV_4>python -m pip install --upgrade pip

Sphinx Python文件生成工具，支持 LaTeX(可以產生PDF)

(base) D:\OpenCV_4>pip install sphinx

NumPy 是Python語言的一個擴充程式庫。支援高階大量的維度陣列與矩陣運算。

(base) D:\OpenCV_4>pip install numpy

Visual Studio Installer 安裝

個別元件/VC++ 2017 v141 工具組

OpenCV 的額外模組

git clone https://github.com/opencv/opencv_contrib open_contrib

git clone https://github.com/opencv/opencv_contrib --branch 3.4 open_contrib_3.4

CMake

D:/OpenCV_4/opencv

D:/OpenCV_4/build_64

Visual Studio 14 2015 Win64

EIGEN_INCLUDE_PATH=D:/OpenCV_4/dep/eigen-eigen-b3f3d4950030

OPENCV/OPENCV_EXTRA_MODULES_PATH=D:\OpenCV_4\opencv34\modules
BUILD/BUILD_EXAMPLES checked

opencv_cudev 專案出現 LNK2019 cv::__termination 錯誤

修改 modules/opencv_cudev/CMakeLists.txt, 在

set(the_description "CUDA device layer")

增加下一行

set(OPENCV_SKIP_DLLMAIN_GENERATION ON)

opencv_cvv 專案 stringutils.cpp 出現 C2001, C2143, C2146 等錯誤

是因為檔案字元集的問題，開啟 stringutils.cpp, 重新存檔成

UTF-8 with signature

以下增加 Tesseract 功能，參考 Tesseract OCR 安裝
修改 modules/opencv_text/CMakeLists.txt, 增加下一行
set(Tesseract_FOUND 1)
if(NOT CMAKE_CROSSCOMPILING OR OPENCV_FIND_TESSERACT)

修改 opencv_text Property Pages/Configuration Properties/
C/C++/General/Additional Include Directories 增加
C:\Program Files\tesseract\include
Linker/General/Additional Library Directories 增加
D:\TensorFlow\OCR\tesseract\win64\Debug
C:\Program Files\tesseract\lib
Linker/Input/Additional Dependencies 增加

tesseract40d.lib (Debug)

tesseract40.lib (Release)

Tesseract OSD_example.cpp

// 參照 Tesseract API for VS2017

// 決定頁面，行到行，字到字 的方向

#include "pch.h"
#include <iostream>

#include <tesseract/baseapi.h>
#include <leptonica/allheaders.h>

int main()
{
// 頁面方向
static const char* const sOrientation[] {
"PAGE_UP",
"PAGE_RIGHT",
"PAGE_DOWN",
"PAGE_LEFT",
};
// 字到字的方向
static const char* const sWritingDirection[] {
"LEFT_TO_RIGHT",
"RIGHT_TO_LEFT",
"TOP_TO_BOTTOM",
};
// 行到行的方向
static const char* const sTextlineOrder[]{
"LEFT_TO_RIGHT",
"RIGHT_TO_LEFT",
"TOP_TO_BOTTOM",
};

Pix *image = pixRead("D:\\TensorFlow\\OCR\\aaa.png");
tesseract::TessBaseAPI *api = new tesseract::TessBaseAPI();
if (api->Init(NULL, "eng")) {
std::cerr << "Could not initialize tesseract.\n";
}
api->SetPageSegMode(tesseract::PSM_AUTO_OSD);
api->SetImage(image);
api->Recognize(0);
tesseract::PageIterator* it = api->AnalyseLayout();
tesseract::PageIteratorLevel level = tesseract::RIL_WORD;
if (it != 0) {
do {
tesseract::Orientation orientation;
tesseract::WritingDirection direction;
tesseract::TextlineOrder order;
float deskew_angle;

it->Orientation(&orientation, &direction, &order, &deskew_angle);
printf("Orientation: %s;\nWritingDirection: %s\nTextlineOrder: %s\n" \
"Deskew angle: %.4f\n",
sOrientation[orientation], sWritingDirection[direction],
sTextlineOrder[order], deskew_angle);
int left, top, right, bottom;
it->BoundingBox(level, &left, &top, &right, &bottom);
printf("BoundingBox: (%d, %d) (%d, %d)\n",
left, top, right, bottom);
} while (it->Next(level));
}

api->End();
pixDestroy(&image);
}

Tesseract ResultIterator.cpp

// 參照 Tesseract API for VS2017

// 依據字符辨識，列出所有候選字

#include "pch.h"

#include <iostream>

#include <tesseract/baseapi.h>

#include <leptonica/allheaders.h>

int main()

{

Pix *image = pixRead("D:\\TensorFlow\\OCR\\aaa.png");

tesseract::TessBaseAPI *api = new tesseract::TessBaseAPI();

if (api->Init(NULL, "eng")) {

std::cerr << "Could not initialize tesseract.\n";

}

api->SetPageSegMode(tesseract::PSM_AUTO_OSD);

api->SetImage(image);

api->Recognize(0);

tesseract::ResultIterator* ri = api->GetIterator();

tesseract::PageIteratorLevel level = tesseract::RIL_SYMBOL;

//tesseract::PageIteratorLevel level = tesseract::RIL_TEXTLINE;

if (ri != 0) {

do {

const char* word = ri->GetUTF8Text(level);

float conf = ri->Confidence(level);

int x1, y1, x2, y2;

ri->BoundingBox(level, &x1, &y1, &x2, &y2);

printf("word: '%s';  \tconf: %.2f; BoundingBox: %d,%d,%d,%d;\n",

word, conf, x1, y1, x2, y2);

if (level = tesseract::RIL_SYMBOL) {

// 列出所有可能的候選字

tesseract::ChoiceIterator ci(*ri);

do {

const char* choice = ci.GetUTF8Text();

printf("\t\t%s conf: %f\n", choice, ci.Confidence());

} while (ci.Next());

printf("---------------------------------------------\n");

}

delete[] word;

} while (ri->Next(level));

}

api->End();

pixDestroy(&image);

}

Tesseract GetComponentImages.cpp

// 參照 Tesseract API for VS2017
// 依據行辨識

int main()

{

Pix *image = pixRead("D:\\temp\\OpenCV_err.png");

tesseract::TessBaseAPI *api = new tesseract::TessBaseAPI();

if (api->Init(NULL, "eng")) {

std::cerr << "Could not initialize tesseract.\n";

}

api->SetImage(image);

Boxa* boxes = api->GetComponentImages(tesseract::RIL_TEXTLINE, true, NULL, NULL);

fprintf(stdout, "Found %d textline image components.\n", boxes->n);

for (int i = 0; i < boxes->n; i++) {

BOX* box = boxaGetBox(boxes, i, L_CLONE);

api->SetRectangle(box->x, box->y, box->w, box->h);

char* ocrResult = api->GetUTF8Text();

int conf = api->MeanTextConf();

fprintf(stdout, "Box[%d]: x=%d, y=%d, w=%d, h=%d, confidence: %d, text: %s",

i, box->x, box->y, box->w, box->h, conf, ocrResult);

delete[] ocrResult;

}

api->End();

pixDestroy(&image);

}

Tesseract BasicExample.cpp

// 參照 Tesseract API for VS2017
// 整張圖片一次辨識
#include "pch.h"
#include <iostream>

#include <tesseract/baseapi.h>
#include <leptonica/allheaders.h>

int main()
{
char *outText = NULL;
tesseract::TessBaseAPI *api = new tesseract::TessBaseAPI();
// 英文加中文
if (api->Init(NULL, "eng+chi_tra")) {
std::cerr << "Could not initialize tesseract.\n";
}
Pix *image = pixRead("D:\\TensorFlow\\OCR\\bbb.png");
// 以下兩行，可 擇一 或 都不執行
api->SetPageSegMode(tesseract::PSM_SINGLE_BLOCK); // 預設值
//api->SetPageSegMode(tesseract::PSM_SINGLE_LINE);
api->SetImage(image);
//api->SetRectangle(40, 5, 150, 30);
outText = api->GetUTF8Text();
// 要顯示中文要經很多轉換
int len = ::MultiByteToWideChar(CP_UTF8, NULL, outText, -1, NULL, 0);
wchar_t* wszString = new wchar_t[len + 1];
::MultiByteToWideChar(CP_UTF8, NULL, outText, -1, wszString, len);
wszString[len] = '\0';
len = ::WideCharToMultiByte(CP_ACP, 0, wszString, -1, NULL, 0, NULL, NULL);
char* szBig5 = new char[len + 1];
::WideCharToMultiByte(CP_ACP, 0, wszString, -1, szBig5, len, NULL, NULL);
szBig5[len] = '\0';

std::cout << "=======================\n";
std::cout << outText << "\n";
std::cout << "=======================\n";
std::cout << wszString << "\n";
std::cout << "=======================\n";
std::cout << szBig5 << "\n";
std::cout << "=======================\n";
api->End();
if (outText) delete[] outText;
if (wszString) delete[] wszString;
pixDestroy(&image);
}

Tesseract API for VS2017

參照 Tesseract OCR 安裝

c:\users\userName\.cppan\stroage 目錄下搜尋 leptonica
可以找到 C:\Users\userName\.cppan\storage\src\8f\a3\90d7\src 目錄

拷貝所有檔案至 C:\Program Files\tesseract\include\leptonica

VS2013 不能編譯, VS2017 才可以

Property Pages/Platform 選 x64

Property Pages/Configuration 選 Debug

Property Pages/Configuration Properties/Debugging/Environment

PATH=%PATH%;D:\TensorFlow\OCR\tesseract\win64\bin\Debug

Property Pages/Configuration Properties/C/C++/General/Additional Include Directories

新增 C:\Program Files\tesseract\include

Property Pages/Configuration Properties/Linker/General/Additional Library Directories

新增 D:\TensorFlow\OCR\tesseract\win64\Debug

Property Pages/Configuration Properties/Linker/Input/Additional Dependencies

新增 tesseract40d.lib

新增 pvt.cppan.demo.danbloomberg.leptonica-1.76.0.lib

Property Pages/Platform 選 x64

Property Pages/Configuration 選 Release

Property Pages/Configuration Properties/Debugging/Environment

PATH=%PATH%;C:\Program Files\tesseract\bin

Property Pages/Configuration Properties/C/C++/General/Additional Include Directories

新增 C:\Program Files\tesseract\include

Property Pages/Configuration Properties/Linker/General/Additional Library Directories

新增 C:\Program Files\tesseract\lib

Property Pages/Configuration Properties/Linker/Input/Additional Dependencies

新增 tesseract40.lib

新增 pvt.cppan.demo.danbloomberg.leptonica-1.76.0.lib

執行 Debug 程式時，若是使用 Release 的 lib, delete GetUTF8Text() 產生的記憶體
會產生 Exception

Tesseract OCR

https://digi.bib.uni-mannheim.de/tesseract/
可以下載安裝版
雖然它只可以執行，不能開發程式，但還是先安裝，因為要使用它的 tessdata
等用完再移除吧
https://github.com/UB-Mannheim/tesseract/wiki/Windows-build
有一些安裝檔如何產生的說明，但它是利用 Linux 跨平台編譯產生的

使用 Vcpkg
開啟 PowerShell
git clone https://github.com/Microsoft/vcpkg.git vcpkg
cd vcpkg
.\bootstrap-vcpkg.bat
產生 vcpkg.exe
.\vcpkg install tesseract:x64-windows
產生 installed\x64-windows\tools\tesseract
.\vcpkg install tesseract:x64-windows-static
產生 installed\x64-windows-static\tools\tesseract
.\vcpkg install tesseract:x86-windows-static
有 include, dll, lib, 但卻是 3.05 版

使用 cmake, cppan, vs2017
原先使用之前的 cmake(3.10版), 一直失敗, 更新成 cmake(3.12版)才成功
下載 cppan
cppan 會使用 c:\users\userName\.cppan 目錄，若有失敗要重新開始，刪除這個目錄
設定 PATH 到 cmake 和 cppan
開啟 PowerShell

git clone https://github.com/tesseract-ocr/tesseract tesseract

cd tesseract
mkdir win64
cd win64
PS D:\Tesseract\tesseract\win64> $env:Path += ";D:\Tesseract\cppan-master-Windows-client;C:\Program Files\CMake\bin"
PS D:\Tesseract\tesseract\win64> $env:path.split(";")
cppan ..
cmake .. -G "Visual Studio 15 2017 Win64"
開啟 vs2017

開啟 tesseract\win64\tesseract.sln

先編譯 "CPPAN Targets/Service/cppan-d-b-d" 專案，會產生錯誤

最主要為程式內含有錯誤的字元

開啟這些檔案，另存新檔，選擇 Save 旁邊的小按鈕，選擇 Save with encoding

Encoding 選擇 Unicode (UTF-8 with signature)

ALL_BUILD 可以成功，接著 build INSTALL

此時會產生 MSB307 setlocal 錯誤

主要是因為沒有權限安裝程式到 C:\Program Files\tesseract

使用 Administrator 身分重新開啟 vs2017

重新 build 即可

增加中文字(含手寫)的支援

到 https://github.com/tesseract-ocr/tessdata 下載 tessdata

但是我不知道要下載那些檔案，乾脆使用安裝檔內的 tessdata

設定環境變數 TESSDATA_PREFIX=C:\Program Files\tesseract\tessdata

---

發現在部分電腦上速度會非常慢，可關閉 openmp 改善
修改 project libtesseract 和 tesseract 的 property
C/C++/Language/Open MP Support: No(/openmp-)