MMOCR 學習紀錄

參考 https://mmocr.readthedocs.io/en/dev-1.x/

參考 https://github.com/open-mmlab/mmocr

$ docker run --gpus all -it --rm nvcr.io/nvidia/pytorch:21.05-py3

$ docker run --gpus all -it --name MMOCR nvcr.io/nvidia/pytorch:21.05-py3

$ docker start MMOCR

$ docker attach MMOCR

# <ctrl+p><ctrl+q>

$ docker attach MMOCR

$ docker stop MMOCR

$ docker rm MMOCR

$ docker run --gpus all -it --name MMOCR --shm-size=8G \

  -v /mnt/Data/MMOCR/mmocr:/workspace/mmocr \

  -v /mnt/QNAP_A/ImageData/ICDAR:/mmocr/data \

  nvcr.io/nvidia/pytorch:21.05-py3

# pip install -U openmim

#### 出現下列錯誤

####ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

####We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

#### 不用擔心

確認 https://mmocr.readthedocs.io/en/dev-1.x/get_started/install.html 底部的版本資訊

安裝正確版本套件

# mim list

# mim install mmengine==

# mim install mmengine

# mim install mmcv==2.0.1

#### 出現下列錯誤

####    cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline

####AttributeError: partially initialized module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline' (most likely due to a circular import)

#### 解決

#### 因為目前 opencv-python 版本為 4.10.0.84, 降版本

# pip install opencv-python==4.5.1.48

# mim install mmcv==2.0.1

# mim install mmdet==3.1.0

# cd /workspace/mmocr/

# pip install -v -e .

# pip install opencv-python-headless==4.5.1.48

# pip install -r requirements/albu.txt

# pip install -r requirements.txt

# python tools/infer.py demo/images --det DBNet --rec CRNN --print-result \

  --save_pred --save_vis --out-dir='results/' --batch-size=2

SwinTransformer 學習紀錄

參考 https://github.com/microsoft/Swin-Transformer?tab=readme-ov-file

參考 https://github.com/SwinTransformer/Swin-Transformer-Object-Detection

參考 https://mmdetection.readthedocs.io/en/latest/get_started.html

參考 https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md

參考 https://github.com/open-mmlab/mmcv

參考 https://github.com/open-mmlab/mim?tab=readme-ov-file

Swin-Transformer 是原始版本，主要針對圖片分類

Swin-Transformer-Object-Detection 主要針對物件偵測(基於 mmdetection)

mmdetection 主要針對物件偵測，其中不只是 Swin-Transformer, 包含各式各樣先進的演算法

mmdetection 可在 mmdet/__init__.py 可以查詢到 mmcv 所需要的版本

Swin-Transformer-Object-Detection 要求使用 mmcv(1.4.0)

但 mmdetection 最後對應到 mmcv(1.4.0) 的版本為(v2.18.1)

所以以下放棄使用 Swin-Transformer-Object-Detection, 直接使用 mmdetection 的最新版本

$ docker run --gpus all -it --rm nvcr.io/nvidia/pytorch:21.05-py3

$ docker run --gpus all -it --name SwinTransformer nvcr.io/nvidia/pytorch:21.05-py3

$ docker start SwinTransformer

$ docker attach SwinTransformer

# <ctrl+p><ctrl+q>

$ docker attach SwinTransformer

$ docker stop SwinTransformer

$ docker rm SwinTransformer

$ git clone https://github.com/microsoft/Swin-Transformer.git

$ git clone https://github.com/SwinTransformer/Swin-Transformer-Object-Detection.git

$ git clone https://github.com/open-mmlab/mmdetection.git

$ docker run --gpus all -it --name SwinTransformer --shm-size=8G \

  -v /mnt/Data/SwinTransformer/Swin-Transformer:/workspace/Swin-Transformer \

  -v /mnt/Data/SwinTransformer/mmdetection:/workspace/mmdetection \

  -v /mnt/Data/SwinTransformer/Swin-Transformer-Object-Detection:/workspace/Swin-Transformer-Object-Detection \

  -v /mnt/QNAP_A/ImageData/ImageNet:/workspace/ImageNet \

  nvcr.io/nvidia/pytorch:21.05-py3

# pip install timm==0.4.12

# pip install opencv-python==4.4.0.46 termcolor==1.1.0 yacs==0.1.8 pyyaml scipy

# cd /workspace/Swin-Transformer/kernels/window_process/

# python setup.py install

# cd /workspace/Swin-Transformer/

# wget https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_tiny_patch4_window7_224.pth

# python -m torch.distributed.launch --nproc_per_node 1 --master_port 12345 main.py \

  --eval --cfg configs/swin/swin_tiny_patch4_window7_224.yaml \

  --resume swin_tiny_patch4_window7_224.pth --data-path /workspace/ImageNet \

  --batch_size=64

#### 出現下列錯誤

####  File "/opt/conda/lib/python3.8/site-packages/PIL/_typing.py", line 10, in <module>

####    NumpyArray = npt.NDArray[Any]

####AttributeError: module 'numpy.typing' has no attribute 'NDArray'

#### 解決

# pip install Pillow==9.5.0

#### 出現下列錯誤

####RuntimeError: Found 0 files in subfolders of: /workspace/ImageNet/val

####Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif,.tiff,.webp

#### 解決

$ cd /mnt/QNAP_A/ImageData/ImageNet/

$ mv val val_a ;mkdir val; mv val_a val

#### 出現下列錯誤

ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).

#### 解決 docker run 時須加上 --shm-size=8G

# cd /workspace/mmdetection

# pip install -U openmim

#### 出現下列錯誤

####ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

####We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

#### 不用擔心

# mim list

# mim install mmengine

# mim install "mmcv>=2.0.0"

# pip install -v -e .

    # "-v" means verbose, or more output

    # "-e" means installing a project in editable mode,

    # thus any local modifications made to the code will take effect without reinstallation.

# mim download mmdet --config yolov3_mobilenetv2_8xb24-320-300e_coco --dest .

# mim list

# ls

# python demo/image_demo.py demo/demo.jpg yolov3_mobilenetv2_8xb24-320-300e_coco.py \

  --weights yolov3_mobilenetv2_320_300e_coco_20210719_215349-d18dff72.pth

#### 出現下列錯誤

####    import cv2

####  File "/opt/conda/lib/python3.8/site-packages/cv2/__init__.py", line 5, in <module>

####    from .cv2 import *

####ImportError: libGL.so.1: cannot open shared object file: No such file or directory

#### 出現下列錯誤

####    cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline

####AttributeError: partially initialized module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline' (most likely due to a circular import)

#### 解決

# pip install opencv-python-headless==4.4.0.46

#### 出現下列錯誤

####    assert (mmcv_version >= digit_version(mmcv_minimum_version)

####AssertionError: MMCV==2.2.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.

#### 解決

# mim install "mmcv==2.0.0rc4"

# mim download mmdet --config mask-rcnn_swin-t-p4-w7_fpn_1x_coco --dest .

# python demo/image_demo.py demo/demo.jpg mask-rcnn_swin-t-p4-w7_fpn_1x_coco.py \

  --weights mask_rcnn_swin-t-p4-w7_fpn_1x_coco_20210902_120937-9d6b7cfa.pth

Jetson Container 使用紀錄

mic-733ao@ubuntu:~/Data/AgentStudio/jetson-containers$ ./run.sh --workdir=/opt/text-generation-webui $(./autotag text-generation-webui) python3 server.py --model-dir=/data/models/text-generation-webui --listen --verbose --trust-remote-code

mic-733ao@ubuntu:~/Data/AgentStudio/jetson-containers$ ./run.sh --workdir=/opt/text-generation-webui $(./autotag text-generation-webui) /bin/bash -c 'python3 download-model.py --output=/data/models/text-generation-webui/THUDM_cogvlm2-llama3-chinese-chat-19B-int4 THUDM/cogvlm2-llama3-chinese-chat-19B-int4'

產生授權錯誤

raise HTTPError(http_error_msg, response=self)

requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/taide/TAIDE-LX-7B/resolve/main/README_en.md

到 https://huggingface.co/ 網站 右上角使用者/Settings/Access Tokens

建立 Fine-grained token, 打勾所有權限

$ export HUGGINGFACE_TOKEN=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

chrome://flags/#unsafely-treat-insecure-origin-as-secure

有用的 docker 命令

$ sudo docker exec -i -t container_name /bin/bash

$ docker cp container_id:/path /host/path

docker change Root Dir

$ systemctl stop docker

$ sud rsync -a /var/lib/docker/ /mnt/Data/docker-data

$ sudo vi /etc/docker/daemon.json

{

    "data-root": "/mnt/Data/docker-data",

    "runtimes": {

        "nvidia": {

            "path": "nvidia-container-runtime",

            "runtimeArgs": []

        }

    }

}

$ systemctl start docker

$ systemctl status docker

$ sudo docker info

deepstream create_pipeline.create_common_elements 分析

參考之前 https://yingrenn.blogspot.com/2023/03/deepstream-createpipeline.html

create_common_elements {

    common_elements.segvisual_bin = create_segvisual_bin

    common_elements.secondary_gie_bin = create_secondary_gie_bin {

        bin.sink - tee - queue - bin.src

        bin->sub_bins[] = create_secondary_gie {

            secondary_gie = nvinfer | nvinferserver

            if (is_parent || has_children) {

                queue

            }

            if (no_children) {

                queue - secondary_gie - fakesink

            } else {

                queue - secondary_gie - tee

            }

        }

        if (parent_index == -1) {

            tee - sub_bins[i].queue

        } else if (sub_bins[parent_index].tee) {

            sub_bins[parent_index].tee - sub_bins[i].queue

        } else {

            sub_bins[parent_index].secondary_gie - sub_bins[i].secondary_gie

        }

    }

    common_elements.secondary_preprocess_bin = create_secondary_preprocess_bin

    common_elements.dsanalytics_bin = create_dsanalytics_bin

    common_elements.tracker_bin = create_tracking_bin

    common_elements.primary_gie_bin = create_primary_gie_bin

    common_elements.preprocess_bin = create_preprocess_bin

    common_elements.msg_conv = gst_element_factory_make

    common_elements.tee = gst_element_factory_make

    

    sink_elem - preprocess_bin - primary_gie_bin - tracker_bin - dsanalytics_bin - secondary_preprocess_bin - secondary_gie_bin - segvisual_bin - msg_conv - tee - src_elem

}

  

Fine-tuning Whisper in a Google Colab

參考 https://research.google.com/colaboratory/local-runtimes.html

使得 colab 可以用 local 的 cpu 和 gpu

文件說可以使用 docker 或 jupyter

但只有 jupyter 成功

建立 huggingface 帳號，並且登入

開啟 https://huggingface.co/settings/tokens

按下 New token

選擇 Role(有 read 和 write)

按下 copy

在執行下列命令時，貼上 token

$ huggingface-cli login

    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|

    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|

    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|

    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|

    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|

    

    To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .

Token: 

Add token as git credential? (Y/n) Y

Token is valid (permission: read).

$ huggingface-cli login

    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|

    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|

    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|

    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|

    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|

    

    A token is already saved on your machine. Run `huggingface-cli whoami` to get more information or `huggingface-cli logout` if you want to log out.

    Setting a new token will erase the existing one.

    To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .

Token: 

Add token as git credential? (Y/n) Y

Token is valid (permission: write).