網頁

2024年10月16日 星期三

Jetson Container 使用紀錄,二

因為這次更新到 Jetpack 6.0, 所以測試得很順利

參考 https://jetsonhacks.com/2023/08/07/speech-ai-on-nvidia-jetson-tutorial/
註冊 NGC
生成 API 密鑰
$ ngc config set
$ ngc registry resource download-version nvidia/riva/riva_quickstart_arm64:2.17.0
$ cd riva_quickstart
$ vi config.sh
service_enabled_nlp=false
service_enabled_nmt=false
asr_language_code=("en-US")
#asr_language_code=("zh-CN")
tts_language_code=("en-US")
#tts_language_code=("zh-CN")
$ sudo bash riva_init.sh
$ sudo bash riva_start.sh
安裝 https://github.com/nvidia-riva/python-clients.git

### 文字介面
$ jetson-containers run --env HUGGINGFACE_TOKEN=hf_abc123def \
  $(./autotag nano_llm) \
  python3 -m nano_llm.chat --api=mlc \
    --model meta-llama/Llama-2-7b-chat-hf \
    --quantization q4f16_ft

### llamaspeak
$ jetson-containers run --env HUGGINGFACE_TOKEN=hf_xyz123abc456 \
  $(autotag nano_llm) \
  python3 -m nano_llm.agents.web_chat --api=mlc \
    --model meta-llama/Meta-Llama-3-8B-Instruct \
    --asr=riva --language-code=en-US --tts=piper \
    --voice=en_US-libritts-high --web-port 8050

$ jetson-containers run --env HUGGINGFACE_TOKEN=hf_xyz123abc456 \
  $(autotag nano_llm) \
  python3 -m nano_llm.agents.web_chat --api=mlc \
    --model meta-llama/Meta-Llama-3-8B-Instruct \
    --asr=riva --language-code=zh-CN --tts=piper \
    --voice=zh_CN-huayan-x_low --web-port 8050

參考 https://ithelp.ithome.com.tw/articles/10347557
輸入的中文識別:修改/opt/NanoLLM/nano_llm/plugins/speech/riva_asr.py,將裡面的language_code=從en-US改成zh-CN
輸出的中文語音:修改/opt/NanoLLM/nano_llm/plugins/speech/piper_tts.py,將裡面的en_US-libritts-high模型(有兩處)改成zh_CN-huayan-medium模型。這裡可以選擇的中文模型,可以在/data/models/piper/voices.json裡找到,請用關鍵字zh-CN搜索,會找到zh_CN-huayan-medium與zh_CN-huayan-x_low兩組可用的中文語言

還須設定 ws-port(49000) 的 nat port 轉換
can you describe the image?

##Multimodality
$ jetson-containers run $(autotag nano_llm) \
  python3 -m nano_llm.agents.web_chat --api=mlc \
    --model Efficient-Large-Model/VILA-7b \
    --asr=riva --tts=piper

### Agent Studio
# python3 -m nano_llm.studio
https://IP_ADDRESS:8050

### text-generation-webui
$ jetson-containers run $(autotag text-generation-webui)
http://<IP_ADDRESS>:7860

中文模型
ckip-joint_bloom-3b-zh
參考  https://www.atyun.com/models/info/ckip-joint/bloom-3b-zh.html
taide_Llama3-TAIDE-LX-8B-Chat-Alpha1-4bit
參考  https://taide.tw/index
yentinglin_Llama-3-Taiwan-8B-Instruct
參考  https://medium.com/@simon3458/project-tame-llama-3-taiwan-1b249b88ab67

### ollama (Open WebUI)
$ jetson-containers run --name ollama $(autotag ollama)
# /bin/ollama run llama3
$ docker run -it --rm --network=host --add-host=host.docker.internal:host-gateway ghcr.io/open-webui/open-webui:main
http://JETSON_IP:8080

### Stable Diffusion
$ jetson-containers run $(autotag stable-diffusion-webui)
http://<IP_ADDRESS>:7860
txt2img: a person sitting at an office desk working
img2img: 先放一張圖
1. 右上 Generate 下方有 Interrogate CLIP 和 Interrogate DeepBooru 兩按鈕,可讀取圖片資訊
2. 可修改讀取出的資訊,按 Generate, 重新產生圖
3. 選 Resize and fill, 並配合不同的 Denising strength, 按 Generate, 重新產生圖
4. 選 Just resize, 並配合不同的 Denising strength, 按 Generate, 重新產生圖
5. Inpaint: 在 prompt 上填入 sunglasses, 將圖片上的眼睛位置塗黑, 按 Generate, 重新產生圖

### Stable Diffusion XL
$ CONTAINERS_DIR=`pwd`
$ MODEL_DIR=$CONTAINERS_DIR/data/models/stable-diffusion/models/Stable-diffusion/
$ sudo chown -R $USER $MODEL_DIR
$ wget -P $MODEL_DIR https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors
$ wget -P $MODEL_DIR https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/resolve/main/sd_xl_refiner_1.0.safetensors
左上角 Stable Diffusion checkpoint 選擇 sd_xl_base_1.0.safetensors
Generation tab 點選 Refiner 選擇 sd_xl_refiner_1.0.safetensors
改 width/height 到 1024
txt2img: photograph of a friendly robot alongside a person climbing a mountain 

### Jetson Copilot
git clone https://github.com/NVIDIA-AI-IOT/jetson-copilot/
cd jetson-copilot
./setup_environment.sh
./launch_jetson_copilot.sh
http://JETSON_IP:8501


$ virtualenv --system-site-packages tensorrt

$ docker ps -a
$ docker kill wqerqwer

2024年9月26日 星期四

模型優化之指派問題(linear sum assignment problem)

參考 匈牙利算法
python 使用函數
scipy.optimize.linear_sum_assignment
lap.lapjv


MMTracking 學習紀錄

參考 https://mmtracking.readthedocs.io/en/latest/index.html
參考 https://github.com/open-mmlab/mmtracking

$ git clone https://github.com/open-mmlab/mmtracking.git

$ docker run --gpus all -it --rm nvcr.io/nvidia/pytorch:21.05-py3
$ docker run --gpus all -it --name MMTracking nvcr.io/nvidia/pytorch:21.05-py3
$ docker start MMTracking
$ docker attach MMTracking
# <ctrl+p><ctrl+q>
$ docker attach MMTracking
$ docker stop MMTracking
$ docker rm MMTracking

$ echo $DISPLAY
$ export DISPLAY=:0
$ xhost +
$ docker run --gpus all -it --name MMTracking --shm-size=8G \
  --net=host -e DISPLAY=$DISPLAY -e XAUTHORITY=/tmp/xauth \
  -e QT_X11_NO_MITSHM=1 \
  -v /tmp/.X11-unix/:/tmp/.X11-unix \
  -v ~/.Xauthority:/tmp/xauth \
  -v /etc/localtime:/etc/localtime \
  -v /mnt/Data/MMTracking/mmtracking:/workspace/mmtracking \
  nvcr.io/nvidia/pytorch:21.05-py3

# pip install git+https://github.com/votchallenge/toolkit.git
# nvidia-smi
CUDA Version: 12.2
# pip list
opencv-python                    4.10.0.84
torch                            1.9.0a0+2ecb2c7

# pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu122/torch1.9.0/index.html
#### 出現下列錯誤
####    cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
####AttributeError: partially initialized module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline' (most likely due to a circular import)
#### 解決
# pip install opencv-python==4.5.1.48

# pip install mmdet==2.28.2
# pip install mmengine==0.10.4
# pip list
mmcv-full                        1.7.2
mmdet                            2.28.2
mmengine                         0.10.4
pillow                           10.4.0
# cd /workspace/mmtracking/
# pip install -r requirements/build.txt
# pip install -v -e .
# pip install git+https://github.com/JonathonLuiten/TrackEval.git
# pip install git+https://github.com/lvis-dataset/lvis-api.git
# pip install git+https://github.com/TAO-Dataset/tao.git

# python demo/demo_mot_vis.py configs/mot/deepsort/deepsort_faster-rcnn_fpn_4e_mot17-private-half.py \
  --input demo/demo.mp4 --output mot.mp4 --show
#### 出現下列錯誤
####    from .cv2 import *
####ImportError: libGL.so.1: cannot open shared object file: No such file or directory
#### 解決
# apt-get update
# apt-get install libgl1
#### 出現下列錯誤
####    NumpyArray = npt.NDArray[Any]
####AttributeError: module 'numpy.typing' has no attribute 'NDArray'
#### 解決
# pip install Pillow==9.5.0
#### 出現下列錯誤
#### Could not load the Qt platform plugin "xcb" in "/opt/conda/lib/python3.8/site-packages/cv2/qt/plugins" even though it was found.
####This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
#### 查詢錯誤所在
# export QT_DEBUG_PLUGINS=1
####Cannot load library /opt/conda/lib/python3.8/site-packages/cv2/qt/plugins/platforms/libqxcb.so: (libSM.so.6: cannot open shared object file: No such file or directory)
#### 解決
# apt-get install -y libsm6 libxext6 libxrender-dev
# pip install opencv-contrib-python==4.5.1.48

# wget https://download.openmmlab.com/mmtracking/mot/ocsort/mot_dataset/ocsort_yolox_x_crowdhuman_mot17-private-half_20220813_101618-fe150582.pth
# python demo/demo_mot_vis.py configs/mot/ocsort/ocsort_yolox_x_crowdhuman_mot17-private-half.py \
  --checkpoint ocsort_yolox_x_crowdhuman_mot17-private-half_20220813_101618-fe150582.pth \
  --input demo/demo.mp4 --output mot.mp4 --show

# wget https://download.openmmlab.com/mmtracking/vid/selsa/selsa_faster_rcnn_r101_dc5_1x_imagenetvid/selsa_faster_rcnn_r101_dc5_1x_imagenetvid_20201218_172724-aa961bcc.pth
# python demo/demo_vid.py configs/vid/selsa/selsa_faster_rcnn_r101_dc5_1x_imagenetvid.py \
  --checkpoint selsa_faster_rcnn_r101_dc5_1x_imagenetvid_20201218_172724-aa961bcc.pth \
  --input demo/demo.mp4 --show

# python demo/demo_sot.py configs/sot/siamese_rpn/siamese_rpn_r50_20e_lasot.py \
  --input demo/demo.mp4 --output sot.mp4 --show
#### 用滑鼠框出要追蹤的物件,按空白鍵

MMOCR 學習紀錄

參考 https://mmocr.readthedocs.io/en/dev-1.x/
參考 https://github.com/open-mmlab/mmocr

$ docker run --gpus all -it --rm nvcr.io/nvidia/pytorch:21.05-py3
$ docker run --gpus all -it --name MMOCR nvcr.io/nvidia/pytorch:21.05-py3
$ docker start MMOCR
$ docker attach MMOCR
# <ctrl+p><ctrl+q>
$ docker attach MMOCR
$ docker stop MMOCR
$ docker rm MMOCR
$ docker run --gpus all -it --name MMOCR --shm-size=8G \
  -v /mnt/Data/MMOCR/mmocr:/workspace/mmocr \
  -v /mnt/QNAP_A/ImageData/ICDAR:/mmocr/data \
  nvcr.io/nvidia/pytorch:21.05-py3

# pip install -U openmim
#### 出現下列錯誤
####ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.
####We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
#### 不用擔心

確認 https://mmocr.readthedocs.io/en/dev-1.x/get_started/install.html 底部的版本資訊
安裝正確版本套件
# mim list
# mim install mmengine==
# mim install mmengine
# mim install mmcv==2.0.1
#### 出現下列錯誤
####    cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
####AttributeError: partially initialized module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline' (most likely due to a circular import)
#### 解決
#### 因為目前 opencv-python 版本為 4.10.0.84, 降版本
# pip install opencv-python==4.5.1.48
# mim install mmcv==2.0.1
# mim install mmdet==3.1.0
# cd /workspace/mmocr/
# pip install -v -e .
# pip install opencv-python-headless==4.5.1.48
# pip install -r requirements/albu.txt
# pip install -r requirements.txt
# python tools/infer.py demo/images --det DBNet --rec CRNN --print-result \
  --save_pred --save_vis --out-dir='results/' --batch-size=2

2024年8月28日 星期三

SwinTransformer 學習紀錄

參考 https://github.com/microsoft/Swin-Transformer?tab=readme-ov-file
參考 https://github.com/SwinTransformer/Swin-Transformer-Object-Detection
參考 https://mmdetection.readthedocs.io/en/latest/get_started.html
參考 https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md
參考 https://github.com/open-mmlab/mmcv
參考 https://github.com/open-mmlab/mim?tab=readme-ov-file

Swin-Transformer 是原始版本,主要針對圖片分類
Swin-Transformer-Object-Detection 主要針對物件偵測(基於 mmdetection)
mmdetection 主要針對物件偵測,其中不只是 Swin-Transformer, 包含各式各樣先進的演算法
mmdetection 可在 mmdet/__init__.py 可以查詢到 mmcv 所需要的版本
Swin-Transformer-Object-Detection 要求使用 mmcv(1.4.0)
但 mmdetection 最後對應到 mmcv(1.4.0) 的版本為(v2.18.1)
所以以下放棄使用 Swin-Transformer-Object-Detection, 直接使用 mmdetection 的最新版本

$ docker run --gpus all -it --rm nvcr.io/nvidia/pytorch:21.05-py3
$ docker run --gpus all -it --name SwinTransformer nvcr.io/nvidia/pytorch:21.05-py3
$ docker start SwinTransformer
$ docker attach SwinTransformer
# <ctrl+p><ctrl+q>
$ docker attach SwinTransformer
$ docker stop SwinTransformer
$ docker rm SwinTransformer
$ git clone https://github.com/microsoft/Swin-Transformer.git
$ git clone https://github.com/SwinTransformer/Swin-Transformer-Object-Detection.git
$ git clone https://github.com/open-mmlab/mmdetection.git
$ docker run --gpus all -it --name SwinTransformer --shm-size=8G \
  -v /mnt/Data/SwinTransformer/Swin-Transformer:/workspace/Swin-Transformer \
  -v /mnt/Data/SwinTransformer/mmdetection:/workspace/mmdetection \
  -v /mnt/Data/SwinTransformer/Swin-Transformer-Object-Detection:/workspace/Swin-Transformer-Object-Detection \
  -v /mnt/QNAP_A/ImageData/ImageNet:/workspace/ImageNet \
  nvcr.io/nvidia/pytorch:21.05-py3

# pip install timm==0.4.12
# pip install opencv-python==4.4.0.46 termcolor==1.1.0 yacs==0.1.8 pyyaml scipy

# cd /workspace/Swin-Transformer/kernels/window_process/
# python setup.py install

# cd /workspace/Swin-Transformer/
# wget https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_tiny_patch4_window7_224.pth
# python -m torch.distributed.launch --nproc_per_node 1 --master_port 12345 main.py \
  --eval --cfg configs/swin/swin_tiny_patch4_window7_224.yaml \
  --resume swin_tiny_patch4_window7_224.pth --data-path /workspace/ImageNet \
  --batch_size=64
#### 出現下列錯誤
####  File "/opt/conda/lib/python3.8/site-packages/PIL/_typing.py", line 10, in <module>
####    NumpyArray = npt.NDArray[Any]
####AttributeError: module 'numpy.typing' has no attribute 'NDArray'
#### 解決
# pip install Pillow==9.5.0
#### 出現下列錯誤
####RuntimeError: Found 0 files in subfolders of: /workspace/ImageNet/val
####Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif,.tiff,.webp
#### 解決
$ cd /mnt/QNAP_A/ImageData/ImageNet/
$ mv val val_a ;mkdir val; mv val_a val
#### 出現下列錯誤
ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).
#### 解決 docker run 時須加上 --shm-size=8G

# cd /workspace/mmdetection
# pip install -U openmim
#### 出現下列錯誤
####ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.
####We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
#### 不用擔心

# mim list
# mim install mmengine
# mim install "mmcv>=2.0.0"
# pip install -v -e .
    # "-v" means verbose, or more output
    # "-e" means installing a project in editable mode,
    # thus any local modifications made to the code will take effect without reinstallation.
# mim download mmdet --config yolov3_mobilenetv2_8xb24-320-300e_coco --dest .
# mim list
# ls
# python demo/image_demo.py demo/demo.jpg yolov3_mobilenetv2_8xb24-320-300e_coco.py \
  --weights yolov3_mobilenetv2_320_300e_coco_20210719_215349-d18dff72.pth
#### 出現下列錯誤
####    import cv2
####  File "/opt/conda/lib/python3.8/site-packages/cv2/__init__.py", line 5, in <module>
####    from .cv2 import *
####ImportError: libGL.so.1: cannot open shared object file: No such file or directory
#### 出現下列錯誤
####    cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
####AttributeError: partially initialized module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline' (most likely due to a circular import)
#### 解決
# pip install opencv-python-headless==4.4.0.46
#### 出現下列錯誤
####    assert (mmcv_version >= digit_version(mmcv_minimum_version)
####AssertionError: MMCV==2.2.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
#### 解決
# mim install "mmcv==2.0.0rc4"

# mim download mmdet --config mask-rcnn_swin-t-p4-w7_fpn_1x_coco --dest .
# python demo/image_demo.py demo/demo.jpg mask-rcnn_swin-t-p4-w7_fpn_1x_coco.py \
  --weights mask_rcnn_swin-t-p4-w7_fpn_1x_coco_20210902_120937-9d6b7cfa.pth

Jetson Container 使用紀錄

mic-733ao@ubuntu:~/Data/AgentStudio/jetson-containers$ ./run.sh --workdir=/opt/text-generation-webui $(./autotag text-generation-webui) python3 server.py --model-dir=/data/models/text-generation-webui --listen --verbose --trust-remote-code

mic-733ao@ubuntu:~/Data/AgentStudio/jetson-containers$ ./run.sh --workdir=/opt/text-generation-webui $(./autotag text-generation-webui) /bin/bash -c 'python3 download-model.py --output=/data/models/text-generation-webui/THUDM_cogvlm2-llama3-chinese-chat-19B-int4 THUDM/cogvlm2-llama3-chinese-chat-19B-int4'

產生授權錯誤
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/taide/TAIDE-LX-7B/resolve/main/README_en.md
到 https://huggingface.co/ 網站 右上角使用者/Settings/Access Tokens
建立 Fine-grained token, 打勾所有權限
$ export HUGGINGFACE_TOKEN=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

chrome://flags/#unsafely-treat-insecure-origin-as-secure

2024年8月2日 星期五

有用的 docker 命令

$ sudo docker exec -i -t container_name /bin/bash
$ docker cp container_id:/path /host/path