參考
Building Tensorflow 1.13 on Jetson Xavier
安裝 bazel
nvidia@jetson-0423418048807:~/XavierSSD/Downloads$ wget https://github.com/bazelbuild/bazel/releases/download/0.19.2/bazel-0.19.2-dist.zip
nvidia@jetson-0423418048807:~/XavierSSD/Downloads$ mkdir bazel
nvidia@jetson-0423418048807:~/XavierSSD/Downloads$ cd bazel
nvidia@jetson-0423418048807:~/XavierSSD/Downloads/bazel$ unzip bazel-0.19.2-dist.zip
nvidia@jetson-0423418048807:~/XavierSSD/Downloads/bazel$ env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh
nvidia@jetson-0423418048807:~/XavierSSD/Downloads/bazel$ cd ..
nvidia@jetson-0423418048807:~/XavierSSD/Downloads$ mv bazel ~/XavierSSD
nvidia@jetson-0423418048807:~/XavierSSD/Downloads$ cd ../bazel/
nvidia@jetson-0423418048807:~/XavierSSD/bazel$ vi ~/.bashrc
加入下一行到檔案底部，並且也執行一遍
export PATH=~/XavierSSD/bazel/output${PATH:+:${PATH}}
下載 tensorflow
nvidia@jetson-0423418048807:~/XavierSSD/bazel$ cd ..
nvidia@jetson-0423418048807:~/XavierSSD$ git clone https://github.com/tensorflow/tensorflow.git
nvidia@jetson-0423418048807:~/XavierSSD$ cd tensorflow/
設定 git 環境，取得 r1.13 版
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git config --global user.email "name@yahoo.com.tw"
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git config --global user.name "name"
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git checkout r1.13
為 Nvidia Jetson AGX Xavier 修改
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ vi tensorflow/lite/kernels/internal/BUILD
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git add tensorflow/lite/kernels/internal/BUILD
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git commit -m "Update 1"
[r1.13 982e077b2a] Update 1
1 file changed, 3 deletions(-)
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git log
commit 982e077b2a4e2123f7a299dbaf95d97383303d17 (HEAD -> r1.13)
Author: name <name@yahoo.com.tw>
Date: Mon Jun 24 14:49:30 2019 +0800
Update 1
commit 93dd14dce2e8751bcaab0a0eb363d55eb0cc5813 (origin/r1.13)
Author: Mihai Maruseac <mihaimaruseac@google.com>
Date: Tue May 21 10:08:18 2019 -0700
Update png_archive version to 1.6.37
PiperOrigin-RevId: 249272809
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git diff 93dd1 982e0
diff --git a/tensorflow/lite/kernels/internal/BUILD b/tensorflow/lite/kernels/internal/BUILD
index 4be3226938..7226f96fdf 100644
--- a/tensorflow/lite/kernels/internal/BUILD
+++ b/tensorflow/lite/kernels/internal/BUILD
@@ -22,15 +22,12 @@ HARD_FP_FLAGS_IF_APPLICABLE = select({
NEON_FLAGS_IF_APPLICABLE = select({
":arm": [
"-O3",
- "-mfpu=neon",
],
":armeabi-v7a": [
"-O3",
- "-mfpu=neon",
],
":armv7a": [
"-O3",
- "-mfpu=neon",
],
"//conditions:default": [
"-O3",
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ vi third_party/aws/BUILD.bazel
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git add third_party/aws/BUILD.bazel
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git commit -m "Update 2"
[r1.13 a3d6ea2fce] Update 2
1 file changed, 1 insertion(+), 1 deletion(-)
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git log
commit a3d6ea2fce8fff7bcf74ee52cd77074416d24bf2 (HEAD -> r1.13)
Author: mark <ingrenn@yahoo.com.tw>
Date: Mon Jun 24 14:56:09 2019 +0800
Update 2
commit 982e077b2a4e2123f7a299dbaf95d97383303d17
Author: mark <ingrenn@yahoo.com.tw>
Date: Mon Jun 24 14:49:30 2019 +0800
Update 1
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git diff 982e0 a3d6ea
diff --git a/third_party/aws/BUILD.bazel b/third_party/aws/BUILD.bazel
index 5426f79e46..e08f8fc108 100644
--- a/third_party/aws/BUILD.bazel
+++ b/third_party/aws/BUILD.bazel
@@ -24,7 +24,7 @@ cc_library(
"@org_tensorflow//tensorflow:raspberry_pi_armeabi": glob([
"aws-cpp-sdk-core/source/platform/linux-shared/*.cpp",
]),
- "//conditions:default": [],
+ "//conditions:default": glob(["aws-cpp-sdk-core/source/platform/linux-shared/*.cpp",]),
}) + glob([
"aws-cpp-sdk-core/include/**/*.h",
"aws-cpp-sdk-core/source/*.cpp",
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ vi third_party/gpus/crosstool/BUILD.tpl
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git add third_party/gpus/cro
sstool/BUILD.tpl
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git commit -m "Update 3"
[r1.13 65ad3b64e5] Update 3
1 file changed, 1 insertion(+)
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git log
commit 65ad3b64e5f16b3496628bee800fabf825a7c1ce (HEAD -> r1.13)
Author: mark <ingrenn@yahoo.com.tw>
Date: Mon Jun 24 15:04:22 2019 +0800
Update 3
commit a3d6ea2fce8fff7bcf74ee52cd77074416d24bf2
Author: mark <ingrenn@yahoo.com.tw>
Date: Mon Jun 24 14:56:09 2019 +0800
Update 2
commit 982e077b2a4e2123f7a299dbaf95d97383303d17
Author: mark <ingrenn@yahoo.com.tw>
Date: Mon Jun 24 14:49:30 2019 +0800
Update 1
commit 93dd14dce2e8751bcaab0a0eb363d55eb0cc5813 (origin/r1.13)
Author: Mihai Maruseac <mihaimaruseac@google.com>
Date: Tue May 21 10:08:18 2019 -0700
Update png_archive version to 1.6.37
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ git diff a3d6ea 65ad3
diff --git a/third_party/gpus/crosstool/BUILD.tpl b/third_party/gpus/crosstool/BUILD.tpl
index db76306ffb..184cd35b87 100644
--- a/third_party/gpus/crosstool/BUILD.tpl
+++ b/third_party/gpus/crosstool/BUILD.tpl
@@ -24,6 +24,7 @@ cc_toolchain_suite(
"x64_windows|msvc-cl": ":cc-compiler-windows",
"x64_windows": ":cc-compiler-windows",
"arm": ":cc-compiler-local",
+ "aarch64": ":cc-compiler-local",
"k8": ":cc-compiler-local",
"piii": ":cc-compiler-local",
"ppc": ":cc-compiler-local",
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
修改完成
安裝所需版本的 g++ 和 gcc
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ sudo apt-get install g++-5
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ sudo apt-get install gcc-5
設定編譯環境
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ ./configure
Extracting Bazel installation...
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.19.2- (@non-git) installed.
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python3
Found possible Python library paths:
/usr/lib/python3.6/dist-packages
/usr/lib/python3/dist-packages
/usr/local/lib/python3.6/dist-packages
Please input the desired Python library path to use. Default is [/usr/lib/python3.6/dist-packages]
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: n
No XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.
Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 10.0]:
Please specify the location where CUDA 10.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/local/cuda-10.0
Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7]: 7.3
Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda-10.0]: /usr/lib/aarch64-linux-gnu
Do you wish to build TensorFlow with TensorRT support? [y/N]: y
TensorRT support will be enabled for TensorFlow.
Please specify the location where TensorRT is installed. [Default is /usr/lib/aarch64-linux-gnu]:
Please specify the locally installed NCCL version you want to use. [Default is to use https://github.com/nvidia/nccl]:
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 3.5,7.0]: 7.2
Do you want to use clang as CUDA compiler? [y/N]: n
nvcc will be used as CUDA compiler.
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: /usr/bin/gcc-5
Do you wish to build TensorFlow with MPI support? [y/N]: n
No MPI support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl # Build with MKL support.
--config=monolithic # Config for mostly static monolithic build.
--config=gdr # Build with GDR support.
--config=verbs # Build with libverbs support.
--config=ngraph # Build with Intel nGraph support.
--config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws # Disable AWS S3 filesystem support.
--config=nogcp # Disable GCP support.
--config=nohdfs # Disable HDFS support.
--config=noignite # Disable Apacha Ignite support.
--config=nokafka # Disable Apache Kafka support.
--config=nonccl # Disable NVIDIA NCCL support.
Configuration finished
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
建立 tensorflow 安裝資料
此處會執行很久，甚至會報錯，如找不到 numpy 等
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ bazel build --config=opt --config=nonccl //tensorflow/tools/pip_package:build_pip_package --incompatible_remove_native_http_archive=false --verbose_failures --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"
產生 tensorflow-1.13.1-cp36-cp36m-linux_aarch64.whl
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ sudo bazel-bin/tensorflow/tools/pip_package/build_pip_package ../
移除舊的 tensorflow，並安裝新的
nvidia@jetson-0423418048807:~$ source XavierSSD/envs/tensorflow/bin/activate
(tensorflow) nvidia@jetson-0423418048807:~$ cd XavierSSD/tensorflow/
((tensorflow) nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ pip3 uninstall tensorflow-gpu
(tensorflow) nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ pip3 install ../tensorflow-1.13.1-cp36-cp36m-linux_aarch64.whl
(tensorflow) nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ deactivate
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
建立 tensorflow c++ 的 shared library libtensorflow_cc.so
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ bazel build --config=opt --config=nonccl //tensorflow:libtensorflow_cc.so --incompatible_remove_native_http_archive=false --verbose_failures --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ ls -al bazel-bin/tensorflow/
libtensorflow_cc.so
-r-xr-xr-x 1 nvidia nvidia 303026864 Jun 25 12:22 bazel-bin/tensorflow/libtensorflow_cc.so
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ mkdir tensorflow/cc/example
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ vi tensorflow/cc/example/example.cc
// tensorflow/cc/example/example.cc
#include "tensorflow/cc/client/client_session.h"
#include "tensorflow/cc/ops/standard_ops.h"
#include "tensorflow/core/framework/tensor.h"
int main() {
using namespace tensorflow;
using namespace tensorflow::ops;
Scope root = Scope::NewRootScope();
// Matrix A = [3 2; -1 0]
auto A = Const(root, { {3.f, 2.f}, {-1.f, 0.f} });
// Vector b = [3 5]
auto b = Const(root, { {3.f, 5.f} });
// v = Ab^T
auto v = MatMul(root.WithOpName("v"), A, b, MatMul::TransposeB(true));
std::vector<Tensor> outputs;
ClientSession session(root);
// Run and fetch v
TF_CHECK_OK(session.Run({v}, &outputs));
// Expect outputs[0] == [19; -3]
LOG(INFO) << outputs[0].matrix<float>();
return 0;
}
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ vi tensorflow/cc/example/BUILD
load("//tensorflow:tensorflow.bzl", "tf_cc_binary")
tf_cc_binary(
name = "example",
srcs = ["example.cc"],
deps = [
"//tensorflow/cc:cc_ops",
"//tensorflow/cc:client_session",
"//tensorflow/core:tensorflow",
],
)
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
編譯範例 example.cc
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ bazel build -c opt //tensorflow/cc/example:example
跑了好久，測試執行
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$ bazel-bin/tensorflow/cc/exam
ple/example
2019-06-25 16:10:03.922559: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:965] ARM64 does not support NUMA - returning NUMA node zero
2019-06-25 16:10:03.922959: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties:
name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.5
pciBusID: 0000:00:00.0
totalMemory: 15.45GiB freeMemory: 8.57GiB
2019-06-25 16:10:03.923054: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0
2019-06-25 16:10:03.924346: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-06-25 16:10:03.924409: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990] 0
2019-06-25 16:10:03.924452: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0: N
2019-06-25 16:10:03.925152: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8340 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2)
2019-06-25 16:10:07.582973: I tensorflow/cc/example/example.cc:22] 19
-3
nvidia@jetson-0423418048807:~/XavierSSD/tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD$ git clone https://github.com/bitbionic/
keras-to-tensorflow.git
Cloning into 'keras-to-tensorflow'...
remote: Enumerating objects: 3719, done.
remote: Total 3719 (delta 0), reused 0 (delta 0), pack-reused 3719
Receiving objects: 100% (3719/3719), 227.81 MiB | 786.00 KiB/s, done.
Resolving deltas: 100% (7/7), done.
Checking out files: 100% (3688/3688), done.
nvidia@jetson-0423418048807:~/XavierSSD$
nvidia@jetson-0423418048807:~/XavierSSD$ cd keras-to-tensorflow
nvidia@jetson-0423418048807:~/XavierSSD/keras-to-tensorflow$ vi main.c
因為編譯時會有兩種錯誤
data.ToString(); 改成 std::string(data);
tensorflow::StringPiece(file_name).ends_with(".png")
改成
tensorflow::str_util::EndsWith(file_name, ".png")
nvidia@jetson-0423418048807:~/XavierSSD/keras-to-tensorflow$
nvidia@jetson-0423418048807:~/XavierSSD/keras-to-tensorflow$ g++-5 -std=gnu++11 -c ./main.cpp -D_GLIBCXX_USE_CXX11_ABI=0 -I../tensorflow -I../tensorflow/bazel-tensorflow/external/eigen_archive -I../tensorflow/bazel-tensorflow/external/protobuf_archive/src -I../tensorflow/bazel-tensorflow/external/com_google_absl -I../tensorflow/bazel-genfiles
