CentOS7 TensorFlowビルド 詳細
Table of Contents
情報収集
Build from source
- Build from source
- Linux
- Version Python version Compiler Build tools
- tensorflow-1.12.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.15.0
- tensorflow-1.11.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.15.0
- tensorflow-1.10.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.15.0
- tensorflow-1.9.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.11.0
- tensorflow-1.8.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.10.0
- tensorflow-1.7.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.10.0
- tensorflow-1.6.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.9.0
- tensorflow-1.5.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.8.0
- tensorflow-1.4.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.5.4
- tensorflow-1.3.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.4.5
ビルド手順
configure
$ ./configure
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.19.2 installed.
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python3
Found possible Python library paths:
/usr/lib64/python3.4/site-packages
/usr/lib/python3.4/site-packages
Please input the desired Python library path to use. Default is [/usr/lib64/python3.4/site-packages]
Do you wish to build TensorFlow with XLA JIT support? [Y/n]:
XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]:
No CUDA support will be enabled for TensorFlow.
Do you wish to download a fresh release of clang? (Experimental) [y/N]:
Clang will not be downloaded.
Do you wish to build TensorFlow with MPI support? [y/N]:
No MPI support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl # Build with MKL support.
--config=monolithic # Config for mostly static monolithic build.
--config=gdr # Build with GDR support.
--config=verbs # Build with libverbs support.
--config=ngraph # Build with Intel nGraph support.
--config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws # Disable AWS S3 filesystem support.
--config=nogcp # Disable GCP support.
--config=nohdfs # Disable HDFS support.
--config=noignite # Disable Apacha Ignite support.
--config=nokafka # Disable Apache Kafka support.
--config=nonccl # Disable NVIDIA NCCL support.
Configuration finished
パッケージ作成
インストールするためのパッケージを作成
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg 2019年 3月 3日 日曜日 11:13:11 JST : === Preparing sources in dir: /tmp/tmp.C5E9NtSeox ~/source/tensorflow ~/source/tensorflow ~/source/tensorflow 2019年 3月 3日 日曜日 11:13:23 JST : === Building wheel warning: no files found matching '*.pyd' under directory '*' warning: no files found matching '*.pd' under directory '*' warning: no files found matching '*.dll' under directory '*' warning: no files found matching '*.lib' under directory '*' warning: no files found matching '*.h' under directory 'tensorflow/include/tensorflow' warning: no files found matching '*' under directory 'tensorflow/include/Eigen' warning: no files found matching '*.h' under directory 'tensorflow/include/google' warning: no files found matching '*' under directory 'tensorflow/include/third_party' warning: no files found matching '*' under directory 'tensorflow/include/unsupported' 2019年 3月 3日 日曜日 11:13:57 JST : === Output wheel file is in: /tmp/tensorflow_pkg
インストール済のものをアンインストール
$ sudo /usr/local/bin/pip3 uninstall tensorflow
DEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be the last one supporting it. Please upgrade your Python as Python 3.4 won't be maintained after March 2019 (cf PEP 429).
Uninstalling tensorflow-1.13.1:
Would remove:
/usr/bin/freeze_graph
/usr/bin/saved_model_cli
/usr/bin/tensorboard
/usr/bin/tf_upgrade_v2
/usr/bin/tflite_convert
/usr/bin/toco
/usr/bin/toco_from_protos
/usr/lib/python3.4/site-packages/tensorflow/*
/usr/lib64/python3.4/site-packages/tensorflow-1.13.1.dist-info/*
Proceed (y/n)? y
Successfully uninstalled tensorflow-1.13.1
ビルドしたパッケージをインストール
依存関係クリアしインストールできた。
$ sudo /usr/local/bin/pip3 install /tmp/tensorflow_pkg/tensorflow-1.13.1-cp34-cp34m-linux_x86_64.whl DEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be the last one supporting it. Please upgrade your Python as Python 3.4 won't be maintained after March 2019 (cf PEP 429). Processing ./tensorflow-1.13.1-cp34-cp34m-linux_x86_64.whl Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/lib64/python3.4/site-packages (from tensorflow==1.13.1) (1.0.9) Requirement already satisfied: six>=1.10.0 in /usr/lib/python3.4/site-packages (from tensorflow==1.13.1) (1.10.0) Requirement already satisfied: tensorboard<1.14.0,>=1.13.0 in /usr/lib/python3.4/site-packages (from tensorflow==1.13.1) (1.13.0) Requirement already satisfied: tensorflow-estimator<1.14.0rc0,>=1.13.0 in /usr/lib/python3.4/site-packages (from tensorflow==1.13.1) (1.13.0) Requirement already satisfied: wheel>=0.26 in /usr/lib/python3.4/site-packages (from tensorflow==1.13.1) (0.29.0) Requirement already satisfied: protobuf>=3.6.1 in /usr/lib64/python3.4/site-packages (from tensorflow==1.13.1) (3.6.1) Requirement already satisfied: gast>=0.2.0 in /usr/lib/python3.4/site-packages (from tensorflow==1.13.1) (0.2.2) Requirement already satisfied: keras-applications>=1.0.6 in /usr/lib64/python3.4/site-packages (from tensorflow==1.13.1) (1.0.7) Requirement already satisfied: astor>=0.6.0 in /usr/lib/python3.4/site-packages (from tensorflow==1.13.1) (0.7.1) Requirement already satisfied: termcolor>=1.1.0 in /usr/lib/python3.4/site-packages (from tensorflow==1.13.1) (1.1.0) Requirement already satisfied: absl-py>=0.1.6 in /usr/lib/python3.4/site-packages (from tensorflow==1.13.1) (0.7.0) Requirement already satisfied: numpy>=1.13.3 in /usr/lib64/python3.4/site-packages (from tensorflow==1.13.1) (1.16.2) Requirement already satisfied: grpcio>=1.8.6 in /usr/lib64/python3.4/site-packages (from tensorflow==1.13.1) (1.19.0) Requirement already satisfied: werkzeug>=0.11.15 in /usr/lib64/python3.4/site-packages (from tensorboard<1.14.0,>=1.13.0->tensorflow==1.13.1) (0.14.1) Requirement already satisfied: markdown>=2.6.8 in /usr/lib64/python3.4/site-packages (from tensorboard<1.14.0,>=1.13.0->tensorflow==1.13.1) (3.0.1) Requirement already satisfied: mock>=2.0.0 in /usr/lib/python3.4/site-packages (from tensorflow-estimator<1.14.0rc0,>=1.13.0->tensorflow==1.13.1) (2.0.0) Requirement already satisfied: setuptools in /usr/lib/python3.4/site-packages (from protobuf>=3.6.1->tensorflow==1.13.1) (36.2.0) Requirement already satisfied: h5py in /usr/lib64/python3.4/site-packages (from keras-applications>=1.0.6->tensorflow==1.13.1) (2.9.0) Requirement already satisfied: pbr>=0.11 in /usr/lib/python3.4/site-packages (from mock>=2.0.0->tensorflow-estimator<1.14.0rc0,>=1.13.0->tensorflow==1.13.1) (3.1.1) Installing collected packages: tensorflow Successfully installed tensorflow-1.13.1
躓いたところ
ビルドエラー 'cc1plus'
$ /usr/local/bin/bazel build --local_resources=2048,8,1.0 --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package
WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.build/docs/bazel-user-manual.html#sandboxing for more information
WARNING: /home/wurly/source/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated targd. Switch to SavedModel immediately.
WARNING: /home/wurly/source/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated targtch to SavedModel immediately.
INFO: Found 1 target...
ERROR: /home/wurly/.cache/bazel/_bazel_wurly/86f429c2e3e66e831dddc7b7dd84faef/external/protobuf/BUILD:263:1: C++ compilation of rule '@protobuf//:js_embed' failed: gcc failed: error executing comma
(cd /home/wurly/.cache/bazel/_bazel_wurly/86f429c2e3e66e831dddc7b7dd84faef/execroot/tensorflow && \
exec env - \
PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/wurly/.local/bin:/home/wurly/bin \
/usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffost/bin/external/protobuf/_objs/js_embed/external/protobuf/src/google/protobuf/compiler/js/embed.d '-frandom-seed=bazel-out/host/bin/external/protobuf/_objs/js_embed/external/protobuf/src/google/prut/host/genfiles/external/protobuf -iquote external/bazel_tools -iquote bazel-out/host/genfiles/external/bazel_tools -isystem external/bazel_tools/tools/cpp/gcc3 -fno-canonical-system-headers -Wno-acted"' '-D__TIME__="redacted"' -c external/protobuf/src/google/protobuf/compiler/js/embed.cc -o bazel-out/host/bin/external/protobuf/_objs/js_embed/external/protobuf/src/google/protobuf/compiler/jn: Process exited with status 1.
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 15.511s, Critical Path: 0.26s
gcc-c++ をインストールすることで解決
$ sudo yum install gcc-c++ 読み込んだプラグイン:fastestmirror, langpacks Repository google-chrome is listed more than once in the configuration Loading mirror speeds from cached hostfile * base: mirror.0x.sg * epel: my.fedora.ipserverone.com * extras: mirror.0x.sg * ius: mirrors.kernel.org * updates: mirror.0x.sg gitlab_gitlab-ce/x86_64/signature gitlab_gitlab-ce/x86_64/signature gitlab_gitlab-ce-source/signature gitlab_gitlab-ce-source/signature 依存性の解決をしています --> トランザクションの確認を実行しています。 ---> パッケージ gcc-c++.x86_64 0:4.8.5-36.el7 を インストール --> 依存性の処理をしています: libstdc++-devel = 4.8.5-36.el7 のパッケージ: gcc-c++-4.8.5-36.el7.x86_64 --> トランザクションの確認を実行しています。 ---> パッケージ libstdc++-devel.x86_64 0:4.8.5-36.el7 を インストール --> 依存性解決を終了しました。 依存性を解決しました ===================================================================================================================================================================================================== Package アーキテクチャー バージョン ===================================================================================================================================================================================================== インストール中: gcc-c++ x86_64 4.8.5-36.el7 依存性関連でのインストールをします: libstdc++-devel x86_64 4.8.5-36.el7 トランザクションの要約 ===================================================================================================================================================================================================== インストール 1 パッケージ (+1 個の依存関係のパッケージ) 総ダウンロード容量: 8.7 M インストール容量: 25 M Is this ok [y/d/N]: y Downloading packages: (1/2): libstdc++-devel-4.8.5-36.el7.x86_64.rpm (2/2): gcc-c++-4.8.5-36.el7.x86_64.rpm ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 合計 Running transaction check Running transaction test Transaction test succeeded Running transaction インストール中 : libstdc++-devel-4.8.5-36.el7.x86_64 インストール中 : gcc-c++-4.8.5-36.el7.x86_64 検証中 : gcc-c++-4.8.5-36.el7.x86_64 検証中 : libstdc++-devel-4.8.5-36.el7.x86_64 インストール: gcc-c++.x86_64 0:4.8.5-36.el7 依存性関連をインストールしました: libstdc++-devel.x86_64 0:4.8.5-36.el7 完了しました!
ビルドエラー this rule is missing dependency
$ /usr/local/bin/bazel build --local_resources=2048,8,1.0 --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.build/docs/bazel-user-manual.html#sandboxing for more information WARNING: /home/wurly/source/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated targd. Switch to SavedModel immediately. WARNING: /home/wurly/source/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated targtch to SavedModel immediately. INFO: Found 1 target... ERROR: /home/wurly/.cache/bazel/_bazel_wurly/86f429c2e3e66e831dddc7b7dd84faef/external/nasm/BUILD.bazel:8:1: undeclared inclusion(s) in rule '@nasm//:nasm': this rule is missing dependency declarations for the following files included by 'external/nasm/regvals.c': '/usr/include/stdc-predef.h' '/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/stddef.h' '/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/stdarg.h' '/usr/include/stdio.h' '/usr/include/features.h' '/usr/include/sys/cdefs.h' '/usr/include/bits/wordsize.h' '/usr/include/gnu/stubs.h' '/usr/include/gnu/stubs-64.h' '/usr/include/bits/types.h' '/usr/include/bits/typesizes.h' '/usr/include/libio.h' '/usr/include/_G_config.h' '/usr/include/wchar.h' '/usr/include/bits/stdio_lim.h' '/usr/include/bits/sys_errlist.h' '/usr/include/bits/stdio.h' '/usr/include/bits/stdio2.h' '/usr/include/inttypes.h' '/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/stdint.h' '/usr/include/stdint.h' '/usr/include/bits/wchar.h'. Target //tensorflow/tools/pip_package:build_pip_package failed to build
上記を参考に、下記のcleanを行うことで解決
$ /usr/local/bin/bazel clean --expunge