Jupyter Notebook
Table of Contents
概要
とりあえず Windows環境で Anaconda をインストールして使ってみた。
使ってみるまでわからなかったが、サーバを立ち上げてWeb上でPythonが書けるものであった。
これだったら、Linuxでサーバ構築して使えるようにした方がよいな、と思い、
CentOSで、マルチユーザで使用できる JupyterHub 環境を構築した。
Windows PCで使う
Windows の Anacondaでの起動
Windows で Anaconda をインストールし、スタートメニューから 「Anaconda3 (64bit)」→「Anaconda Prompt」を開いて
$ jupyter notebook
で起動
サーバ構築:CentOS7 にインストール(Python2)
サーバ構築:CentOS7 にインストール(Python3)
epel
$ sudo yum install epel-release
python34
$ sudo yum install python34 $ sudo yum install python34-devel
関連パッケージ
$ sudo yum install zlib-devel bzip2-devel openssl-devel sqlite-devel readline-devel gcc-gfortran atlas atlas-devel blas blas-devel lapack lapack-devel libpng-devel freetype-devel tk-devel
get-pip
$ wget https://bootstrap.pypa.io/get-pip.py
pipにてライブラリのインストール
$ sudo python3 get-pip.py $ sudo pip3 install numpy scipy scikit-learn pandas ipython jupyter matplotlib
pip にて jupyterhubのインストール
$ sudo pip3 install jupyterhub
この状態だと、jupyterhub を起動しても、
Failed to find proxy ['configurable-http-proxy']
と言われるので
nodejs,npmのインストール
$ sudo yum install nodejs npm
configurable-http-proxy のインストール
$ sudo npm install -g configurable-http-proxy
jupyterhubの設定
設定ファイルの作成
$ cd /opt/ $ sudo jupyterhub --generate-config $ sudo vi jupyterhub_config.py
ログイン→ログアウト後、再度ログインしようとすると認証に失敗する不具合を回避する設定
c.PAMAuthenticator.open_sessions = False
jupyterhubの設定(最終的に非採用)
c.Spawner.notebook_dir
各ユーザが ~/notebook/ディレクトリを作成し、そこをNotebookディレクトリにする。
※ただしディレクトリが存在しないとログイン時に Internal Server Error になるので注意
#c.Spawner.notebook_dir = '' c.Spawner.notebook_dir = '~/notebook'
Whitelist設定
#c.Authenticator.whitelist = set() c.Authenticator.whitelist = {'user01', 'user02'}
管理ユーザ設定
#c.Authenticator.admin_users = set() c.Authenticator.admin_users = {'user00'}
jupyterhub の立ち上げ
一時的に使うなら
$ sudo jupyterhub --no-ssl
サーバを立ち上げておくなら
$ sudo nohup jupyterhub --no-ssl &
設定等
jupyter kernelspec
$ sudo jupyter kernelspec list Available kernels: python3 /usr/share/jupyter/kernels/python3
その他のインストール(pip3)
chainer
$ sudo pip3 install chainer Collecting chainer Downloading chainer-2.0.2.tar.gz (312kB) 100% |????????????????????????????????| 317kB 2.3MB/s Collecting filelock (from chainer) Downloading filelock-2.0.12.tar.gz Collecting mock (from chainer) Downloading mock-2.0.0-py2.py3-none-any.whl (56kB) 100% |????????????????????????????????| 61kB 3.7MB/s Collecting nose (from chainer) Downloading nose-1.3.7-py3-none-any.whl (154kB) 100% |????????????????????????????????| 163kB 4.0MB/s Requirement already satisfied: numpy>=1.9.0 in /usr/lib64/python3.4/site-packages (from chainer) Collecting protobuf>=2.6.0 (from chainer) Downloading protobuf-3.4.0-cp34-cp34m-manylinux1_x86_64.whl (6.2MB) 100% |????????????????????????????????| 6.2MB 151kB/s Requirement already satisfied: six>=1.9.0 in /usr/lib/python3.4/site-packages (from chainer) Collecting pbr>=0.11 (from mock->chainer) Downloading pbr-3.1.1-py2.py3-none-any.whl (99kB) 100% |????????????????????????????????| 102kB 6.9MB/s Requirement already satisfied: setuptools in /usr/lib/python3.4/site-packages (from protobuf>=2.6.0->chainer) Building wheels for collected packages: chainer, filelock Running setup.py bdist_wheel for chainer ... done Stored in directory: /root/.cache/pip/wheels/49/7a/65/011f3cb35e105e60f018f6ac887e87cccf4bbc4bf5342fdb4e Running setup.py bdist_wheel for filelock ... done Stored in directory: /root/.cache/pip/wheels/68/e0/0c/ca5c34b3b61ef8d3caf4b7fbbfb2f9f840128ce4015ad52566 Successfully built chainer filelock Installing collected packages: filelock, pbr, mock, nose, protobuf, chainer Successfully installed chainer-2.0.2 filelock-2.0.12 mock-2.0.0 nose-1.3.7 pbr-3.1.1 protobuf-3.4.0
bokeh
$ sudo pip3 install bokeh
pandas
$ sudo pip3 install pandas
pandas-datareader
$ sudo pip3 install pandas-datareader
2019.03
tensorflow
$ sudo pip3 install tensorflow Collecting tensorflow Downloading https://files.pythonhosted.org/packages/37/f0/7b2fd5c7cddb3f7f11e7859850f543e19009761cdb965ed8bd98f66d60fd/tensorflow-1.13.1-cp34-cp34m-manylinux1_x86_64.whl (93.2MB) 100% |????????????????????????????????| 93.2MB 8.7kB/s Collecting astor>=0.6.0 (from tensorflow) Downloading https://files.pythonhosted.org/packages/35/6b/11530768cac581a12952a2aad00e1526b89d242d0b9f59534ef6e6a1752f/astor-0.7.1-py2.py3-none-any.whl Collecting keras-preprocessing>=1.0.5 (from tensorflow) Downloading https://files.pythonhosted.org/packages/c0/bf/0315ef6a9fd3fc2346e85b0ff1f5f83ca17073f2c31ac719ab2e4da0d4a3/Keras_Preprocessing-1.0.9-py2.py3-none-any.whl (59kB) 100% |????????????????????????????????| 61kB 5.3MB/s Collecting tensorflow-estimator<1.14.0rc0,>=1.13.0 (from tensorflow) Downloading https://files.pythonhosted.org/packages/bb/48/13f49fc3fa0fdf916aa1419013bb8f2ad09674c275b4046d5ee669a46873/tensorflow_estimator-1.13.0-py2.py3-none-any.whl (367kB) 100% |????????????????????????????????| 368kB 2.6MB/s Collecting gast>=0.2.0 (from tensorflow) Downloading https://files.pythonhosted.org/packages/4e/35/11749bf99b2d4e3cceb4d55ca22590b0d7c2c62b9de38ac4a4a7f4687421/gast-0.2.2.tar.gz Collecting termcolor>=1.1.0 (from tensorflow) Downloading https://files.pythonhosted.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz Collecting keras-applications>=1.0.6 (from tensorflow) Downloading https://files.pythonhosted.org/packages/90/85/64c82949765cfb246bbdaf5aca2d55f400f792655927a017710a78445def/Keras_Applications-1.0.7-py2.py3-none-any.whl (51kB) 100% |????????????????????????????????| 61kB 7.8MB/s Requirement already satisfied: wheel>=0.26 in /usr/lib/python3.4/site-packages (from tensorflow) Collecting numpy>=1.13.3 (from tensorflow) Downloading https://files.pythonhosted.org/packages/cf/8d/6345b4f32b37945fedc1e027e83970005fc9c699068d2f566b82826515f2/numpy-1.16.2.zip (5.1MB) 100% |????????????????????????????????| 5.1MB 207kB/s Collecting absl-py>=0.1.6 (from tensorflow) Downloading https://files.pythonhosted.org/packages/31/bc/ab68120d1d89ae23b694a55fe2aece2f91194313b71f9b05a80b32d3c24b/absl-py-0.7.0.tar.gz (96kB) 100% |????????????????????????????????| 102kB 8.0MB/s Requirement already satisfied: six>=1.10.0 in /usr/lib/python3.4/site-packages (from tensorflow) Collecting tensorboard<1.14.0,>=1.13.0 (from tensorflow) Downloading https://files.pythonhosted.org/packages/fa/7b/3ee06856ec30d5136cd2002408df1d111fcff269f3691147dbf3b8dc0ba2/tensorboard-1.13.0-py3-none-any.whl (3.2MB) 100% |????????????????????????????????| 3.2MB 326kB/s Collecting protobuf>=3.6.1 (from tensorflow) Downloading https://files.pythonhosted.org/packages/61/52/463bac997cee19c74e1aa9153e4aef1af26737f90c91837c13efa0225c25/protobuf-3.6.1-cp34-cp34m-manylinux1_x86_64.whl (1.1MB) 100% |????????????????????????????????| 1.1MB 905kB/s Collecting grpcio>=1.8.6 (from tensorflow) Downloading https://files.pythonhosted.org/packages/62/23/fa20590eb7005f711f8da4d27770c838b80bb1b18af88d1427f89ed42fe7/grpcio-1.19.0-cp34-cp34m-manylinux1_x86_64.whl (10.8MB) 100% |????????????????????????????????| 10.8MB 92kB/s Requirement already satisfied: mock>=2.0.0 in /usr/lib/python3.4/site-packages (from tensorflow-estimator<1.14.0rc0,>=1.13.0->tensorflow) Collecting h5py (from keras-applications>=1.0.6->tensorflow) Downloading https://files.pythonhosted.org/packages/9c/fb/16dfe209a415c700577e41d16b46ca3514f920b96ebeed2895d01bf97316/h5py-2.9.0-cp34-cp34m-manylinux1_x86_64.whl (2.8MB) 100% |????????????????????????????????| 2.8MB 375kB/s Collecting werkzeug>=0.11.15 (from tensorboard<1.14.0,>=1.13.0->tensorflow) Downloading https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl (322kB) 100% |????????????????????????????????| 327kB 3.0MB/s Collecting markdown>=2.6.8 (from tensorboard<1.14.0,>=1.13.0->tensorflow) Downloading https://files.pythonhosted.org/packages/7a/6b/5600647404ba15545ec37d2f7f58844d690baf2f81f3a60b862e48f29287/Markdown-3.0.1-py2.py3-none-any.whl (89kB) 100% |????????????????????????????????| 92kB 7.8MB/s Requirement already satisfied: setuptools in /usr/lib/python3.4/site-packages (from protobuf>=3.6.1->tensorflow) Requirement already satisfied: pbr>=0.11 in /usr/lib/python3.4/site-packages (from mock>=2.0.0->tensorflow-estimator<1.14.0rc0,>=1.13.0->tensorflow) Building wheels for collected packages: gast, termcolor, numpy, absl-py Running setup.py bdist_wheel for gast ... done Stored in directory: /root/.cache/pip/wheels/5c/2e/7e/a1d4d4fcebe6c381f378ce7743a3ced3699feb89bcfbdadadd Running setup.py bdist_wheel for termcolor ... done Stored in directory: /root/.cache/pip/wheels/7c/06/54/bc84598ba1daf8f970247f550b175aaaee85f68b4b0c5ab2c6 Running setup.py bdist_wheel for numpy ... done Stored in directory: /root/.cache/pip/wheels/8c/a8/49/e458f0fdbc4fe3759be6b9371e172b3f0e82c09f5a750e977e Running setup.py bdist_wheel for absl-py ... done Stored in directory: /root/.cache/pip/wheels/90/db/f8/2c3101f72ef1ad434e4662853174126ce30201a3e163dcbeca Successfully built gast termcolor numpy absl-py Installing collected packages: astor, numpy, keras-preprocessing, absl-py, tensorflow-estimator, gast, termcolor, h5py, keras-applications, werkzeug, grpcio, protobuf, markdown, tensorboard, tensorflow Found existing installation: numpy 1.13.1 Uninstalling numpy-1.13.1: Successfully uninstalled numpy-1.13.1 Found existing installation: protobuf 3.4.0 Uninstalling protobuf-3.4.0: Successfully uninstalled protobuf-3.4.0 Successfully installed absl-py-0.7.0 astor-0.7.1 gast-0.2.2 grpcio-1.19.0 h5py-2.9.0 keras-applications-1.0.7 keras-preprocessing-1.0.9 markdown-3.0.1 numpy-1.16.2 protobuf-3.6.1 tensorboard-1.13.0 tensorflow-1.13.1 tensorflow-estimator-1.13.0 termcolor-1.1.0 werkzeug-0.14.1 You are using pip version 9.0.1, however version 19.0.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command.
keras
$ sudo pip3 install keras Collecting keras Downloading https://files.pythonhosted.org/packages/5e/10/aa32dad071ce52b5502266b5c659451cfd6ffcbf14e6c8c4f16c0ff5aaab/Keras-2.2.4-py2.py3-none-any.whl (312kB) 100% |????????????????????????????????| 317kB 2.7MB/s Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/lib64/python3.4/site-packages (from keras) Requirement already satisfied: six>=1.9.0 in /usr/lib/python3.4/site-packages (from keras) Requirement already satisfied: numpy>=1.9.1 in /usr/lib64/python3.4/site-packages (from keras) Requirement already satisfied: scipy>=0.14 in /usr/lib64/python3.4/site-packages (from keras) Requirement already satisfied: pyyaml in /usr/lib64/python3.4/site-packages (from keras) Requirement already satisfied: h5py in /usr/lib64/python3.4/site-packages (from keras) Requirement already satisfied: keras-applications>=1.0.6 in /usr/lib64/python3.4/site-packages (from keras) Installing collected packages: keras Successfully installed keras-2.2.4 You are using pip version 9.0.1, however version 19.0.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command.
sequential
$ sudo pip3 install sequential Collecting sequential Downloading https://files.pythonhosted.org/packages/49/6b/f148875fe84ab5eed83efe45d4cfdfa0a951e08ed5e3660ca935e63ba311/sequential-1.0.0.tar.gz Building wheels for collected packages: sequential Running setup.py bdist_wheel for sequential ... done Stored in directory: /root/.cache/pip/wheels/5a/89/ee/1f8d0c0156b5cbf5ae26a0caa7c6de3db7c1581d8b5a04ea27 Successfully built sequential Installing collected packages: sequential Successfully installed sequential-1.0.0 You are using pip version 9.0.1, however version 19.0.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command.
pip 19.0.3
$ sudo pip3 install --upgrade pip Collecting pip Downloading https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl (1.4MB) 100% |????????????????????????????????| 1.4MB 686kB/s Installing collected packages: pip Found existing installation: pip 9.0.1 Uninstalling pip-9.0.1: Successfully uninstalled pip-9.0.1 Successfully installed pip-19.0.3
jupyter-tensorboard
$ sudo /usr/local/bin/pip3 install jupyter-tensorboard DEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be the last one supporting it. Please upgrade your Python as Python 3.4 won't be maintained after March 2019 (cf PEP 429). Collecting jupyter-tensorboard Downloading https://files.pythonhosted.org/packages/89/3f/62e1150d97f3c99a236d525a07bd9e540137d02c0269497508c265ad4ae2/jupyter_tensorboard-0.1.9.tar.gz Requirement already satisfied: notebook>=5.0 in /usr/lib/python3.4/site-packages (from jupyter-tensorboard) (5.0.0) Requirement already satisfied: nbconvert in /usr/lib/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (5.2.1) Requirement already satisfied: terminado>=0.3.3; sys_platform != "win32" in /usr/lib/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (0.6) Requirement already satisfied: nbformat in /usr/lib/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (4.3.0) Requirement already satisfied: traitlets>=4.2.1 in /usr/lib/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (4.3.2) Requirement already satisfied: ipython-genutils in /usr/lib/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (0.2.0) Requirement already satisfied: ipykernel in /usr/lib/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (4.6.1) Requirement already satisfied: jupyter-core in /usr/lib/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (4.3.0) Requirement already satisfied: jupyter-client in /usr/lib/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (5.1.0) Requirement already satisfied: jinja2 in /usr/lib64/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (2.9.6) Requirement already satisfied: tornado>=4 in /usr/lib64/python3.4/site-packages (from notebook>=5.0->jupyter-tensorboard) (4.5.1) Requirement already satisfied: testpath in /usr/lib/python3.4/site-packages (from nbconvert->notebook>=5.0->jupyter-tensorboard) (0.3.1) Requirement already satisfied: pandocfilters>=1.4.1 in /usr/lib/python3.4/site-packages (from nbconvert->notebook>=5.0->jupyter-tensorboard) (1.4.1) Requirement already satisfied: mistune!=0.6 in /usr/lib/python3.4/site-packages (from nbconvert->notebook>=5.0->jupyter-tensorboard) (0.7.4) Requirement already satisfied: bleach in /usr/lib/python3.4/site-packages (from nbconvert->notebook>=5.0->jupyter-tensorboard) (2.0.0) Requirement already satisfied: entrypoints>=0.2.2 in /usr/lib/python3.4/site-packages (from nbconvert->notebook>=5.0->jupyter-tensorboard) (0.2.3) Requirement already satisfied: pygments in /usr/lib64/python3.4/site-packages (from nbconvert->notebook>=5.0->jupyter-tensorboard) (2.2.0) Requirement already satisfied: ptyprocess in /usr/lib/python3.4/site-packages (from terminado>=0.3.3; sys_platform != "win32"->notebook>=5.0->jupyter-tensorboard) (0.5.2) Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in /usr/lib/python3.4/site-packages (from nbformat->notebook>=5.0->jupyter-tensorboard) (2.6.0) Requirement already satisfied: decorator in /usr/lib/python3.4/site-packages (from traitlets>=4.2.1->notebook>=5.0->jupyter-tensorboard) (4.1.1) Requirement already satisfied: six in /usr/lib/python3.4/site-packages (from traitlets>=4.2.1->notebook>=5.0->jupyter-tensorboard) (1.10.0) Requirement already satisfied: ipython>=4.0.0 in /usr/lib/python3.4/site-packages (from ipykernel->notebook>=5.0->jupyter-tensorboard) (6.1.0) Requirement already satisfied: pyzmq>=13 in /usr/lib64/python3.4/site-packages (from jupyter-client->notebook>=5.0->jupyter-tensorboard) (16.0.2) Requirement already satisfied: python-dateutil>=2.1 in /usr/lib/python3.4/site-packages (from jupyter-client->notebook>=5.0->jupyter-tensorboard) (2.6.1) Requirement already satisfied: MarkupSafe>=0.23 in /usr/lib64/python3.4/site-packages (from jinja2->notebook>=5.0->jupyter-tensorboard) (1.0) Requirement already satisfied: backports-abc>=0.4 in /usr/lib/python3.4/site-packages (from tornado>=4->notebook>=5.0->jupyter-tensorboard) (0.5) Requirement already satisfied: html5lib>=0.99999999 in /usr/lib/python3.4/site-packages (from bleach->nbconvert->notebook>=5.0->jupyter-tensorboard) (0.999999999) Requirement already satisfied: pexpect; sys_platform != "win32" in /usr/lib/python3.4/site-packages (from ipython>=4.0.0->ipykernel->notebook>=5.0->jupyter-tensorboard) (4.2.1) Requirement already satisfied: pickleshare in /usr/lib/python3.4/site-packages (from ipython>=4.0.0->ipykernel->notebook>=5.0->jupyter-tensorboard) (0.7.4) Requirement already satisfied: jedi>=0.10 in /usr/lib/python3.4/site-packages (from ipython>=4.0.0->ipykernel->notebook>=5.0->jupyter-tensorboard) (0.10.2) Requirement already satisfied: typing; python_version <= "3.4" in /usr/lib/python3.4/site-packages (from ipython>=4.0.0->ipykernel->notebook>=5.0->jupyter-tensorboard) (3.6.1) Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.4 in /usr/lib/python3.4/site-packages (from ipython>=4.0.0->ipykernel->notebook>=5.0->jupyter-tensorboard) (1.0.14) Requirement already satisfied: simplegeneric>0.8 in /usr/lib/python3.4/site-packages (from ipython>=4.0.0->ipykernel->notebook>=5.0->jupyter-tensorboard) (0.8.1) Requirement already satisfied: setuptools>=18.5 in /usr/lib/python3.4/site-packages (from ipython>=4.0.0->ipykernel->notebook>=5.0->jupyter-tensorboard) (36.2.0) Requirement already satisfied: webencodings in /usr/lib/python3.4/site-packages (from html5lib>=0.99999999->bleach->nbconvert->notebook>=5.0->jupyter-tensorboard) (0.5.1) Requirement already satisfied: wcwidth in /usr/lib/python3.4/site-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython>=4.0.0->ipykernel->notebook>=5.0->jupyter-tensorboard) (0.1.7) Building wheels for collected packages: jupyter-tensorboard Building wheel for jupyter-tensorboard (setup.py) ... done Stored in directory: /root/.cache/pip/wheels/49/16/c7/1e93200833f116d611204dc89e7ba0e69ff88197cb7273937e Successfully built jupyter-tensorboard Installing collected packages: jupyter-tensorboard Successfully installed jupyter-tensorboard-0.1.9
Jupyter NotebookでPython以外のプログラミング言語を使う
bash_kernel を追加
$ sudo /usr/local/bin/pip3 install bash_kernel $ python3 -m bash_kernel.install Installing IPython kernel spec
Python 3.6
その他のインストール(yum)
zeromq,zeromq-devel
$ sudo yum install zeromq $ sudo yum install zeromq-devel
参考
jupyter使い方
- データ分析の必需品「Jupyter Notebook」の魅力とは
- chainer で MINSTを使う例とか
- Jupyter Notebook を使ってみよう
- Jupyter Notebookをインストール・設定して勉強ノート作成環境をつくる (Mac)
- Jupyter の Docker イメージを使ってみる
- "Docker イメージを使ってみる"というタイトルだが pandas や bokeh の使用例も載っている
- Python で 株価データ取得する pandas.io モジュール が pandas_datareader に変わっていた件 〜 pip install 時のパッケージ名は、pandas-datareader だが、import時は、pandas_datareader で わかりにくいので注意
- 上記の pandas の使用が上手くいかなかったので
jupyter環境構築
- Jupyter Notebook(IPython)のインストールと始め方
- Jupyterをサーバ上で起動する
- 上記は CentOS7 上に、Anaconda を使わずに、環境構築している。
- Jupyter Notebook データ分析専用サーバーを構築する
- 上記は Anaconda 使用。
コメントに
チームだと、jupyterhubが便利な気がします。
とある
JupyterHubの構築
- JupyterHubの構築
- こちらは CentOS だが Anaconda を使っている
- JupyterHub のセットアップ