tensorflow 에서 제공하는 Install 가이드를(https://www.tensorflow.org/install/install_linux) 통해서 설치해 보았다.


Determine how to install TensorFlow


Docker completely isolates the TensorFlow installation from pre-existing packages on your machine. The Docker container contains TensorFlow and all its dependencies. Note that the Docker image can be quite large (hundreds of MBs). You might choose the Docker installation if you are incorporating TensorFlow into a larger application architecture that already uses Docker.


Installing with Docker

Take the following steps to install TensorFlow through Docker:

  1. Install Docker on your machine as described in the Docker documentation.
  2. Optionally, create a Linux group called docker to allow launching containers without sudo as described in the Docker documentation. (If you don't do this step, you'll have to use sudo each time you invoke Docker.)
  3. To install a version of TensorFlow that supports GPUs, you must first install nvidia-docker, which is stored in github.
  4. Launch a Docker container that contains one of the TensorFlow binary images.

The remainder of this section explains how to launch a Docker container.


Docker를 통한 GPU 버전을 설치하기전에 nvidia-docker를 설치해줘야 한다.

현재 설치 할 시스템의 OS가 CentOs7이라서  https://github.com/NVIDIA/nvidia-docker 에서  "CentOS/RHEL 7 x86_64"  가이드 된 대로 진행을 해보면 뭔가 적절하게 되지 않는다.


그래서 구글링을 하게 되었다. http://blog.exxactcorp.com/installing-using-docker-nv-docker-centos-7/ 이 블로거가 가이드한대로 설치하면 잘 설치가 된다.

Installing and getting DOCKER and NV-DOCKER running in CentOS 7 is a straight forward process:

# Assumes CentOS 7
# Assumes NVIDIA Driver is installed as per requirements ( < 340.29 )
# Install DOCKER
sudo curl -fsSL https://get.docker.com/ | sh
# Start DOCKER
sudo systemctl start docker
# Add dockeruser, usermod change
sudo adduser dockeruser
usermod -aG docker dockeruser
# Install NV-DOCKER
# GET NVIDIA-DOCKER
wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker-1.0.1-1.x86_64.rpm
# INSTALL
sudo rpm -i /tmp/nvidia-docker*.rpm
# Start NV-DOCKER Service
systemctl start nvidia-docker


그럼 이제 nvidia-docker 가 잘 설치가 되었다.


GPU support

Prior to installing TensorFlow with GPU support, ensure that your system meets all NVIDIA software requirements. To launch a Docker container with NVidia GPU support, enter a command of the following format:


$ nvidia-docker run -it -p hostPort:containerPort TensorFlowGPUImage

where:

  • -p hostPort:containerPort is optional. If you plan to run TensorFlow programs from the shell, omit this option. If you plan to run TensorFlow programs as Jupyter notebooks, set both hostPort and containerPort to 8888.
  • TensorFlowGPUImage specifies the Docker container. You must specify one of the following values:
    • gcr.io/tensorflow/tensorflow:latest-gpu, which is the latest TensorFlow GPU binary image.
    • gcr.io/tensorflow/tensorflow:latest-devel-gpu, which is the latest TensorFlow GPU Binary image plus source code.
    • gcr.io/tensorflow/tensorflow:version-gpu, which is the specified version (for example, 0.12.1) of the TensorFlow GPU binary image.
    • gcr.io/tensorflow/tensorflow:version-devel-gpu, which is the specified version (for example, 0.12.1) of the TensorFlow GPU binary image plus source code.

We recommend installing one of the latest versions. For example, the following command launches the latest TensorFlow GPU binary image in a Docker container from which you can run TensorFlow programs in a shell:

$ nvidia-docker run -it gcr.io/tensorflow/tensorflow:latest-gpu bash

The following command also launches the latest TensorFlow GPU binary image in a Docker container. In this Docker container, you can run TensorFlow programs in a Jupyter notebook:

$ nvidia-docker run -it -p 8888:8888 gcr.io/tensorflow/tensorflow:latest-gpu

The following command installs an older TensorFlow version (0.12.1):

$ nvidia-docker run -it -p 8888:8888 gcr.io/tensorflow/tensorflow:0.12.1-gpu

Docker will download the TensorFlow binary image the first time you launch it. For more details see the TensorFlow docker readme.


위에 내용을 참조하여 내가 실제로 수행한 명령어들은 와래와 같다.


$ sudo nvidia-docker run -d -p 8888:8888 -p 8889:8889 -p 6901:6901 --name tensorflow_gpu gcr.io/tensorflow/tensorflow:latest-gpu

위의 명령어를 수행하면 tensorflow + cuda + notebook 이 설치된 빈 깡통 Ubuntu16.04 OS가 설치된다.

※ 여기에서 중요한점은 -p 는 포트포워딩할 포트인데 내가 사용할 서버의 포트를 어떻게 사용할 것인지를 충분히 고려하여야 한다는점이다. 현재까지는 docker는 런타임중에 포트를 추가할 수 없게 되어 있다. 포트를 추가 하려면 다양한 방법이 있겠지만

나와 같은 경우에는 현재 이미지를 commit 하여 저장한 상태에서 다시 포트를 추가하여 container를 만들었다.


$ sudo nvidia-docker exec -it tensorflow_gpu bash

이 명령을 수행하게 되면 내가 생성한 tensorflow_gpu bash 컨테이너에 접속할 수 있게 된다.


Validate your installation

Run a short TensorFlow program

Invoke python from your shell as follows:

$ python

Enter the following short program inside the python interactive shell:

Python
import tensorflow as tf
hello
= tf.constant('Hello, TensorFlow!')

sess
= tf.Session()
print(sess.run(hello))

If the system outputs the following, then you are ready to begin writing TensorFlow programs:

Hello, TensorFlow!

위에도 언급했지만, nvidia-docker를 이용하여 컨테이너를 생성하면 빈깡통 ubuntu가 설치된다.

그러므로 기본 유틸리티들을 설치해주어야 한다.


Ubuntu 설치 후 기본 도구 설치

http://cafe.naver.com/telcosn/562 글 이외의 설치 목록은 아래에 있다.

apt-getinstall net-tools

apt-get install tcpdump

apt-get install apt-utils

apt install-y  vim htop  iftop tree  openssh-server  lrzsz openvswitch-switch

sudo apt-get install git


Ubuntu 설치 후 데스크탑 설치

apt-get update
apt-get upgrade
apt-get install tasksel


 

 

 

 

2


참고: https://imitator.kr/Linux/1305


Ubuntu 설치 후 tightvncserver 설치

참고: http://cafe.naver.com/telcosn/437

명령어: vncserver :100 -rfbport 6901 -geometry 1920x1080


Ubuntu 설치 후 시간 동기화

sudo dpkg-reconfigure tzdata -> asia, seoul 선택

Posted by 이성윤

영상 분석을 위한 서버 환경이 CentOs 가 설치되어 있다.


tensorflow.org는 기본적으로 우분투 환경에서 완벽하게 지원하고 있는것으로 보인다.

궁극적으로는 ubuntu로 가는것이 맞는것으로 보이지만, 일단 회사 정책(기조?)가 CentOs로 넘어가고 있는 분위기라서

CentOs의 개발환경이 필요해진 상황이다.

ubuntu Install: http://cafe.naver.com/telcosn/552


영상분석 서버의 스펙은 아래와 같다.

OS: CentOS Linux release 7.3.1611

CPU: 1x E5-1650 V4 (6 Core, 15M Cache, 3.6 GHz, 140W)

MEM: 8x DDR4 PC19200 32GB ECC-REG

HDD: 3x HDD 3.5인치 SATA3 2TB

SSD: 1x 512GB SSD

POWER: POWER 2000W

GPU: NVIDIA TITAN X


CentOs TensorFlow 설치 과정

※ 많은 사람들이 CentOs에서는 python의 virtualenv 환경으로 설치하기를 권장해서 virtualenv 환경에서 설치

참고: https://www.tensorflow.org/install/install_linux 의 Installing with virtualenv


* GPU를 사용하고 있기 때문에 GPU 환경으로 설치

1. Install base

# yum -y install gcc gcc-c++ atlas atlas-devel gcc-gfortran openssl-devel libffi-devel
2. Installing with virtualenv
Take the following steps to install TensorFlow with Virtualenv:

Install pip and virtualenv by issuing one of the following commands:

 

$ sudo yum install python-pip python-devel python-virtualenv # for Python 2.7
$ sudo yum install python3-pip python3-devel python-virtualenv # for Python 3.n

 

Create a virtualenv environment by issuing one of the following commands:

 

$ virtualenv --system-site-packages targetDirectory # for Python 2.7
$ virtualenv --system-site-packages -p python3 targetDirectory # for Python 3.n

 

where targetDirectory specifies the top of the virtualenv tree. Our instructions assume that targetDirectory is ~/tensorflow, but you may choose any directory.

Activate the virtualenv environment by issuing one of the following commands:

 

$ source ~/tensorflow/bin/activate # bash, sh, ksh, or zsh
$ source ~/tensorflow/bin/activate.csh  # csh or tcsh

 

The preceding source command should change your prompt to the following:

 

(tensorflow)$ 

 

Ensure pip ≥8.1 is installed:

 

$ easy_install -U pip

 

Issue one of the following commands to install TensorFlow in the active virtualenv environment:

 


(tensorflow)$ pip install --upgrade tensorflow      # for Python 2.7(tensorflow)
$ pip3 install --upgrade tensorflow     # for Python 3.n(tensorflow)
$ pip install --upgrade tensorflow-gpu  # for Python 2.7 and GPU(tensorflow)
$ pip3 install --upgrade tensorflow-gpu # for Python 3.n and GPU

 

If the preceding command succeeds, skip Step 6. If the preceding command fails, perform Step 6.

(Optional) If Step 5 failed (typically because you invoked a pip version lower than 8.1), install TensorFlow in the active virtualenv environment by issuing a command of the following format:

 

(tensorflow)$ pip install --upgrade tfBinaryURL   # Python 2.7
(tensorflow)$ pip3 install --upgrade tfBinaryURL  # Python 3.n 

 

where tfBinaryURL identifies the URL of the TensorFlow Python package. The appropriate value of tfBinaryURLdepends on the operating system, Python version, and GPU support. Find the appropriate value for tfBinaryURL for your system here. For example, if you are installing TensorFlow for Linux, Python 3.4, and CPU-only support, issue the following command to install TensorFlow in the active virtualenv environment:

 

(tensorflow)$ pip3 install --upgrade \https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.4.0-cp34-cp34m-linux_x86_64.whl

 

If you encounter installation problems, see Common Installation Problems.
3. Next Steps After installing TensorFlow, validate the installation.
Note that you must activate the virtualenv environment each time you use TensorFlow. 
If the virtualenv environment is not currently active, invoke one of the following commands:
$ source ~/tensorflow/bin/activate      # bash, sh, ksh, or zsh
$ source ~/tensorflow/bin/activate.csh  # csh or tcsh
그래픽카드 사용시, 환경변수에 쿠다 관련 실행경로를 추가 해주어야 한다.
vi ~/venvs/tensorflow/bin/activate
PATH="$VIRTUAL_ENV/bin:$PATH:/usr/local/cuda-8.0/bin"
export PATH
LD_LIBRARY_PATH="/usr/local/cuda-8.0/lib64"
export LD_LIBRARY_PATH

When the virtualenv environment is active, you may run TensorFlow programs from this shell.
Your prompt will become the following to indicate that your tensorflow environment is active:
(tensorflow)$ 
When you are done using TensorFlow, you may deactivate the environment by invoking the deactivate function as follows:
(tensorflow)$ deactivate 
The prompt will revert back to your default prompt (as defined by the PS1 environment variable).
Uninstalling TensorFlow
To uninstall TensorFlow, simply remove the tree you created. For example:
$ rm -r targetDirectory 
※ 나의 경우에는 pip install --upgrade mock 이 설치가 안되어서 설치해줌 또한 쿠다환경은 다른 분석도구를 사용하기 위해서 사전에 설치되어 있었음
Posted by 이성윤
Opensource/Jupyter Notebook2017. 12. 18. 12:12

전부터 ipython을 활용한 jupyter notebook을 설치하여 사용해보고 싶었다.


설치 과정을 기록한다.


참고 URL1

https://m.blog.naver.com/PostView.nhn?blogId=samsjang&logNo=220758501873&proxyReferer=http%3A%2F%2Fwww.google.co.kr%2Furl%3Fsa%3Dt%26rct%3Dj%26q%3D%26esrc%3Ds%26source%3Dweb%26cd%3D2%26ved%3D0ahUKEwjClZiQlsfVAhUDFpQKHayyDh4QFggrMAE%26url%3Dhttp%253A%252F%252Fm.blog.naver.com%252Fsamsjang%252F220758501873%26usg%3DAFQjCNEDdEaUY2-FCzJ7rqTdxqgkw9qxtQ


참고 URL2

http://goodtogreate.tistory.com/entry/IPython-Notebook-%EC%84%A4%EC%B9%98%EB%B0%A9%EB%B2%95


두 URL을 참조하면 쉽게 설치가 가능하다.



[tensorflow@da-4fs-080 ~]$ ipython profile create
[ProfileCreate] Generating default config file:

u'/home/tensorflow/.ipython/profile_default/ipython_config.py'
[ProfileCreate] Generating default config file:

u'/home/tensorflow/.ipython/profile_default/ipython_kernel_config.py'

Securing a notebook server
$ jupyter notebook --generate-config

$ ipython
In [1]: from IPython.lib import passwd
In [2]: passwd()
Enter password:
Verify password:
Out[2]: 'sha1:198e275835de:17f4facc23b40f663c519dd60ff7ae11ebc7dda2'

vi /home/tensorflow/.jupyter/jupyter_notebook_config.py
# Password to use for web authentication
c = get_config()
c.NotebookApp.password =
u'sha1:f24baff49ac5:863dd2ae747212ede58125302d227f0ca7b12bb3'

openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem

vi /home/tensorflow/.jupyter/jupyter_notebook_config.py
c.NotebookApp.certfile = u'/home/tensorflow/.jupyter/mycert.pem'
c.NotebookApp.ip = '10.10.104.80'
c.NotebookApp.port = 8008
c.NotebookApp.notebook_dir = u'/home/tensorflow/jupyter'
c.NotebookApp.open_browser = False

mkdir ~/jupyter

'Opensource > Jupyter Notebook' 카테고리의 다른 글

[펌] Jupyter Notebook 활용하기  (0) 2017.12.19
Posted by 이성윤

텐서플로우를 설치한지는 좀 되었지만, 기록을 위해 적어본다.


먼저, TensorFlow를 CentOs에 설치하기 위해서는 뭔가 복잡하다고 느껴졌었는데, Unbuntu에 설치하니

너무 간단하게 설치가 완료 되었다.


TensorFlow 공식 사이트

https://www.tensorflow.org/install/install_linux#InstallingNativePip


OS: Ubuntu 16.04데스크탑 버전

python3 기반으로 설치하였다.


$ sudo apt-get install python-pip python-dev # for Python 2.7

$ sudo apt-get install python3-pip python3-dev # for Python 3.n


$ pip install tensorflow # Python 2.7; CPU support (no GPU support)

$ pip3 install tensorflow # Python 3.n; CPU support (no GPU support)


이게 끝이다.


추가로 Tensorflow 강의를 듣다 보니 필요한 유틸리티를 설치하였다.

pip3 install numpy
sudo apt-get install python3-matplotlib
sudo apt-get install python3-pandas
pip3 install ipython (이건 내가 편해서)

Posted by 이성윤