Getting started with Microsoft CNTK with Nvidia GPU’s / CUDA

8 min readApr 26, 2018

Installaing Microsoft CNTK along with NVIDIA CUDA

The Microsoft CNTK installation page is pretty detailed int and some times you might tend to skip or miss a step, in this guide i am just trying to help to get Microsoft CNTK working with Nvidia Cuda drivers for (Tesla P80/P100 GPU’s).

I just hope that installation of Microsoft CNTK kit would be easier on Linux distributions in the near future.

Environment :
Ubuntu 16.04
Nvidia P80/P100

NVIDIA Cuda installation

1) Purge the previous installations
sudo apt-get purge cuda
sudo apt-get purge libcudnn6
sudo apt-get purge libcudnn6-dev

2) Get the right packages from Nvidia developer website and install them

wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7_7.0.5.15-1+cuda9.0_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn7-dev_7.0.5.15-1+cuda9.0_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl2_2.1.4-1+cuda9.0_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libnccl-dev_2.1.4-1+cuda9.0_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1604_9.0.176–1_amd64.deb
sudo dpkg -i libcudnn7_7.0.5.15–1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-dev_7.0.5.15–1+cuda9.0_amd64.deb
sudo dpkg -i libnccl2_2.1.4–1+cuda9.0_amd64.deb
sudo dpkg -i libnccl-dev_2.1.4–1+cuda9.0_amd64.deb
sudo apt-get update
sudo apt-get install cuda=9.0.176–1 -y
sudo apt-get install libcudnn7-dev -y
sudo apt-get install libnccl-dev -y
sudo apt-get install cuda

3) Update the environment with the below variables

echo “UPDATING THE BASH RC WITH RESPECTIVE PATH VARIABLES…….”
echo export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}} >> ~/.bashrc
export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} >> ~/.bashrc

4) Once that’s done i assume that you have tensorflow intalled run the sample program to check if things work, by default the tensorflow program is run on GPU’s

5) To check GPU Usage

nvidia-smi
nvidia-smi pmon (check if the processis activily using GPU’s)
nvcc — version

Reference

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#pre-installation-actions
https://www.pugetsystems.com/labs/hpc/NVIDIA-CUDA-with-Ubuntu-16-04-beta-on-a-laptop-if-you-just-cannot-wait-775/
https://www.digitalocean.com/community/tutorials/how-to-install-the-anaconda-python-distribution-on-ubuntu-16-04
https://yangcha.github.io/CUDA90/
https://www.tensorflow.org/install/install_sources#common_installation_problems
https://www.tensorflow.org/install/install_linux#InstallingAnaconda

Once this is done you could continue with Microsoft CNTK installation

1) add-apt-repository ppa:git-core/ppa; apt update; apt install git

2) sudo wget https://github.com/01org/mkl-dnn/releases/download/v0.12/mklml_lnx_2018.0.1.20171227.tgz
3) sudo tar -xzf mklml_lnx_2018.0.1.20171227.tgz -C /usr/local/mklml
4) wget — no-verbose -O — https://github.com/01org/mkl-dnn/archive/v0.12.tar.gz | tar -xzf — && \
cd mkl-dnn-0.12 && \
ln -s /usr/local external && \
mkdir -p build && \
cd build && \
cmake .. && \
make && \
make install && \
cd ../.. && \
rm -rf mkl-dnn-0.12

Install the project…
— Install configuration: “Release”
— Up-to-date: /usr/local/lib/libmklml_intel.so
— Up-to-date: /usr/local/lib/libiomp5.so
— Up-to-date: /usr/local/share/doc/mkldnn/LICENSE
— Installing: /usr/local/lib/libmkldnn.so.0.12.0
— Up-to-date: /usr/local/lib/libmkldnn.so.0
— Up-to-date: /usr/local/lib/libmkldnn.so
— Set runtime path of “/usr/local/lib/libmkldnn.so.0.12.0” to “”
— Up-to-date: /usr/local/include/mkldnn.h
— Up-to-date: /usr/local/include/mkldnn_types.h
— Up-to-date: /usr/local/include/mkldnn.hpp

5) Openmpi

wget https://www.open-mpi.org/software/ompi/v1.10/downloads/openmpi-1.10.3.tar.gz
tar -xzvf ./openmpi-1.10.3.tar.gz
cd openmpi-1.10.3
./configure — prefix=/usr/local/mpi
make -j all
sudo make install

6) Tools

sudo apt-get install autoconf automake libtool curl make g++ unzip

7) Protobuf
wget https://github.com/google/protobuf/archive/v3.1.0.tar.gz
tar -xzf v3.1.0.tar.gz
cd protobuf-3.1.0
./autogen.sh
./configure CFLAGS=-fPIC CXXFLAGS=-fPIC — disable-shared — prefix=/usr/local/protobuf-3.1.0
make -j $(nproc)
sudo make install

8) Zlip
sudo apt-get install zlib1g-dev

wget http://nih.at/libzip/libzip-1.1.2.tar.gz
tar -xzvf ./libzip-1.1.2.tar.gz
cd libzip-1.1.2
./configure
make -j all
sudo make install

export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

9)
sudo apt-get install libbz2-dev
sudo apt-get install python-dev

10) Boost

wget -q -O — https://sourceforge.net/projects/boost/files/boost/1.60.0/boost_1_60_0.tar.gz/download | tar -xzf —
cd boost_1_60_0
./bootstrap.sh — prefix=/usr/local/boost-1.60.0
sudo ./b2 -d0 -j”$(nproc)” install

libs/iostreams/src/bzip2.cpp:20:56: fatal error: bzlib.h: No such file or directory
compilation terminated.
…skipped <pbin.v2/libs/iostreams/build/gcc-5.4.0/release/link-static/threading-multi>libboost_iostreams.a(clean) for lack of <pbin.v2/libs/iostreams/build/gcc-5.4.0/release/link-static/threading-multi>bzip2.o…
…skipped <pbin.v2/libs/iostreams/build/gcc-5.4.0/release/link-static/threading-multi>libboost_iostreams.a for lack of <pbin.v2/libs/iostreams/build/gcc-5.4.0/release/link-static/threading-multi>bzip2.o…
…skipped <p/usr/local/boost-1.60.0/lib>libboost_iostreams.a for lack of <pbin.v2/libs/iostreams/build/gcc-5.4.0/release/link-static/threading-multi>libboost_iostreams.a…

11) Inistall the Boost Dependencies
apt-get install libghc-bzlib-dev libghc-bzlib-doc libghc-bzlib-prof

12) ~/boost/boost_1_60_0# sudo ./b2 -d0 -j”$(nproc)” install
Performing configuration checks

- 32-bit : no (cached)
— 64-bit : yes (cached)
— arm : no (cached)
— mips1 : no (cached)
— power : no (cached)
— sparc : no (cached)
— x86 : yes (cached)
— symlinks supported : yes (cached)
— lockfree boost::atomic_flag : yes (cached)
— has_icu builds : no (cached)
warning: Graph library does not contain MPI-based parallel components.
note: to enable them, add “using mpi ;” to your user-config.jam
— zlib : yes (cached)
— iconv (libc) : yes (cached)
— icu : no (cached)
— icu (lib64) : no (cached)
— compiler-supports-visibility : yes (cached)
— compiler-supports-ssse3 : yes (cached)
— compiler-supports-avx2 : yes (cached)
— gcc visibility : yes (cached)
— long double support : yes (cached)
warning: skipping optional Message Passing Interface (MPI) library.
note: to enable MPI support, add “using mpi ;” to user-config.jam.
note: to suppress this message, pass “ — without-mpi” to bjam.
note: otherwise, you can safely ignore this message.
— zlib : yes (cached)

Component configuration:

- atomic : building
— chrono : building
— container : building
— context : building
— coroutine : building
— coroutine2 : building
— date_time : building
— exception : building
— filesystem : building
— graph : building
— graph_parallel : building
— iostreams : building
— locale : building
— log : building
— math : building
— mpi : building
— program_options : building
— python : building
— random : building
— regex : building
— serialization : building
— signals : building
— system : building
— test : building
— thread : building
— timer : building
— type_erasure : building
— wave : building

~/boost/boost_1_60_0# sudo ./b2 -d0 -j”$(nproc)” install
Performing configuration checks

Component configuration:

13) Install swig from source into

/usr/local/swig-3.0.10/

14) Install Annoconda

Annoconda 3.6
https://repo.continuum.io/archive/Anaconda3-5.1.0-Linux-x86_64.sh

# For cntk-py36:
conda env update — file [CNTK clone root]/Scripts/install/linux/conda-linux-cntk-py36-environment.yml — name cntk-py36
# For cntk-py35:
conda env update — file [CNTK clone root]/Scripts/install/linux/conda-linux-cntk-py35-environment.yml — name cntk-py35
# For cntk-py27:
conda env update — file [CNTK clone root]/Scripts/install/linux/conda-linux-cntk-py27-environment.yml — name cntk-py27

conda env update — file [CNTK clone root]/Scripts/install/linux/conda-linux-cntk-py36-environment.yml — name cntk-py36

15) Install CNTK

git clone https://github.com/Microsoft/CNTK.git

16) Create the Annoconda environment

conda env update — file /root/cntk/CNTK/Scripts/install/linux/conda-linux-cntk-py36-environment.yml — name cntk-py36

Running setup.py bdist_wheel for args … done
Stored in directory: /root/.cache/pip/wheels/58/54/ea/d995d18af68c057eb76b87b02c92bc66ac34d360ef141780f4
Successfully built easydict gym keras pydot-ng atari-py PyOpenGL theano clint future args
Installing collected packages: easydict, future, pyglet, atari-py, PyOpenGL, gym, theano, keras, pydot-ng, pygame, py, pytest, sphinx-rtd-theme, sphinx, nbsphinx, requests-toolbelt, pkginfo, args, clint, twine, protobuf
Found existing installation: Sphinx 1.7.2
Uninstalling Sphinx-1.7.2:
Successfully uninstalled Sphinx-1.7.2
Successfully installed PyOpenGL-3.1.0 args-0.1.0 atari-py-0.1.1 clint-0.5.1 easydict-1.6 future-0.16.0 gym-0.8.1 keras-2.0.6 nbsphinx-0.2.14 pkginfo-1.4.2 protobuf-3.2.0 py-1.5.3 pydot-ng-1.0.0 pygame-1.9.3 pyglet-1.3.2 pytest-3.0.3 requests-toolbelt-0.8.0 sphinx-1.5.4 sphinx-rtd-theme-0.2.4 theano-1.0.1 twine-1.8.1
#
# To activate this environment, use:
# > source activate cntk-py36
#
# To deactivate an active environment, use:
# > source deactivate
#

# For a Python 3.6 based version:
conda env create — file [CNTK clone root]/Scripts/install/linux/conda-linux-cntk-py36-environment.yml
# For a Python 3.5 based version:
conda env create — file [CNTK clone root]/Scripts/install/linux/conda-linux-cntk-py35-environment.yml
# For a Python 2.7 based version:
conda env create — file [CNTK clone root]/Scripts/install/linux/conda-linux-cntk-py27-environment.yml

conda env update — file [CNTK clone root]/Scripts/install/linux/conda-linux-cntk-py36-environment.yml — name cntk-py36

17) Activate the environment created
source activate cntk-py36

18) configure — with-swig=/usr/local/swig-3.0.10 — with-py36-path=$HOME/anaconda3/envs/cntk-py36

— with-swig[=directory]

cd [CNTK clone root]
export PYTHONPATH=$PWD/bindings/python:$PYTHONPATH
cd [CNTK clone root]/bindings/python
export LD_LIBRARY_PATH=$PWD/cntk/libs:$LD_LIBRARY_PATH

19) Install CNTK (regular)
./install-cntk.sh — py-version 36

20) If are are trying with GPU then the GPU Specific package
wget https://cntk.ai/BinaryDrop/CNTK-2-5-1-Linux-64bit-GPU.tar.gz

21) Install CNTK with annoconda and with py version 36
./install-cntk.sh — py-version 36 — anaconda-basepath /root/anaconda3

+ pip install https://cntk.ai/PythonWheel//GPU/cntk_gpu-2.5.1-cp36-cp36m-linux_x86_64.whl
Collecting cntk-gpu==2.5.1 from https://cntk.ai/PythonWheel//GPU/cntk_gpu-2.5.1-cp36-cp36m-linux_x86_64.whl
Downloading https://cntk.ai/PythonWheel//GPU/cntk_gpu-2.5.1-cp36-cp36m-linux_x86_64.whl (474.8MB)
100% |████████████████████████████████| 474.8MB 2.8kB/s
Requirement already satisfied: scipy>=0.17 in /root/anaconda3/envs/cntk-py36/lib/python3.6/site-packages (from cntk-gpu==2.5.1)
Requirement already satisfied: numpy>=1.11 in /root/anaconda3/envs/cntk-py36/lib/python3.6/site-packages (from cntk-gpu==2.5.1)
Installing collected packages: cntk-gpu
Successfully installed cntk-gpu-2.5.1
You are using pip version 9.0.1, however version 10.0.0 is available.
You should consider upgrading via the ‘pip install — upgrade pip’ command.
+ set +x
+ LD_LIBRARY_PATH_SETTING=/root/cntk/cntk/cntk/lib:/root/cntk/cntk/cntk/dependencies/lib
+ ‘[‘ 0 = 1 ‘]’
+ LD_LIBRARY_PATH_SETTING+=’:$LD_LIBRARY_PATH’
+ ACTIVATE_SCRIPT_NAME=activate-cntk
+ cat
+ cat