These are my notes from setting up my Ubuntu 16.04 box with GPU enabled Tensorflow:

Hardware Dependencies

The Cuda documentation encourages us to check a couple things:

Do we have a CUDA capable GPU? Any output here is good:

lspci | grep -i nvidia

I have a 980 Ti, so I see:

01:00.0 VGA compatible controller: NVIDIA Corporation GM200 [GeForce GTX 980 Ti] (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 0fb0 (rev a1)

Software Dependencies

Do we have a supported version of linux? We're looking for x86_64:

uname -m && cat /etc/*release | head -n0

Make sure we have gcc, any output is fine:

gcc --version

Install the headers for your kernel

sudo apt-get install -y linux-headers-$(uname -r)

Uninstall any old debs

Look for cuda packages that are shown with "install" in the last column:

dpkg --get-selections|grep cuda|grep ' install'

Remove each package with e.g.:

dpkg --remove cuda-repo-ubuntu1604

Install Cuda and Drivers

Disable the Nouveau driver as directed here: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/#runfile

Download cuda 8 for tensorflow from https://developer.nvidia.com/cuda-80-ga2-download-archive, get the base installer and the patch

wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda_8.0.61_375.26_linux-run
wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/patches/2/cuda_8.0.61.2_linux-run

or download the latest runfile from https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=runfilelocal, not sure if this will work though.

Install Cuda BUT NOT THE DRIVER via the runfile, e.g.:

sudo sh cuda_8.0.61_375.26_linux-run

Press q, accept, n, y, [enter], y, y,[enter]`

Do you accept the previously read EULA?
accept/decline/quit: accept

Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 375.26?
(y)es/(n)o/(q)uit: n

Install the CUDA 8.0 Toolkit?
(y)es/(n)o/(q)uit: y

Enter Toolkit Location
 [ default is /usr/local/cuda-8.0 ]:

Do you want to install a symbolic link at /usr/local/cuda?
(y)es/(n)o/(q)uit: y

Install the CUDA 8.0 Samples?
(y)es/(n)o/(q)uit: y

Enter CUDA Samples Location
 [ default is /home/ntsoi ]:

Installing the CUDA Toolkit in /usr/local/cuda-8.0 ...

Then install the update

sudo sh cuda_8.0.61.2_linux-run 

and run ldconfig

sudo ldconfig

Now, enter into runlevel 3 by hitting CTRL+ALT+F1 at the login page then:

sudo service gdm stop

Edit /etc/profile.d/cuda.sh and set the contents to:

export PATH=/usr/local/cuda/bin:$PATH 
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Add the ubuntu graphics driver ppa

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update

Search for the latest driver version for your card here: http://www.nvidia.com/Download/index.aspx, but don't download it.

Once you know the version, install it via apt, for example if the nvidia site lists the latest driver version as 384.98, run:

sudo apt install nvidia-384

Install the libcupti library (CUDA Profile Tools Interface)

sudo apt-get install -y libcupti-dev

Reboot and you're good to go

Driver upgrades

It should now be possible to upgrade the driver via apt, but first, run:

sudo apt-get purge nvidia*

Then install the new driver, after looking up the latest version on the nvidia website as mentioned above.

Virtualenv Tensorflow installation

Install pyenv

curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash

Install python 3, then create and activate a virtualenv for tensorflow. If you get an error that pyenv hasn't been loaded, restart your shell and try to activate again

export PYVERSION=3.6.2
pyenv install $PYVERSION
pyenv virtualenv $PYVERSION tensorflow-$PYVERSION
pyenv activate tensorflow-$PYVERSION

Now install Tensorflow with gpu support via pip:

pip install --upgrade tensorflow-gpu

You're installation is done. To use in the future, be sure to pyenv activate tensorflow-3.6.3 before running commands