How do I use TensorFlow GPU?

I tried following the above tutorial. Thing is tensorflow changes a lot and so do the NVIDIA versions needed for running on a GPU. The next issue is that your driver version determines your toolkit version etc. As of today this information about the software requirements should shed some light on how they interplay:

NVIDIA® GPU drivers —CUDA 9.0 requires 384.x or higher.
CUDA® Toolkit —TensorFlow supports CUDA 9.0.
CUPTI ships with the CUDA Toolkit.
cuDNN SDK (>= 7.2) Note: Make sure your GPU has compute compatibility >3.0
(Optional) NCCL 2.2 for multiple GPU support.
(Optional) TensorRT 4.0 to improve latency and throughput for inference on some models.

And here you’ll find the up-to-date requirements stated by tensorflow (which will hopefully be updated by them on a regular basis).

First you need to install tensorflow-gpu, because this package is responsible for gpu computations. Also remember to run your code with environment variable CUDA_VISIBLE_DEVICES = 0 (or if you have multiple gpus, put their indices with comma). There might be some issues related to using gpu. if your tensorflow does not use gpu anyway, try this

Strangely, even though the tensorflow website 1 mentions that CUDA 10.1 is compatible with tensorflow-gpu-1.13.1, it doesn’t work so far. tensorflow-gpu gets installed properly though but it throws out weird errors when running.

So far, the best configuration to run tensorflow with GPU is CUDA 9.0 with tensorflow_gpu-1.12.0 under python3.6.

Following this configuration with the steps mentioned in https://stackoverflow.com/a/51307381/2562870 (the answer above), worked for me 🙂

The ‘new’ way to install tensorflow GPU if you have Nvidia, is with Anaconda. Works on Windows too. With 1 line.

conda create --name tf_gpu tensorflow-gpu 

This is a shortcut for 3 commands, which you can execute separately if you want or if you already have a conda environment and do not need to create one.

1. Create an anaconda environment conda create --name tf_gpu

2. Activate the environment conda activate tf_gpu

3. Install tensorflow-GPU conda install tensorflow-gpu

You can use the conda environment.

Uninstall tensorflow and install only tensorflow-gpu; this should be sufficient. By default, this should run on the GPU and not the CPU. However, further you can do the following to specify which GPU you want it to run on.

If you have an nvidia GPU, find out your GPU id using the command nvidia-smi on the terminal. After that, add these lines in your script:

os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = #GPU_ID from earlier

config = tf.ConfigProto()
sess = tf.Session(config=config)

For the functions where you wish to use GPUs, write something like the following:

with tf.device(tf.DeviceSpec(device_type="GPU", device_index=gpu_id)):

Follow the steps in the latest version of the documentation. Note: GPU and CPU functionality is now combined in a single tensorflow package

pip install tensorflow

# OLDER VERSIONS pip install tensorflow-gpu

https://www.tensorflow.org/install/gpu

This is a great guide for installing drivers and CUDA if needed:
https://www.quantstart.com/articles/installing-tensorflow-22-on-ubuntu-1804-with-an-nvidia-gpu/

For conda environment.

• conda search tensorflow to search the available versions of tensorflow. The ones that have mkl are optimized for CPU. You can choose the ones with gpu.
• Then check the version of your cuda using nvcc --version and find the proper version of tensorflow in this page, according to your version of cuda.
• For example, for cuda/10.1,and python3.8, you can use
  • conda install tensorflow=2.2.0=gpu_py38hb782248_0

There are 2 steps:

1. Update your graphics driver to latest and proprietary.(not open source)
2. Run the simple script I have created to Create conda virtual environment and Download all necessary requirements Script here

or

    echo 'Name of the TENSORFLOW ENVIRONMENT:'
    read ENVNAME

    #CREATING THE ENV
    conda create --name $ENVNAME -y

    #ACTIVATE THE eNV
    conda activate $ENVNAME6

    # INSTALLING CUDA DRIVERS
    conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0 -y

    # INSTALLING TENSORFLOW
    conda install tensorflow-gpu -y
    conda install -c anaconda ipykernel -y
    conda install ipykernel -y

    # ADDING ENV TO JUPYTER LIST
    python3 -m ipykernel install --user --name=$ENVNAME

    # 'VERIFY GPU SUPPORT'
    python3 -c "import tensorflow as tf; 
    print(tf.config.list_physical_devices('GPU'))"

On my computer, I have installed only NVIDIA graphics card drivers (only for the display, not CUDA and CUDNN). I followed your instructions, i.e.

echo 'Name of the TENSORFLOW ENVIRONMENT:'
read ENVNAME

#CREATING THE ENV
conda create --name $ENVNAME -y

#ACTIVATE THE eNV
conda activate $ENVNAME6

# INSTALLING CUDA DRIVERS
conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0 -y

# INSTALLING TENSORFLOW
conda install tensorflow-gpu -y
conda install -c anaconda ipykernel -y
conda install ipykernel -y

# ADDING ENV TO JUPYTER LIST
python3 -m ipykernel install --user --name=$ENVNAME

# 'VERIFY GPU SUPPORT'
python3 -c "import tensorflow as tf; 
print(tf.config.list_physical_devices('GPU'))"

But I am getting back the following message:

>python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

Output> []

Does that mean that my python can’t see my Graphic Card?

At this point it’s worth mentioning that my graphics card is an NVIDIA geforce gtx 560, and on the NVIDIA site it says the compatible cards are "geforce gtx 560 TI, geforce gtx 560M". Does this mean my graphics card is not CUDA compatible, and if so why when I install numba and run the following code it seems to work:

from numba import jit, cuda
import numpy as np
# to measure exec time
from timeit import default_timer as timer   

# normal function to run on cpu
def func(a):                                
    for i in range(10000000):
        a[i]+= 1      

# function optimized to run on gpu 
@jit(target_backend='cuda')                         
def func2(a):
    for i in range(10000000):
        a[i]+= 1
if __name__=="__main__":
    n = 10000000                            
    a = np.ones(n, dtype = np.float64)
    
    start = timer()
    func(a)
    print("without GPU:", timer()-start)    
    
    start = timer()
    func2(a)
    print("with GPU:", timer()-start)