How to tell if tensorflow is using gpu acceleration from inside python shell?

Question:

I have installed tensorflow in my ubuntu 16.04 using the second answer here with ubuntu’s builtin apt cuda installation.

Now my question is how can I test if tensorflow is really using gpu? I have a gtx 960m gpu. When I import tensorflow this is the output

I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally

Is this output enough to check if tensorflow is using gpu ?

Asked By: Tamim Addari

||

Answers:

No, I don’t think "open CUDA library" is enough to tell, because different nodes of the graph may be on different devices.

When using tensorflow2:

print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))

For tensorflow1, to find out which device is used, you can enable log device placement like this:

sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

Check your console for this type of output.

Answered By: Yao Zhang

This will confirm that tensorflow using GPU while training also ?

Code

sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

Output

I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties: 
name: GeForce GT 730
major: 3 minor: 5 memoryClockRate (GHz) 0.9015
pciBusID 0000:01:00.0
Total memory: 1.98GiB
Free memory: 1.72GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GT 730, pci bus id: 0000:01:00.0)
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GT 730, pci bus id: 0000:01:00.0
I tensorflow/core/common_runtime/direct_session.cc:255] Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GT 730, pci bus id: 0000:01:00.0
Answered By: himanshurobo

Apart from using sess = tf.Session(config=tf.ConfigProto(log_device_placement=True)) which is outlined in other answers as well as in the official TensorFlow documentation, you can try to assign a computation to the gpu and see whether you have an error.

import tensorflow as tf
with tf.device('/gpu:0'):
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)

with tf.Session() as sess:
    print (sess.run(c))

Here

  • “/cpu:0”: The CPU of your machine.
  • “/gpu:0”: The GPU of your machine, if you have one.

If you have a gpu and can use it, you will see the result. Otherwise you will see an error with a long stacktrace. In the end you will have something like this:

Cannot assign a device to node ‘MatMul’: Could not satisfy explicit
device specification ‘/device:GPU:0’ because no devices matching that
specification are registered in this process


Recently a few helpful functions appeared in TF:

You can also check for available devices in the session:

with tf.Session() as sess:
  devices = sess.list_devices()

devices will return you something like

[_DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:CPU:0, CPU, -1, 4670268618893924978),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 6127825144471676437),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:XLA_GPU:0, XLA_GPU, 17179869184, 16148453971365832732),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:0, TPU, 17179869184, 10003582050679337480),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:1, TPU, 17179869184, 5678397037036584928)
Answered By: Salvador Dali

I prefer to use nvidia-smi to monitor GPU usage. if it goes up significantly when you start you program, it’s a strong sign that your tensorflow is using GPU.

Answered By: scott huang

Run the following in Jupyter,

import tensorflow as tf
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

If you’ve set up your environment properly, you’ll get the following output in the terminal where you ran “jupyter notebook”,

2017-10-05 14:51:46.335323: I c:tf_jenkinshomeworkspacerelease-winmwindows-gpupy35tensorflowcorecommon_runtimegpugpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Quadro K620, pci bus id: 0000:02:00.0)
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Quadro K620, pci bus id: 0000:02:00.0
2017-10-05 14:51:46.337418: I c:tf_jenkinshomeworkspacerelease-winmwindows-gpupy35tensorflowcorecommon_runtimedirect_session.cc:265] Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Quadro K620, pci bus id: 0000:02:00.0

You can see here I’m using TensorFlow with an Nvidia Quodro K620.

Answered By: wafflecat

This should give the list of devices available for Tensorflow (under Py-3.6):

tf = tf.Session(config=tf.ConfigProto(log_device_placement=True))
tf.list_devices()
# _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 268435456)
Answered By: f0nzie

I find just querying the gpu from the command line is easiest:

nvidia-smi

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.98                 Driver Version: 384.98                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 980 Ti  Off  | 00000000:02:00.0  On |                  N/A |
| 22%   33C    P8    13W / 250W |   5817MiB /  6075MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1060      G   /usr/lib/xorg/Xorg                            53MiB |
|    0     25177      C   python                                      5751MiB |
+-----------------------------------------------------------------------------+

if your learning is a background process the pid from
jobs -p should match the pid from nvidia-smi

Answered By: Tim

Following piece of code should give you all devices available to tensorflow.

from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())

Sample Output

[name: “/cpu:0”
device_type: “CPU”
memory_limit: 268435456
locality {
}
incarnation: 4402277519343584096,

name: “/gpu:0”
device_type: “GPU”
memory_limit: 6772842168
locality {
bus_id: 1
}
incarnation: 7471795903849088328
physical_device_desc: “device: 0, name: GeForce GTX 1070, pci bus id: 0000:05:00.0”
]

Answered By: Sheraz

I think there is an easier way to achieve this.

import tensorflow as tf
if tf.test.gpu_device_name():
    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))
else:
    print("Please install GPU version of TF")

It usually prints like

Default GPU Device: /device:GPU:0

This seems easier to me rather than those verbose logs.

Edit:-
This was tested for TF 1.x versions. I never had a chance to do stuff with TF 2.0 or above so keep in mind.

Answered By: Ishan Bhatt

Ok, first launch an ipython shell from the terminal and import TensorFlow:

$ ipython --pylab
Python 3.6.5 |Anaconda custom (64-bit)| (default, Apr 29 2018, 16:14:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.
Using matplotlib backend: Qt5Agg

In [1]: import tensorflow as tf

Now, we can watch the GPU memory usage in a console using the following command:

# realtime update for every 2s
$ watch -n 2 nvidia-smi

Since we’ve only imported TensorFlow but have not used any GPU yet, the usage stats will be:

tf non-gpu usage

Notice how the GPU memory usage is very less (~ 700MB); Sometimes the GPU memory usage might even be as low as 0 MB.


Now, let’s load the GPU in our code. As indicated in tf documentation, do:

In [2]: sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

Now, the watch stats should show an updated GPU usage memory as below:

tf gpu-watch

Observe now how our Python process from the ipython shell is using ~ 7 GB of the GPU memory.


P.S. You can continue watching these stats as the code is running, to see how intense the GPU usage is over time.

Answered By: kmario23

In addition to other answers, the following should help you to make sure that your version of tensorflow includes GPU support.

import tensorflow as tf
print(tf.test.is_built_with_cuda())
Answered By: karaspd

You can check if you are currently using the GPU by running the following code:

import tensorflow as tf
tf.test.gpu_device_name()

If the output is '', it means you are using CPU only;
If the output is something like that /device:GPU:0, it means GPU works.


And use the following code to check which GPU you are using:

from tensorflow.python.client import device_lib 
device_lib.list_local_devices()
Answered By: Hu Xixi

Tensorflow 2.0

Sessions are no longer used in 2.0. Instead, one can use tf.test.is_gpu_available:

import tensorflow as tf

assert tf.test.is_gpu_available()
assert tf.test.is_built_with_cuda()

If you get an error, you need to check your installation.

Answered By: ma3oun

Put this near the top of your jupyter notebook. Comment out what you don’t need.

# confirm TensorFlow sees the GPU
from tensorflow.python.client import device_lib
assert 'GPU' in str(device_lib.list_local_devices())

# confirm Keras sees the GPU (for TensorFlow 1.X + Keras)
from keras import backend
assert len(backend.tensorflow_backend._get_available_gpus()) > 0

# confirm PyTorch sees the GPU
from torch import cuda
assert cuda.is_available()
assert cuda.device_count() > 0
print(cuda.get_device_name(cuda.current_device()))

NOTE: With the release of TensorFlow 2.0, Keras is now included as part of the TF API.

Originally answerwed here.

Answered By: Paul Williams

With the recent updates of Tensorflow, you can check it as follow :

tf.test.is_gpu_available( cuda_only=False, min_cuda_compute_capability=None)

This will return True if GPU is being used by Tensorflow, and return False otherwise.

If you want device device_name you can type : tf.test.gpu_device_name().
Get more details from here

Answered By: smerllo

This is the line I am using to list devices available to tf.session directly from bash:

python -c "import os; os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'; import tensorflow as tf; sess = tf.Session(); [print(x) for x in sess.list_devices()]; print(tf.__version__);"

It will print available devices and tensorflow version, for example:

_DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 268435456, 10588614393916958794)
_DeviceAttributes(/job:localhost/replica:0/task:0/device:XLA_GPU:0, XLA_GPU, 17179869184, 12320120782636586575)
_DeviceAttributes(/job:localhost/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 13378821206986992411)
_DeviceAttributes(/job:localhost/replica:0/task:0/device:GPU:0, GPU, 32039954023, 12481654498215526877)
1.14.0
Answered By: y.selivonchyk

You have some options to test whether GPU acceleration is being used by your TensorFlow installation.

You can type in the following commands in three different platforms.

import tensorflow as tf
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
  1. Jupyter Notebook – Check the console which is running the Jupyter Notebook. You will be able to see the GPU being used.
  2. Python Shell – You will be able to directly see the output. (Note- do not assign the output of the second command to the variable ‘sess’; if that helps).
  3. Spyder – Type in the following command in the console.

    import tensorflow as tf
    tf.test.is_gpu_available()

Answered By: simpleParadox

The following will also return the name of your GPU devices.

import tensorflow as tf
tf.test.gpu_device_name()
Answered By: Maz

I found below snippet is very handy to test the gpu ..

Tensorflow 2.0 Test

import tensorflow as tf
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
with tf.device('/gpu:0'):
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)

with tf.Session() as sess:
    print (sess.run(c))

Tensorflow 1 Test

import tensorflow as tf
with tf.device('/gpu:0'):
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)

with tf.Session() as sess:
    print (sess.run(c))
Answered By: ajayramesh

If you are using TensorFlow 2.0, you can use this for loop to show the devices:

with tf.compat.v1.Session() as sess:
  devices = sess.list_devices()
devices
Answered By: Doug

if you are using tensorflow 2.x use:

sess = tf.compat.v1.Session(config=tf.compat.v1.ConfigProto(log_device_placement=True))
Answered By: Cheptii

For Tensorflow 2.0

import tensorflow as tf

tf.test.is_gpu_available(
    cuda_only=False,
    min_cuda_compute_capability=None
)

source here

other option is:

tf.config.experimental.list_physical_devices('GPU')
Answered By: ChaosPredictor

Run this command in Jupyter or your IDE to check if Tensorflow is using a GPU or not: tf.config.list_physical_devices('GPU')

Answered By: Tenzin Jinsel

UPDATE FOR TENSORFLOW >= 2.1.

The recommended way in which to check if TensorFlow is using GPU is the following:

tf.config.list_physical_devices('GPU') 

As of TensorFlow 2.1, tf.test.gpu_device_name() has been deprecated in favour of the aforementioned.

Then, in the terminal you can use nvidia-smi to check how much GPU memory has been alloted; at the same time, using watch -n K nvidia-smi would tell you for example every K seconds how much memory you are using (you may want to use K = 1 for real-time)

If you have multiple GPUs and you want to use multiple networks, each one on a separated GPU, you can use:

 with tf.device('/GPU:0'):
      neural_network_1 = initialize_network_1()
 with tf.device('/GPU:1'):
      neural_network_2 = initialize_network_2()
Answered By: Timbus Calin

With tensorflow 2.0 >=

import tensorflow as tf
sess = tf.compat.v1.Session(config=tf.compat.v1.ConfigProto(log_device_placement=True))

enter image description here

Answered By: leplandelaville

Tensorflow 2.1

A simple calculation that can be verified with nvidia-smi for memory usage on the GPU.

import tensorflow as tf 

c1 = []
n = 10

def matpow(M, n):
    if n < 1: #Abstract cases where n < 1
        return M
    else:
        return tf.matmul(M, matpow(M, n-1))

with tf.device('/gpu:0'):
    a = tf.Variable(tf.random.uniform(shape=(10000, 10000)), name="a")
    b = tf.Variable(tf.random.uniform(shape=(10000, 10000)), name="b")
    c1.append(matpow(a, n))
    c1.append(matpow(b, n))
Answered By: cannin
>>> import tensorflow as tf 
>>> tf.config.list_physical_devices('GPU')

2020-05-10 14:58:16.243814: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-05-10 14:58:16.262675: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-10 14:58:16.263119: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 1060 6GB computeCapability: 6.1
coreClock: 1.7715GHz coreCount: 10 deviceMemorySize: 5.93GiB deviceMemoryBandwidth: 178.99GiB/s
2020-05-10 14:58:16.263143: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-05-10 14:58:16.263188: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-05-10 14:58:16.264289: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-05-10 14:58:16.264495: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-05-10 14:58:16.265644: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-05-10 14:58:16.266329: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-05-10 14:58:16.266357: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-05-10 14:58:16.266478: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-10 14:58:16.266823: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-10 14:58:16.267107: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

As suggested by @AmitaiIrron:

This section indicates that a gpu was found

2020-05-10 14:58:16.263119: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:

pciBusID: 0000:01:00.0 name: GeForce GTX 1060 6GB computeCapability: 6.1
coreClock: 1.7715GHz coreCount: 10 deviceMemorySize: 5.93GiB deviceMemoryBandwidth: 178.99GiB/s

And here that it got added as an available physical device

2020-05-10 14:58:16.267107: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
Answered By: bLeDy

For TF2.4+ listed as the "official" way on tensorflow website to check if TF is using GPU or Not

>>> import tensorflow as tf
>>> print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))
Num GPUs Available:  2
Answered By: Aryan

In the new versions of TF(>2.1) the recommended way for checking whether TF is using GPU is:

tf.config.list_physical_devices('GPU')
Answered By: Aadil Srivastava

I found the most simple and comprehensive approach. Just set tf.debugging.set_log_device_placement(True) and you should see if ops are actually run on GPU e.g. Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0

More in the docs: https://www.tensorflow.org/guide/gpu#logging_device_placement

Answered By: w00dy

maybe try this:

python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

to see if the system returns the tensor

according to the site

Answered By: Chenghao Wang
Categories: questions Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.