Quickstart

Docker environments with CUDA are available on SURF Research Cloud. Inside the workspace, TensorFlow can be installed with small tweaks that are covered in the sections below. This tutorial assumes that a running Docker workspace with CUDA already exists. The catalog item you need to start is called "Docker Environment with CUDA". 


If you are unsure how to create a workspace, please follow the general tutorial Start a simple workspace. The Docker container ships CUDA 12.8. The host driver supports a newer CUDA version than that, and NVIDIA drivers are backward-compatible. As the host CUDA is not used at all, the only host requirement is a driver new enough for the container's CUDA. 


Log in to the machine and make sure that the GPU is visible on the host 

nvidia-smi

The expected output is a table listing the NVIDIA selected GPUs and a driver version. If this fails, please proceed to the Troubleshooting section.

Check your permissions 

On SRC workspaces, your user account is typically not a member of the docker group, so plain docker commands fail with:

permission denied while trying to connect to the docker API at unix:///var/run/docker.sock

You have two options:

  • Use sudo — prefix every docker command with sudo (the commands below show this), or
  • Join the docker group — run sudo usermod -aG docker $USER, then log out and back in. After that, docker works without sudo. Note that membership of the docker group is root-equivalent on the host.

Create a test script

To verify that TensorFlow works and sees the GPU, copy the following code into a file named test_tensorflow.py in your current directory:

import tensorflow as tf

print("TensorFlow version:", tf.__version__)

# Check if a GPU is available
gpus = tf.config.list_physical_devices('GPU')
if gpus:
    print("Found GPU(s):", [gpu.name for gpu in gpus])
else:
    print("No GPU found")

# Create a constant tensor
hello = tf.constant('Hello, TensorFlow!')
print(hello.numpy().decode())

# Run a small computation and confirm which device executed it
a = tf.random.normal([1000, 1000])
b = tf.random.normal([1000, 1000])
c = tf.matmul(a, b)
print("Matrix multiplication result shape:", c.shape)
print("Executed on device:", c.device)


Note — do not use TensorFlow 1.x examples. Many older tutorials test TensorFlow with tf.Session() and tf.test.is_gpu_available(). These are TensorFlow 1.x APIs: tf.Session() was removed in TensorFlow 2 and fails with AttributeError: module 'tensorflow' has no attribute 'Session', and tf.test.is_gpu_available() is deprecated. The NGC images ship TensorFlow 2 — use tf.config.list_physical_devices('GPU') as shown above.


 Run TensorFlow on the GPU

Use the verified image below. The extra flags raise the container's shared-memory and stack limits, which NVIDIA recommends for TensorFlow — the 64 MB shared-memory default causes hard-to-diagnose crashes under load. More information: https://docs.nvidia.com/deeplearning/frameworks/tensorflow-user-guide/

sudo docker run --rm --gpus all \
    --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
    -v "$(pwd)":/workspace -w /workspace \
    nvcr.io/nvidia/tensorflow:25.02-tf2-py3 \
    python3 test_tensorflow.py

If Executed on device: ends in GPU:0, TensorFlow is installed correctly and computations run on the GPU. If it says No GPU found or the device is CPU:0, see Troubleshooting.

To run your own code, use the same command and replace test_tensorflow.py with the name of your script. Your current directory is mounted into the container, so any file in it is available under /workspace.


Flag explanation: 

  • --ulimit memlock=-1 removes the limit on locked memory, and --ulimit stack=67108864 sets a 64 MB stack. These are separate from the shared-memory setting.

  • --gpus all gives the container access to the GPU. Without it, TensorFlow will not see the GPU.
  • -v "$(pwd)":/workspace mounts your current directory into the container so your code and data are available at /workspace.


Choosing the TensorFlow version

To change the TensorFlow version, you can change the tag in

nvcr.io/nvidia/tensorflow:25.02-tf2-py3



However, do not use the latest-gpu tag. It is a rolling tag that periodically ships with a mismatched cuDNN and fails out of the box with Cannot dlopen some GPU libraries and an empty []. 


Troubleshooting

"Cannot dlopen some GPU libraries" / cuDNN error / empty []

The GPU library inside the image failed to load. If nvidia-smi works on the host and a plain CUDA container can see the GPU, the fault is the image, not the machine. 

Fix: use the verified NGC tag above rather than latest-gpu or an unpinned tag.

TensorFlow reports no GPU, but nvidia-smi works on the host

Almost always one of two things:

  1. The container was started without --gpus all. GPU access must be granted at container start. You cannot add it to an already-running container. Relaunch with the flag.
  2. The image is broken. Switch to the verified tag.

Quick isolation test — does Docker hand the GPU to any container?

sudo docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi


If the GPU table prints from inside this container, passthrough works and the problem is your TensorFlow image. If it does not, the problem is host-side, not TensorFlow.

nvidia-smi fails with "Driver/library version mismatch"

The loaded GPU kernel module and the on-disk NVIDIA libraries are different versions. This happens after a driver install or upgrade before a reboot.

Fix: reboot the machine. After reboot the kernel module matches the userspace libraries and nvidia-smi succeeds.


NVIDIA documentation

Here, you can find more information on user guides from NVIDIA

TensorFlow user guide: https://docs.nvidia.com/deeplearning/frameworks/tensorflow-user-guide/index.html 


  • No labels