Cuda Driver Runtime

导言

Divide the bulky and outdated content about cuda runtime env into individual posts, ensuring both the thematic integrity and a balanced size for each blog entry.

设备参数

Cuda Version & GPU Version

在 CMakeLists.txt里设置 set (CMAKE_CUDA_ARCHITECTURES 61)可用的最大版本号以获得最好的驱动支持。ref

max block & max thread

通过cuda-samples程序，我们可以profile，GPU的基本参数细节。

# 下载对应nvcc对应的cuda version的版本
git clone https://github.com/NVIDIA/cuda-samples.git
cd
make -j16

# shaojiemike @ snode0 in ~/github/cuda-samples-11.0 [23:08:29]
$ ./bin/x86_64/linux/release/deviceQuery
./bin/x86_64/linux/release/deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 7 CUDA Capable device(s)

Device 0: "Tesla P40"
  CUDA Driver Version / Runtime Version          11.4 / 11.0
  CUDA Capability Major/Minor version number:    6.1
  Total amount of global memory:                 22919 MBytes (24032378880 bytes)
  (30) Multiprocessors, (128) CUDA Cores/MP:     3840 CUDA Cores
  GPU Max Clock rate:                            1531 MHz (1.53 GHz)
  Memory Clock rate:                             3615 Mhz
  Memory Bus Width:                              384-bit
  L2 Cache Size:                                 3145728 bytes (3 Gbytes)
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes (64 Kbytes)
  Total amount of shared memory per block:       49152 bytes (48 Kbytes)
  Total shared memory per multiprocessor(SM):    98304 bytes (96 Kbytes)
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes (2 Gbytes)
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 2 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Enabled
  Device supports Unified Addressing (UVA):      Yes
  Device supports Managed Memory:                Yes
  Device supports Compute Preemption:            Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   0 / 4 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

核心Pascal GP102

各种参数什么意思？

1. Texture和贴图有关？
2. global memory 显存
3. Constant memory: 为特殊的read-only不变量存储来加速，当所有线程同时访问相同的值时，固定内存也是最有效的。
4. Texture memory：同理为read-only贴图资源，最初是为OpenGL和DirectX渲染设计的

CUDA环境的常见问题

Install CUDA Toolkit without sudo

1. Download your runfile according to your OS（
   lsb_release -a unname -a） in here(https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=20.04&target_type=runfile_local).
2. Run md5sum on your run file to make sure it is not corrupted. The correct checksum is on your CUDA download page. Note, somehow, this file is easily being corrupted. Make sure to check it.
3. Execute the runfile with the --toolkitpath option, where the path is where you would like the toolkit to sit on. Thus, there is no root requirement. –toolkit is to only install CUDA toolkit (no driver). The --override option might not be needed but if there is warning you might want to turn it on.
   bash cuda_10.0.130_410.48_linux --silent --override --toolkit --toolkitpath=$HOME/Install/cuda10
4. In your bashrc or zshrc file, specify the three PATHs

PATH=/usr/local/cuda/bin:$PATH
CPATH=/usr/local/cuda/include:$CPATH 
LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Install pre CUDA Toolkit

https://developer.nvidia.com/cuda-toolkit-archive

https://zhuanlan.zhihu.com/p/95939378

Failed to initialize NVML: Driver/library version mismatch

driver version VS runtime version?

cuda有两套主要的API，

一套是 the driver API (e.g. libcuda.so on linux and nvidia-smi) is installed by the GPU driver installer. 识别GPU硬件的驱动

另一套是 the runtime API (e.g. libcudart.so on linux, and also nvcc) is installed by the CUDA toolkit installer (which may also have a GPU driver installer bundled in it). 提供cuda编程的各种常用函数库和接口

关系：

1. 两者不是必须一致。
2. CUDA Driver Version应该是跟着GPU驱动走的，Runtime Version取决于当前设置。Driver Version一般 >= Runtime Version, 否则insufficient。
3. 软件运行时调用的应该是Runtime Version。

check driver version VS runtime version

# runtime version
nvcc -V
cat /usr/local/cuda/version.txt
# driver version
nvidia-smi
cat /proc/driver/nvidia/version
modinfo nvidia|grep version:

how to download driver version

windows

参考文献