Set GPU to "Persistent Mode"
$ nvidia-smi -i <device id> -pm ENABLED
Constantly Monitor GPU Usage
$ nvidia-smi --loop=10
Encoder and Decoder
$ nvidia-smi -q | grep Encoder
$ nvidia-smi -q | grep Decoder
Constantly Monitor GPU's Encoder and Decoder
$ nvidia-smi -q --loop=10 | grep Encoder (or Decoder)
Monitor All the Processes Running on Each Device
$ nvidia-smi pmon
Monitor GPU Temperatures
$ nvidia-smi --query-gpu=temperature.gpu --format=csv 5 -l
Check GPU Connectivity Matrix
$ nvidia-smi topo -m
Display CLOCK
$ nvidia-smi -i <device id> -q -d CLOCK
note Pay attention to
Clocks Throttle Reasons
, andClocks
.