Description: An intermediate-to-advanced course showing users how to use GPUs most effectively on HPCs. It starts with ensuring that the user is indeed using a GPU device and that the deep learning packages were installed properly. Next, a simple training example is demonstrated on a single GPU with TensorFlow and PyTorch. These examples are then extended to multiple GPUs on a single node. Finally, the problem is scaled up to multiple nodes for both frameworks.