How to increase GPU training throughput
This is a guide for increasing GPU training throughput given a batch of training run, as opposed to increasing instantaneous throughput for one training run. Obviously, doing the latter can increase training throughput, but that is not in the scope of this guide.