GPUs and CUDA for Parallel Computing

Introduces heterogeneous computing and the roles of CPUs, GPUs, and parallelism.

Explains threads, blocks, and kernels in the CUDA programming model.

Examines SMs, warps, and resources that shape execution on a GPU.

Connects the memory hierarchy and data locality to kernel performance.

Measures runtime and throughput to identify computing bottlenecks.

Optimizes data transfers and accesses to global, shared, and local memory.

Tunes grid and block configuration for GPU occupancy and utilization.

Integrates CUDA with PyTorch to accelerate operations in machine learning workflows.

Evaluates performance, capacity, and cost for GPU inference workloads.

Reviews key terms and concepts in GPU computing and CUDA.