4.0 Global Memory
Abstract: This chapter provides an overview of Chapter 4 on CUDA programming, introducing the main topics of study. Keywords: Global Memory, CUDA Memory Model, CUDA Memory Management, Global Memory Programming, Global Memory Access Patterns, Global Memory Data Layout, Unified Memory Programming, Improving Memory Throughput.
Global Memory
In the previous chapter, we spent the entire chapter studying CUDA's execution model. While it is true that kernel configuration within the execution model determines program efficiency, execution efficiency is not solely determined by execution structures like warps and thread blocks -- memory also has a significant impact on performance.
For example, consider an old but very fitting analogy (if it is identical to something in another book, consider it borrowed): factory production. We can increase production speed by optimizing the factory's internal assembly lines, worker allocation, and worker quality. But if you build your factory at the top of Mount Everest and your supply trucks (we are currently concerned with production output, not shipping, so we do not care about how finished products are transported out) only arrive once a year, the overall factory efficiency will be very low because the workers and assembly lines are all waiting for raw materials to arrive. This is a typical efficiency model for a GPU or CPU. Memory bandwidth and speed are also critical factors affecting throughput.
In this chapter, we will analyze the relationship between kernel functions and global memory, and their performance implications. The CUDA model is the main focus of study, achieving efficient kernel execution through different memory access patterns.