Cache Memory

Cache Memory

Cache memory is a small-sized type of volatile computer memory that provides high-speed data access to the processor. It stores frequently accessed data and instructions to improve the overall speed of the computer. Cache memory is faster than main memory (RAM) and is used to reduce the time it takes to access data from the main memory.

Memory Hierarchy

Cache memory is part of the memory hierarchy in a computer system. The memory hierarchy is structured in levels, with each level having different speed, size, and cost characteristics. The levels are:

• Registers: Located inside the CPU, these are the fastest and smallest memory units.
• Cache Memory: This is faster than RAM but smaller in size.
• Main Memory (RAM): This is larger than cache memory but slower.
• Secondary Storage: This includes hard drives and SSDs, which are slower and larger than RAM.

Types of Cache Memory

There are three main types of cache memory:

1. L1 Cache: This cache is built into the processor. It is the fastest type of cache and has the smallest size, typically ranging from 16 KB to 128 KB.
2. L2 Cache: This cache is larger than L1 cache, typically ranging from 256 KB to several megabytes. It is slower than L1 cache but still faster than main memory.
3. L3 Cache: This cache is shared among multiple cores in multi-core processors. It is larger than L2 cache, often several megabytes, but also slower.

How Cache Memory Works

Cache memory works on the principle of locality of reference. This principle states that programs tend to access a relatively small portion of their address space at any given time. There are two types of locality:

• Temporal Locality: If a particular data item is accessed, it is likely to be accessed again soon.
• Spatial Locality: If a data item is accessed, nearby data items are likely to be accessed soon.

When the CPU needs to access data, it first checks the cache memory. If the data is found in the cache (this is called a cache hit), it is accessed quickly. If the data is not found (a cache miss), the CPU retrieves it from the main memory, which takes longer.

Cache Miss Types

There are three types of cache misses:

1. Compulsory Miss: This occurs when data is accessed for the first time and is not present in the cache.
2. Capacity Miss: This occurs when the cache cannot store all the data needed by the CPU, leading to some data being evicted.
3. Conflict Miss: This occurs when multiple data items compete for the same cache line due to the cache’s limited associativity.

Cache Organisation

Cache memory can be organised in several ways:

• Direct-Mapped Cache: Each block of main memory maps to exactly one cache line. This is simple but can lead to conflict misses.
• Fully Associative Cache: Any block of main memory can be stored in any cache line. This reduces conflict misses but is more complex and expensive.
• Set-Associative Cache: This is a compromise between direct-mapped and fully associative caches. Each block maps to a set of cache lines, allowing some flexibility while maintaining simplicity.

Cache Replacement Policies

When a cache miss occurs and the cache is full, a replacement policy determines which cache line to evict. Common replacement policies include:

1. Least Recently Used (LRU): This policy evicts the least recently accessed item, assuming it is less likely to be used again soon.
2. First-In, First-Out (FIFO): This policy evicts the oldest item in the cache, regardless of how often it has been accessed.
3. Random Replacement: This policy randomly selects an item to evict, which can sometimes be effective but is less predictable.

Cache Performance Metrics

To evaluate the performance of cache memory, several metrics are used:

• Hit Rate: The percentage of memory accesses that result in a cache hit. A higher hit rate indicates better cache performance.
• Miss Rate: The percentage of memory accesses that result in a cache miss. This is calculated as 1 minus the hit rate.
• Access Time: The time taken to access data from the cache compared to main memory.

Example of Cache Memory Calculation

Let’s say a CPU has a cache with a hit rate of 80%. If the CPU makes 100 memory accesses, we can calculate the number of cache hits and misses:

Hit Rate = 80% = 0.80

Calculating the number of hits:

Number of Hits = Total Accesses × Hit Rate
Number of Hits = 100 × 0.80 = 80

Calculating the number of misses:

Miss Rate = 1 - Hit Rate
Miss Rate = 1 - 0.80 = 0.20
Number of Misses = Total Accesses × Miss Rate
Number of Misses = 100 × 0.20 = 20

In this example, there are 80 cache hits and 20 cache misses.

Cache Memory in Modern Computers

Modern computers typically use multiple levels of cache (L1, L2, and L3) to optimise performance. Each level has different sizes and speeds, allowing the CPU to access data quickly. The use of cache memory is crucial for high-performance computing, gaming, and applications that require fast data access.

Summary

• Cache memory is a high-speed memory that stores frequently accessed data.
• It is part of the memory hierarchy, which includes registers, cache, RAM, and secondary storage.
• Cache memory types include L1, L2, and L3 caches.
• Cache misses can be compulsory, capacity, or conflict misses.
• Cache organisation includes direct-mapped, fully associative, and set-associative caches.
• Replacement policies determine how cache lines are evicted.
• Performance metrics include hit rate, miss rate, and access time.

Check your understanding

1. What is cache memory and why is it important?
2. Explain the difference between L1, L2, and L3 cache.
3. What are the three types of cache misses?
4. Describe the Least Recently Used (LRU) replacement policy.