4.2 Cache Memory Principles

COS2621 - Computer Organisation · 4. Cache Memory

Cache Memory Principles

Overview of Cache Memory

Cache memory is a small, fast type of volatile memory located between the CPU and main memory (RAM). Its primary purpose is to speed up data access for the processor by storing frequently accessed data and instructions. This reduces the time required to retrieve data from slower main memory.

Cache Memory Operation

When the CPU needs to access data, it first checks the cache. If the data is found, this is called a cache hit. If the data is not found, it is termed a cache miss, and the CPU must fetch the data from the slower main memory. The efficiency of cache memory is measured by the hit ratio, calculated as follows:

Hit Ratio = (Number of Cache Hits) / (Total Number of Accesses)

For example, if a CPU makes 10,000 accesses to the cache and experiences 7,500 hits, the hit ratio would be:

Hit Ratio = 7500 / 10000 = 0.75 (or 75%)

Cache Memory Types

There are different levels of cache memory:

  • L1 Cache: The fastest and smallest cache, located inside the CPU. It stores the most frequently used data and instructions.
  • L2 Cache: Larger than L1 but slower, acting as a bridge between the CPU and main memory.
  • L3 Cache: Even larger and typically shared among multiple CPU cores. It is slower than L2 cache.

Cache Lines and Blocks

A cache line is the minimum unit of transfer between cache and main memory. Each cache line consists of a tag and a block. The tag is used for addressing purposes, allowing the cache to identify which data is stored in that line. The block describes the chunk of physical memory transferred.

Cache Mapping Techniques

Cache mapping determines how data is stored in the cache. The simplest technique is direct mapping, where each block of main memory maps to exactly one cache line. Other techniques include:

  • Associative Mapping: Any block of main memory can be stored in any cache line, allowing for greater flexibility but requiring more complex hardware.
  • Set-Associative Mapping: Combines direct and associative mapping, where the cache is divided into sets, each containing multiple cache lines.

Watch out: In direct mapping, if two blocks map to the same cache line, one will replace the other, potentially lowering the hit ratio.

Cache Replacement Policies

When a cache miss occurs and the cache is full, a replacement policy decides which data to remove. Common policies include:

  • Least Recently Used (LRU): Replaces the least recently accessed data.
  • First-In, First-Out (FIFO): Replaces the oldest data in the cache.
  • Random Replacement: Randomly selects a cache line to replace.

Cache Coherence Policies

Cache coherence ensures that multiple caches in a system maintain a consistent view of memory. The inclusive policy guarantees that data in one cache is also found in all lower levels of caches, while the exclusive policy ensures that data in one cache is not found in all lower levels.

Example Calculation

Consider a cache with 512 lines and a hit ratio of 85%. If the CPU makes 15,000 accesses, the number of cache hits can be calculated as follows:

Cache Hits = Hit Ratio × Total Accesses = 0.85 × 15000 = 12750 hits.

Thus, there are 12,750 cache hits and 2,250 cache misses.

Check your understanding

  • What is the purpose of cache memory?
  • Define cache hit and cache miss.
  • Explain the difference between direct mapping and associative mapping.
  • What is the FIFO policy in cache replacement?
  • A logical cache stores data using __________.

Test yourself

5 quick questions on this lesson. Pick an answer to see if you're right.

  1. 1. Briefly describe a typical memory hierarchy. What happens as you move up the hierarchy? What happens as you go down the hierarchy?

  2. 2. __________ flushes the internal cache memory after writing dirty lines to memory

  3. 3. What does a cache hit indicate?

  4. 4. Which type of locality is exploited when recently accessed memory locations are likely to be accessed again soon?

  5. 5. Draw a suitable diagram for a typical memory hierarchy.