Cache working and its performance

CACHE WORKING AND ITS PERFORMANCE

What is Cache?

Cache is a small and fast memory, usually static RAM (SRAM), which is incorporated inside the CPU or placed on a separate chip. The cache stores data which is frequently used in running the programs. By this, the cache increases the speed of accessing the data and the overall performance of the system.

The benefit of using cache is that the CPU does not need to use the motherboard’s system to transfer the data. Whenever the CPU has to use the system bus to access the data, this affects the effectiveness of the motherboard by slowing it down. Hence using cache overcomes this problem.

Cache is so effective that the system performance of a fast running CPU with little cache support is less than that of a slow running CPU with more cache support. The cache can be a L1 (incorporated inside the CPU) or L2 (placed in a separate chip). Some systems contain both L1 and L2 cache.

How does Cache increase the CPU Performance?

The improvement in the CPU performance by cache can be explained by the concept of “Locality of Reference”. At a time, the processor will be accessing data from a particular region of the memory. This block of memory will be stored in the cache for high speed access, hence increasing the performance.

I can further explain this by using the book analogy:

-Lots of books on my shelf: This is like the main memory.
-A few books on my desk: This is like the L2 cache.
-One book that I’m reading: This is like the L1 cache.

Hits and Misses in the Cache

When the required data that the CPU tries to fetch from the cache is found, then it’s called a cache hit and when it’s not found then it’s called a cache miss. A cache hit transfers the data to the CPU in high speed. However, if there’s a cache miss, then the CPU has to access the main memory to fetch the required data. It loads the data from the main memory to the cache which delays the execution of the instructions and lowers the performance.

Cache Mapping

There are various ways by which the main memory is mapped on to the cache:

- Direct Mapping
- Fully Associative Mapping
- Set Associative Mapping

Direct Mapping:

Each memory block is mapped to exactly one cache location. The cache location is decided by the following formula:

Cache Location = (block address) MOD (# of blocks in the cache)

Direct mapping is the most efficient type of mapping scheme; however it does not utilize the cache to the fullest (least effective), as it may not use all the cache lines. There may be a cache miss even when the cache is not full of active lines.

Fully Associative Mapping:

Each memory block is mapped to any random cache location. Here, the cache is fully utilized as none of the cache lines are left unused, but at the expense of speed. The searching of the line which contains a particular memory block is time consuming as the whole of the cache must be scanned to find that block.

illustration not visible in this excerpt

Set Associative Mapping:

Each memory block is mapped to a subset of cache location. The cache location is decided by the following formula:

Set Selection = (block address) MOD (# of sets in the cache)

This is a compromise between direct mapped and fully associative. The cache is divided into a set of tags and the set number is direct mapped from the memory address.

In set-associative mapping, when the number of lines per set is n, the mapping is called n -way associative.

How do we use the memory address to find the block?

We take the following example to explain:

- Main Memory= 16 bits=2^16=64KB
- Cache Memory=8 blocks=128 bits
- Block Size=4 words=16 bytes