CS 208 w20 lecture 18 outline

Which pair of address conflict?

• each address maps to exactly one set
• each set can store a block in more than one way (in more than one line)

• capacity of a cache (\(C\)) is number of sets (\(S\)) times the number of lines per set (\(E\)) times the block size (\(B\), number of bytes stored in a line)
  • \(C = S \times E \times B\)

• a cache has an array of \(S = 2^s\) cache sets, each of which contain \(E\) blocks or cache lines
• each line consists of a valid bit to indicate whether the line contains meaningful information, \(t=m-(b+s)\) tag bits to indentify a block within the cache line, and a data block of \(B=2^b\) bytes
  • management bits (valid bit and tag) plus block data
• hence, the cache organization is parameterized by the tuple \((S, E, B)\) with a capacity (excluding tag and valid bits) of \(C = S\times E\times B\)
• instruction to load m-bit address \(A\) from memory, CPU sends \(A\) to the cache
  • similar to a hash table with a simple hash function, the cache just inspects the bits of \(A\) to determine if it is present
  • \(S\) and \(B\) define the partition of the bits of \(A\)
    • \(s\) set bits index into the array of \(S\) sets
    • \(t\) tag bits indicate which line (if any) within the set contains the target word
      • a line contains the target word if and only if the valid bit is set and the tag bits match
    • \(b\) block offset bits give the offset of the word in the $B$-byte data block

• spreadsheet example (8 byte blocks, accessing int)
  • use \(s\) bits to find set
  • valid? + tag match = hit
    • no match? old line gets envicted and overwritten
  • use block offset to find starting byte to read int
  • what if we were reading a long? we'd need bytes past the end of the block
    • would be slow and messy to read across block boundaries
    • this is why we want everything aligned to an address that's a multiple of its size!

• when there is one line per set (\(E=1\)), the cache is a direct-mapped cache
• three cache resolution steps: set selection, line matchting, word extraction

• spreadsheet example (8 byte blocks, accessing short)
  • use \(s\) bits to find set
  • valid? + tag match = hit (have to compare to both tags)
    • no match? one line is selected for evictiong (random, LRU, etc.)
  • use block offset to find starting byte to read short

• A cache with \(1 < E < C/B\) is called an $E$-way associative cache

• when a single set contains all the cache lines (\(E = C/B\))
• difficult to scale line matching, so only appropriate for small caches

• writing to a cached word is a write hit
  • what happens after the cache updates its copy?
    • write-through: immediately write block containing the update to the next lower level (simple, but causes bus traffic with every write)
    • write-back: defer further writes until updated block is evicted (reduces bus traffic, but cache must maintain a dirty bit to track which lines have been written to)
• miss policies:
  • write-allocate: load the block, then write (usually goes with write-back)
  • no-write-allocate: bypass, write directly to next lower level (usually goes with write-through)

• cache-friendly practices: repeated references to local variables (can store in registers) and stride-1 reference patterns (caches store contiguous blocks)
• minimize the number of cache misses in each inner loop

• miss rate: fraction of memory references that result in a cache miss
• hit rate: fraction of memory references that result in a cache hit (\(1 - miss rate\))
• hit time: time to deliver a resident word to the CPU
• miss penalty: additional time required due to a miss