CS 332 w22 — Deadlock

Table of Contents

1 Dining Philosophers

  • Round table with n philosophers, n plates, and n chopsticks
  • A philosopher needs two chopsticks in order to eat
  • Suppose each philosopher takes left chopstick, then right, eats, then puts chopsticks down
    • Deadlock! (and literal starvation)

dining-philosophers.png

diningDeadlock.png

2 Definitions

  • Resource: any (passive) thing needed by a thread to do its job (CPU, disk space, memory, lock)
    • Preemptable: can be taken away by OS
    • Non-preemptable: must leave with thread
  • Starvation: thread waits indefinitely
  • Deadlock: circular waiting for resources
    • Deadlock → starvation, but not vice versa

3 Deadlock by…

3.1 Recurse Locking

Thread A                             Thread B

lock_acquire(&lock1);                lock_acquire(&lock2);
lock_acquire(&lock2);                lock_acquire(&lock1);
lock_release(&lock2);                lock_release(&lock1);
lock_release(&lock1);                lock_release(&lock2);

3.2 Nested Waiting

Thread A                             Thread B

lock_acquire(&lock1);                lock_acquire(&lock1);
...                                  ...
lock_acquire(&lock2);                lock_acquire(&lock2);
while(need to wait) {                ...
    condvar_wait(&cv, &lock2);       condvar_signal(&cv);
}                                    ...
lock_release(&lock2);                lock_release(&lock1);
...                                  ...
lock_release(&lock1);                lock_release(&lock2);

3.3 Full Buffer

Doesn't have to be locks! Suppose buffer1 and buffer2 are bounded, blocking buffers that start almost full. 

Thread A                             Thread B

buffer_put(&buffer1, data);          buffer_put(&buffer2, data);
buffer_put(&buffer1, data);          buffer_put(&buffer2, data);

buffer_get(&buffer2, dest);          buffer_put(&buffer1, dest);
buffer_get(&buffer2, dest);          buffer_put(&buffer1, dest);

4 Necessary Conditions for Deadlock

  • Limited access to resources
    • If infinite resources, no deadlock!
  • No preemption
    • If resources are preemptable, can break deadlock
  • Hold and Wait
    • Threads don’t voluntarily give up resources
  • Circular chain of requests

5 Preventing Deadlock

  • Make sure at least one of the four conditions can't hold by
    • Exploit or limit program behavior
      • Limit program from doing anything that might lead to deadlock
      • Eliminate wait while holding
        • Release the lock when calling out of the current module
          • Avoids waiting in code that doesn’t know about held locks
          • Can be cumbersome in practice
        • Acquire resources all at once or not at all
          • Immediately attempt to acquire all required resources
          • If any are unavailable, release everything acquired so far and try again later
          • Could lead to starvation if one or more of the resources has a lot of contention
          • If philosopher can’t get both chopsticks, sets down any they’ve picked up and tries again
          • Better: wait until two chopsticks available, atomically acquire both
        • Acquire resources in a fixed order (lock ordering)
          • Each lock/resource has a number, always acquire in ascending order
          • Moving file from one directory to another, needs to get the locks for both directories
            • Is deadlock possible? Another move operation, each holds the lock for one directory
            • Always acquire the lock for the directory with the lower inode number prevents this
    • Predict the future
      • If we know what program will do, we can tell if granting a resource might lead to deadlock
    • Detect and recover
      • If we can rollback a thread, we can fix a deadlock once it occurs
  • Limited resources? Provide enough resources
    • Dining philosophers: how many chopsticks are enough?1

6 Banker's Algorithm Pseudocode

An example of the Banker's Algorithm in the context of a generic Resource Manager class. Here is the state of the system, where we track (i) the current allocation of each resource to each thread, (ii) the maximum allocation possible for each thread, and (iii) the current set of available, unallocated resources: 

struct resource_mgr {
    Lock lock;
    CV cv;
    int r;          // Number of resources
    int t;          // Number of threads
    int avail[];    // avail[i]: instances of resource i available
    int max[][];    // max[i][j]: max of resource i needed by thread j
    int alloc[][];  // alloc[i][j]: current allocation of resource i to thread j
  ...
}

The code for processing a resource request: 

// Invariant: the system is in a safe state.
mgr_request(struct resource_mgr *mgr, int resource_id, int thread_id) {
    lock_acquire(&mgr->lock);
    assert(mgr_isSafe(mgr));
    while (!mgr_wouldBeSafe(mgr, resource_id, thread_id)) {
        condvar_wait(&mgr->cv, &mgr->lock);
    }
    mgr->alloc[resource_id][thread_id]++;
    mgr->avail[resource_id]--;
    assert(mgr_isSafe(mgr));
    lock_release(&mgr->lock);
}

mgr_isSafe() and mgr_wouldBeSafe() are the key to preventing deadlock (pseudocode): 

// A state is safe iff there exists a safe sequence of grants that are sufficient 
// to allow all threads to eventually receive their maximum resource needs.

bool
mgr_isSafe(struct resource_mgr *mgr) {
    int j;
    int toBeAvail[] = copy mgr->avail[];
    int need[][] = mgr->max[][] - mgr->alloc[][];  // need[i][j] is initialized to 
                                                   // max[i][j] - alloc[i][j]
    bool finish[] = [false, false, false, ...]; // finish[j] is true
                                                // if thread j is guaranteed to finish

    while (true) {
        j = any thread_id such that:
             (finish[j] == false) && forall i: need[i][j] <= toBeAvail[i];
        if (no such j exists) {
            if (forall j: finish[j] == true) {
                return true;
            } else {
                return false;
            }
        } else {  // Thread j will eventually finish and return its 
                  // current allocation to the pool.
            finish[j] = true;
            forall i:  toBeAvail[i] = toBeAvail[i] + mrg->alloc[i][j];
        }
    }
}

// Hypothetically grant request and see if resulting state is safe.

bool 
mgr_wouldBeSafe(struct resource_mgr *mgr, int resource_id, int thread_id) {  
    bool result = false;       

    mgr->avail[resource_id]--;
    mgr->alloc[resource_id][thread_id]++;
    if (mgr_isSafe(mgr)) {
        result = true;
    }
    mgr->avail[resource_id]++;
    mgr->alloc[resource_id][thread_id]--;
    return result;
}

6.1 Example

  • 8 pages of memory
  • process A needs 4, process B needs 5, process C needs 5
  • if we hand them out one at a time to each process in order, we will deadlock
Process                        
A 0 1 1 1 2 2 2 3 3 3 wait wait
B 0 0 1 1 1 2 2 2 3 3 3 wait
C 0 0 0 1 1 1 2 2 2 wait wait wait
Total 0 1 2 3 4 5 6 7 8 8 8 8
  • Use the Banker's Algorithm to delay unsafe requests:
Process                                      
A 0 1 1 1 2 2 2 3 3 3 4 0 0 0 0 0 0 0 0
B 0 0 1 1 1 2 2 2 wait wait wait wait 3 4 4 5 0 0 0
C 0 0 0 1 1 1 2 2 2 wait wait wait 3 3 wait wait 4 5 0
Total 0 1 2 3 4 5 6 7 7 7 8 4 6 7 7 8 4 5 0

7 Coffman et al.'s Test for Deadlock

// A state is safe iff there exists a safe sequence of grants that would allow 
// all threads to eventually receive their maximum resource needs.
//
// avail[] holds free resource count 
// alloc[][] holds current allocation
// request[][] holds currently-blocked requests
bool
mgr_isDeadlocked(struct resource_mgr *mgr) {
    int j;
    int toBeAvail[] = copy mgr->avail[];
    bool finish[] = [false, false, false, ...]; // finish[j] is true if thread 
                                                // j is guaranteed to finish

    while(true) {
        j = any thread_id such that (finish[j] == false) && 
                                (forall i: request[i][j] <= toBeAvail[i]);
        if (no such j exists) {
            if (forall j: finish[j] == true) {
                return false;
            } else {
                return true;
            }
        } else {
            // Thread j *may* eventually finish and 
            // return its current allocation to the pool.
            finish[j] = true;
            forall i: toBeAvail[i] = toBeAvail[i] + mgr->alloc[i][j];
        }
    }
}

8 Reading: Dining Philosophers and Deadlock

Read OSTEP section 31.6, (p. 401-404) on the dining philosophers problem and OSTEP section 32.3, (p. 413–422) about deadlock. The former discusses the problem in the context of semaphores, and the latter gives a good overview of the conditions needed for deadlock and some possible solutions including a lock-free approach not covered in the video.

Footnotes:

1

Add one in the middle of the table that anyone can grab