CS 332 w22 — Threads

Consider an Earth Visualizer similar to Google Earth. This application lets the user virtually fly anywhere in the world, see aerial images at different resolutions, and view other information associated with each location. A key part of the design is that the user's controls are always up herbal: when the user moves the mouse to a new location, the image is redrawn in the background at successively better resolutions while the program continues to let the user adjust of you, select additional information about the location for display, or enter search terms.

To implement this application the programmer might write code to draw a portion of the screen, display user interface widgets, process user inputs, and fetch higher-resolution images for newly visible areas. In a sequential program, these functions would run in turn. With threads, they can run concurrently so that the user interface is responsive even while new data is being fetched and the screen being redrawn.

Other examples:

• Imagine a web server, which might like to handle multiple requests concurrently
  • While waiting for the credit card server to approve a purchase for one client, it could be retrieving the data requested by another client from disk, and assembling the response for a third client from cached information
• Imagine a web client (browser), which might like to initiate multiple requests concurrently
  • The Carleton CS home page has dozens of "src= …" HTML commands, each of which is going to involve a lot of sitting around! Wouldn't it be nice to be able to launch these requests concurrently?
• Imagine a parallel program running on a multiprocessor, which might like to employ "physical concurrency"
  • For example, multiplying two large matrices — split the output matrix into \(k\) regions and compute the entries in each region concurrently, using \(k\) processors

• If I want more than one thing running at a time, I already have a mechanism to achieve that
  • Processes
• What's wrong with processes?
  • IPC (inter-process communication) is slow
    • Why?¹
• What's right with processes?
  • Convenient abstraction of CPU, memory, I/O

Read Chapter 26 (p. 303–315) of the OSTEP book. It gives a concise overview of the idea of threads and how they're used. The chapter also previews the challenge we will spend the next three weeks exploring: how can we effectively and efficiently coordinate among concurrent threads?

Key terms:

• thread-local
• interleaving (when the ordering of operations from multiple threads mix together, they interleave—a given potential ordering is called an interleaving)
• race condition or data race
• critical section
• mutual exclusion
• atomic operation

Different aspects of this diagram:

• one thread per process: a simple, single-threaded application running in user mode, using system calls to ask the kernel to perform privileged operations
• many threads per process: a process has multiple concurrent threads, all executing in user mode. At any given time a subset of these threads may be running while the rest are suspended.
• many single-threaded processes: as recently as 20 years ago, many operating systems supported multiple processes, but only one thread per process (this is the model osv uses, in fact). To the kernel, however, each process looks like a thread: a separate sequence of instructions, executing sometimes in the kernel and sometimes at user level. For example, on a multiprocessor, if multiple processes perform system calls at the same time, the kernel, in effect, has multiple threads executing concurrently in kernel mode.
• many kernel threads: to manage complexity, shift work to the background, exploit parallelism, and hide I/O latency, the operating system kernel itself can benefit from using multiple threads. In this case, each kernel thread runs with the privileges of the kernel. The operating system kernel itself implements the thread abstraction for its own use.

Supporting multithreading — that is, separating the concept of a process (address space, files, etc.) from that of a minimal thread of control (execution state), is a big win. It means that creating concurrency does not require creating new processes. Since processes have a lot more overhead than threads (e.g., a whole new address space), this makes concurrency faster / better / cheaper. Multithreading is useful even on a single core processor running only one thread at a time—why?⁵

• thread API analogous to asynchronous procedure call
• thread_create is similar to fork / exec, thread_join is similar to wait

#include <stdio.h>
#include "thread.h"

static void go(int n);

#define NTHREADS 10
static thread_t threads[NTHREADS];

int main(int argc, char **argv) {
    int i;
    long exitValue;

    for (i = 0; i < NTHREADS; i++){
        thread_create(&(threads[i]), &go, i);
    }
    for (i = 0; i < NTHREADS; i++){
        exitValue = thread_join(threads[i]);
        printf("Thread %d returned with %ld\n", 
                        i, exitValue);
    }
    printf("Main thread done.\n");
    return 0;
}

void go(int n) {
    printf("Hello from thread %d\n", n);
    thread_exit(100 + n);
    // Not reached
}

• thread_t is a struct with thread-specific metadata
• the second argument, go is a function pointer—where the newly created thread show begin execution
• the third argument, i, is passed to the function

$ ./threadHello
Hello from thread 0
Hello from thread 1
Thread 0 returned 100
Hello from thread 3
Hello from thread 4
Thread 1 returned 101
Hello from thread 5
Hello from thread 2
Hello from thread 6
Hello from thread 8
Hello from thread 7
Hello from thread 9
Thread 2 returned 102
Thread 3 returned 103
Thread 4 returned 104
Thread 5 returned 105
Thread 6 returned 106
Thread 7 returned 107
Thread 8 returned 108
Thread 9 returned 109

• why might the "Hello" message from thread 2 print after the "Hello" message from thread 5, even though thread 2 was created before thread 5?
  • creating and scheduling are separate operations: only assumption is that threads run on their own virtual processor with unpredictable speed, any interleaving is possible
• why must the "Thread returned" message from thread 2 print before the Thread returned message from thread 5?
  • the threads can finish in any order, but the main thread waits for them in the order they were created
• what is the minimum and maximum number of threads that could exist when thread 5 prints "Hello"?
  • minimum is 2 threads: thread 5 and the main thread
  • maximum is 11 threads: all 10 could have been created, while 5 is the first to run

• thread.h
• thread.c
• threadHello.c
• Makefile

Natural answer: the OS is responsible for creating/managing threads

• For example, the kernel call to create a new thread would
  • allocate an execution stack within the process address space
  • create and initialize a Thread Control Block
    • stack pointer, program counter, register values
  • stick it on the ready queue
• We call these kernel threads
• There is a thread name space
  • Thread ids (TIDs)
  • TIDs are integers (surprise!)
• Why "must" the kernel be responsible for creating/managing threads?
  • it's the scheduler

• OS now manages threads and processes / address spaces
  • all thread operations are implemented in the kernel
  • OS schedules all of the threads in a system
    • if one thread in a process blocks (e.g., on I/O), the OS knows about it, and can run other threads from that process
    • possible to overlap I/O and computation inside a process
      • that is, under programmer control
• Kernel threads are cheaper than processes
  • less state to allocate and initialize
• But, they're still pretty expensive for fine-grained use
  • orders of magnitude more expensive than a procedure call
  • thread operations are all system calls
    • context switch
    • argument checks

• There is an alternative to kernel threads
  • Note that thread management isn't doing anything that feels privileged
• Threads can also be managed at the user level (that is, entirely from within the process)
  • a library linked into the program manages the threads
    • because threads share the same address space, the thread manager doesn't need to manipulate address spaces (which only the kernel can do)
    • threads differ (roughly) only in hardware contexts (PC, SP, registers), which can be manipulated by user-level code
    • the thread package multiplexes user-level threads on top of kernel thread(s)
    • each kernel thread is treated as a "virtual processor"
  • we call these user-level threads

• User-level threads are small and fast
  • managed entirely by user-level library
    • e.g., pthreads (libpthreads.a)
  • each thread is represented simply by a PC, registers, a stack, and a small thread control block (TCB)
  • creating a thread, switching between threads, and synchronizing threads are done via procedure calls
    • no kernel involvement is necessary!
  • user-level thread operations can be 10-100x faster than kernel threads as a result

• The OS schedules the kernel thread
• The kernel thread executes user code, including the thread support library and its associated thread scheduler
• The thread scheduler determines when a user-level thread runs
  • it uses queues to keep track of what threads are doing: run, ready, wait
    • just like the OS and processes
    • but, implemented at user-level as a library