CS 332 w22 — Introduction; What is an Operating System?

It will be useful to have a basic familiarity with the primary hardware components of a computer system. In particular, the role of the processor, memory, and disk are critical. This reading provides a good brief overview. Please post any questions you have on the Moodle course forum or on Slack! It's important not to have lingering confusion about these basics.

An operating system (OS) is a complex piece of software that fulfills many different roles in a computer system. Which roles get emphasized and even which are necessary vary considerably for different systems. This makes answering what is an operating system somewhat difficult. Possible answers include:

• I don't know (though a few years ago I gave it a shot).
• Nobody knows.
• The assigned reading claims to know (and makes a number of good points)
• They're programs—large, complicated programs
  • The Linux source has over 15M lines of C
  • Windows has way, way more…

• key to know how stuff works
• some really interesting problems and ideas
• a step towards real, comprehensive understanding

• Virtualization (illusionist, referee)
  • 1 CPU, 1 memory => many CPUs, many large/private memories
  • CPU: switch between many programs
  • Memory: share among programs (divide up)
• Standard Libraries (glue)
  • Make system easy to use
• Do this while being efficient and secure

One of the key roles of the OS discussed in the reading is the allocation of resources. The processor (or CPU) is one of the resources the OS must coordinate and share. Consider the simple program below. All it does is call Spin(), a function that repeatedly checks the time and returns once it has run for a second. Then, it prints out the string that the user passed in on the command line, and repeats, forever.

/* cpu.c from https://github.com/remzi-arpacidusseau/ostep-code/tree/master/intro */
#include <stdio.h>
#include <stdlib.h>
#include "common.h"

int main(int argc, char *argv[])
{
    if (argc != 2) {
        fprintf(stderr, "usage: cpu <string>\n");
        exit(1);
    }
    char *str = argv[1];

    while (1) {
        printf("%s\n", str);
        Spin(1);
    }
    return 0;
}

Let's say we compile and run this code on a system with a single CPU. Here's what we will see:

$ gcc -o cpu cpu.c -Wall
$ ./cpu "A"
A
A
A
A
^C
$

Not too interesting of a run — the system begins running the program, which repeatedly checks the time until a second has elapsed. Once a second has passed, the code prints the input string passed in by the user (in this example, the letter "A"), and continues. Note the program will run forever; by pressing "Control-c" (which on UNIX-based systems will terminate the program running in the foreground) we can halt the program.

Now, let’s do the same thing, but this time, let’s run many different instances of this same program.

$ ./cpu A & ./cpu B & ./cpu C & ./cpu D &
[1] 7353
[2] 7354
[3] 7355
[4] 7356
A
B
D
C
A
B
D
C
A
...

Well, now things are getting a little more interesting. Even though we have only one processor, somehow all four of these programs seem to be running at the same time! How does this magic happen?¹

It turns out the operating system, with some help from the hardware, can share the CPU among many different running programs (referee), while providing each program the view that it's the only one using the CPU (illusionist). This capability is so central, that it touches on nearly every topic we will cover in this course.

Another aspect of the operating system as referee is facilitating communication between different applications. A common mechanism for this is shared memory. That is, data in memory that multiple applications can access, communicating by reading and writing to the same variable or data structure. I'll demonstrate this with an example of a multi-threaded program below (i.e., an application with multiple independent parts inside of it).

/* thread.c from https://github.com/remzi-arpacidusseau/ostep-code/tree/master/intro */
#include <stdio.h>
#include <stdlib.h>
#include "common.h"
#include "common_threads.h"

volatile int counter = 0; 
int loops;

void *worker(void *arg) {
    int i;
    for (i = 0; i < loops; i++) {
        counter++;
    }
    return NULL;
}

int main(int argc, char *argv[]) {
    if (argc != 2) { 
        fprintf(stderr, "usage: threads <loops>\n"); 
        exit(1); 
    } 
    loops = atoi(argv[1]);
    pthread_t p1, p2;
    printf("Initial value : %d\n", counter);
    Pthread_create(&p1, NULL, worker, NULL); 
    Pthread_create(&p2, NULL, worker, NULL);
    Pthread_join(p1, NULL);
    Pthread_join(p2, NULL);
    printf("Final value   : %d\n", counter);
    return 0;
}

Although you might not understand this example fully at the moment (and we’ll learn a lot more about it starting in week 3), the basic idea is simple. The main program creates two threads using Pthread_create() ². You can think of a thread as a function running within the same memory space as other functions, with more than one of them active at a time. In this example, each thread starts running in a routine called worker(), in which it simply increments a counter in a loop for loops number of times.

Below is a transcript of what happens when we run this program with the input value for the variable loops set to 1000. The value of loops determines how many times each of the two workers will increment the shared counter in a loop. When the program is run with the value of loops set to 1000, what do you expect the final value of counter to be?

$ gcc -o thread thread.c -Wall -pthread
$ ./thread 1000
Initial value : 0
Final value : 2000

As you probably guessed, when the two threads are finished, the final value of the counter is 2000, as each thread incremented the counter 1000 times. Indeed, when the input value of loops is set to \(N\), we would expect the final output of the program to be \(2N\). But life is not so simple, as it turns out. Let’s run the same program, but with higher values for loops, and see what happens:

$ ./thread 100000
Initial value : 0
Final value : 143012 // huh??
$ ./thread 100000
Initial value : 0
Final value : 137298 // what the??

In this run, when we gave an input value of 100,000, instead of getting a final value of 200,000, we instead first get 143,012. Then, when we run the program a second time, we not only again get the wrong value, but also a different value than the last time. In fact, if you run the program over and over with high values of loops, you may find that sometimes you even get the right answer! So why is this happening?

As it turns out, the reason for these odd and unusual outcomes relate to how instructions are executed, which is one at a time. Unfortunately, a key part of the program above, where the shared counter is incremented, takes three instructions: one to load the value of the counter from memory into a register, one to increment it, and one to store it back into memory. Because these three instructions do not execute atomically (all at once), strange things can happen. It is this problem of concurrency that we will address in great detail in weeks 3–6.

As this example illustrates, communication between applications or between threads requires more than just allowing them to access shared memory. These accesses must be carefully synchronized to avoid strange and unpredictable behavior.

Now let’s consider memory. The model of physical memory presented by modern machines is very simple. Memory is just an array of bytes; to read memory, one must specify an address to be able to access the data stored there; to write (or update) memory, one must also specify the data to be written to the given address. Let’s take a look at a program that allocates some memory by calling malloc().

/* mem.c from https://github.com/remzi-arpacidusseau/ostep-code/tree/master/intro */
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include "common.h"

int value;

int main(int argc, char *argv[]) {
    if (argc != 2) { 
        fprintf(stderr, "usage: mem <value>\n"); 
        exit(1); 
    } 
    int *p; 
    p = &value;
    assert(p != NULL);
    printf("(pid:%d) addr of main:         %p\n", (int) getpid(), main);
    printf("(pid:%d) addr of p:            %p\n", (int) getpid(), &p);
    printf("(pid:%d) addr pointed to by p: %p\n", (int) getpid(), p);      // a2
    *p = atoi(argv[1]); // assign value to addr stored in p            // a3
    while (1) {
        Spin(1);
        *p = *p + 1;
        printf("(pid:%d) value of p: %d\n", getpid(), *p);                 // a4
    }
    return 0;
}

The program does a couple of things. First, it allocates some memory (line a1). Then, it prints out the address of the memory (a2), and then puts the number from the command line into the first slot of the newly allocated memory (a3). Finally, it loops, delaying for a second and incrementing the value stored at the address held in p. With every print statement, it also prints out what is called the process identifier (the PID) of the running program. This PID is unique per running process.

The output of this program looks like this (Linux only):

$ setarch $(uname --machine) --addr-no-randomize /bin/bash
$ gcc -o mem mem.c -Wall
$ ./mem 5
(18275) addr pointed to by p: 0x555555756260
(18275) value of p: 6
(18275) value of p: 7
(18275) value of p: 8
(18275) value of p: 9
^C

Again, this first result is not too interesting. The newly allocated memory is at address 0x555555756260. As the program runs, it slowly updates the value and prints out the result. Now, we again run multiple instances of this same program to see what happens.

$ ./mem 5 & ./mem 0 &
[1] 18283
[2] 18284
(18283) addr pointed to by p: 0x555555756260
(18284) addr pointed to by p: 0x555555756260
(18283) value of p: 6
(18284) value of p: 1
(18283) value of p: 7
(18284) value of p: 2
(18283) value of p: 8
(18284) value of p: 3
(18283) value of p: 9
(18284) value of p: 4
(18283) value of p: 10
...

We see from the example that each running program has allocated memory at the same address (0x555555756260), and yet each seems to be updating the value at 0x555555756260 independently! It is as if each running program has its own private memory, instead of sharing the same physical memory with other running programs³

#include <stdio.h>
#include <unistd.h>
#include <assert.h>
#include <fcntl.h>
#include <sys/types.h>
#include <string.h>

void
dowork()
{
    int fd = open("/tmp/file", O_WRONLY | O_CREAT | O_TRUNC, S_IRWXU);
    assert(fd >= 0);
    char buffer[20];
    sprintf(buffer, "hello world\n");
    int rc = write(fd, buffer, strlen(buffer));
    assert(rc == (strlen(buffer)));
    printf("wrote %d bytes\n", rc);
    fsync(fd);
    close(fd);
}

int
main(int argc, char *argv[])
{
    dowork();
    return 0;
}

$ strace ./io
execve("./io", ["./io"], 0x7ffd2bc143e0 /* 45 vars */) = 0
brk(NULL)                               = 0x55fa55617000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=122120, ...}) = 0
mmap(NULL, 122120, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fbf9bb8f000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000b\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=144976, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbf9bb8d000
mmap(NULL, 2221184, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fbf9b765000
mprotect(0x7fbf9b77f000, 2093056, PROT_NONE) = 0
mmap(0x7fbf9b97e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19000) = 0x7fbf9b97e000
mmap(0x7fbf9b980000, 13440, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fbf9b980000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\35\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=2030928, ...}) = 0
mmap(NULL, 4131552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fbf9b374000
mprotect(0x7fbf9b55b000, 2097152, PROT_NONE) = 0
mmap(0x7fbf9b75b000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1e7000) = 0x7fbf9b75b000
mmap(0x7fbf9b761000, 15072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fbf9b761000
close(3)                                = 0
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbf9bb8a000
arch_prctl(ARCH_SET_FS, 0x7fbf9bb8a740) = 0
mprotect(0x7fbf9b75b000, 16384, PROT_READ) = 0
mprotect(0x7fbf9b97e000, 4096, PROT_READ) = 0
mprotect(0x55fa55163000, 4096, PROT_READ) = 0
mprotect(0x7fbf9bbad000, 4096, PROT_READ) = 0
munmap(0x7fbf9bb8f000, 122120)          = 0
set_tid_address(0x7fbf9bb8aa10)         = 17001
set_robust_list(0x7fbf9bb8aa20, 24)     = 0
rt_sigaction(SIGRTMIN, {sa_handler=0x7fbf9b76acb0, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fbf9b777980}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {sa_handler=0x7fbf9b76ad50, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7fbf9b777980}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
openat(AT_FDCWD, "/tmp/file", O_WRONLY|O_CREAT|O_TRUNC, 0700) = 3
write(3, "hello world\n", 12)           = 12
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 16), ...}) = 0
brk(NULL)                               = 0x55fa55617000
brk(0x55fa55638000)                     = 0x55fa55638000
write(1, "wrote 12 bytes\n", 15wrote 12 bytes
)        = 15
fsync(3)                                = 0
close(3)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

• webpage
  • late days
  • academic integrity
  • inclusivity
• graded work
  • labs
  • quizzes
  • final project
• labs/project build on osv, a minimal, working operating system
• labs/project to be done alone or with a partner (fill out introductory survey)
• lab 0: individual introductory assignment

• first OSes: provide standard set of common services to reduce programmer errors
• next step: sharing (see 100 users per computer in 1981 column)
  • batch operating system
  • multitasking/multiprogramming
• how to debug an operating system? Use a virtual machine
  • you will do this in lab 0 (and all the other labs)
• next step: make computer interactive (rather than a system you send jobs to and get results back, eventually)
  • time sharing
  • Windows, MacOS, Linux
• modern OSes
  • smartphones
  • servers (one program, many requests/tasks)
  • virtual machines (multiplex one machine with low overhead)
  • embedded systems (smart TVs, planes, trains, automobiles, medical devices, LEGOs)

Though precisely defining the idea of an operating system may be challenging, we can clearly state the goals any operating system should aim for. We will use four primary goals to frame our discussion of operating systems throughout this course:

• Reliability and Availability. Does the operating system do what you want?
• Security. Can the operating system be corrupted by an attacker?
• Portability. Is the operating system easy to move to new hardware platforms?
• Performance. Is the user interface responsive, or does the operating system impose too much overhead?

This reading goes through the aspects and tradeoffs involved in each of these goals.