Week 3 Redirection, Pipes, and Background Sarah Diesburg 8/3/2010 COP4610 / CGS5765

Redirection Redirection of stdin and stdout happen by placing a redirection character between commands [command] < [in_file] [command] > [out_file] [command] < [in_file] > [out_file]

Input Redirection [command] < [in_file] The command now takes input from in_file instead of stdin Examples cat < file1.txt grep hamburger < menu.txt

Output Redirection [command] > [out_file] Prints the output from command to out_file instead of stdout Example echo hello > file1.txt

Combination Redirection [command] < [in_file] > [out_file] The command takes input from the in_file instead of stdin and prints output to the out_file instead of stdout Example cat < file1.txt > duplicate.txt

How to Implement Redirection We need to duplicate standard I/O using the ‘dup’ command int dup(int oldfd); int dup2(int oldfd, int newfd); Dup makes a duplicate of the oldfd file descriptor and returns the new file descriptor

Steps for Input Redirection Open the input file Close stdin Issue dup() on the input file dup will automatically choose lowest closed file descriptor – stdin Input file and stdin are now the same file descriptor Close input file

Input Redirection Visual shell file descriptors 0 stdin 1 stdout 2 stderr fork() file descriptors 0 stdin 1 stdout 2 stderr shell

Input Redirection Visual open(input_file, O_RDONLY) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 stdout 2 stderr shell shell

Input Redirection Visual open(input_file, O_RDONLY) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 stdout 2 stderr 3 input_file shell shell Can be any number > 2…

Input Redirection Visual close(0) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 stdout 2 stderr 3 input_file shell shell

Input Redirection Visual dup(3) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 1 stdout 2 stderr 3 input_file shell shell

Input Redirection Visual dup(3) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 input_file 1 stdout 2 stderr 3 input_file shell shell

Input Redirection Visual close(3) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 input_file 1 stdout 2 stderr 3 input_file shell shell

Steps for Output Redirection Open the output file Close stdout Issue dup() on the output file dup will automatically choose lowest closed file descriptor – stdout Input file and stdout are now the same file descriptor Close output file

Output Redirection Visual shell file descriptors 0 stdin 1 stdout 2 stderr fork() file descriptors 0 stdin 1 stdout 2 stderr shell

Output Redirection Visual open(output_file, O_RDWR | O_CREAT | O_TRUNC) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 stdout 2 stderr shell shell

Output Redirection Visual open(output_file, O_RDWR | O_CREAT | O_TRUNC) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 stdout 2 stderr 3 output_file shell shell

Output Redirection Visual file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 stdout 2 stderr 3 output_file shell close(1) shell

Output Redirection Visual dup(3) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 2 stderr 3 output_file shell shell

Output Redirection Visual dup(3) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 output_file 2 stderr 3 output_file shell shell

Output Redirection Visual close(3) file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 output_file 2 stderr 3 output_file shell shell

Note on Redirection Only perform redirection in the child process Why?

Pipes One program can receive output from another without an explicit temporary file command1 | command 2 Same as command1 > tmpfile command2 < tmpfile rm tmpfile

Pipes command1 | command 2 Since two commands will be run, we will need to create two children We also need to create an I/O “pipe” between the children processes

Pipes Use the system call ‘pipe’ int pipe(int filedes[2]); Creates a pair of file descriptors for pipe I/O and places them in filedes[] Returns 0 on success

Pipe int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] shell

Pipe shell file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] int p1_to_p2[2]; pipe(p1_to_p2); fork() int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] shell

Pipe shell file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] int p1_to_p2[2]; pipe(p1_to_p2); fork() int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] dup(4) shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] dup(4) shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] dup(3) shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 0 p1_to_p2[0] 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] dup(3) shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 0 p1_to_p2[0] 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 0 p1_to_p2[0] 1 stdout 2 stderr shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 0 p1_to_p2[0] 1 stdout 2 stderr execv(bin_path, argv) shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0] 4 p1_to_p2[1] file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 0 p1_to_p2[0] 1 stdout 2 stderr execv(bin_path, argv) shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 0 p1_to_p2[0] 1 stdout 2 stderr execv(bin_path, argv) shell shell shell

Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin 1 stdout 2 stderr waitpid(…) file descriptors 0 stdin 1 p1_to_p2[1] 2 stderr execv(bin_path, argv) file descriptors 0 p1_to_p2[0] 1 stdout 2 stderr execv(bin_path, argv) shell shell shell

Background Processes sarah@trogdor:~$ ./sleep A process is run in the foreground if the shell waits for the process to complete before returning you to the prompt sarah@trogdor:~$ ./sleep

Background Processes A process is run in the background if the shell prompt returns right away, even through the process is still running. sarah@trogdor:~$ ./sleep & sarah@trogdor:~$ Look to bash for example

Waiting Problem When we call wait() or waitpid(), the parent blocks, or waits, until one of the children has finished Fine for foreground processes How can we make shell continue and return even if the child has not finished yet? Use waitpid() option WNOHANG

Background example – main loop int num_processes=0; int status=0; /* Main loop */ { if((pid = fork())==0) { */child executes long processes*/ } else { num_processes++; if(waitpid(-1, &status, WNOHANG) > 0) { num_processes--; } } /* End main loop */

What about on exit? You must wait on all children still running Use variable like num_processes in previous example

Foreground Execution fork() Shell Shell waitpid(-1, &status, 0)

Foreground Execution fork() Shell Shell execv(bin_path, argv) waitpid(-1, &status, 0)

Foreground Execution fork() Shell Shell waitpid(-1, &status, 0)

Background Execution fork() Shell Shell waitpid(-1, &status, WNOHANG)

Background Execution fork() Shell Shell execv(bin_path, argv) waitpid(-1, &status, WNOHANG)

Background Execution fork() Shell Shell waitpid(-1, &status, WNOHANG)

Summary Pieces necessary to complete project 1 Part 4 – Input/Output redirection Part 5 – Pipes Part 6 – Background processing

Next Recitation Introduction of project 2 Kernel modification!

Reminders Project due date Any questions?