1. Week 3 Redirection, Pipes, and Background
Sarah Diesburg
8/3/2010
COP4610 / CGS5765

2. Redirection
Redirection of stdin and stdout happen by placing a redirection character between commands
[command] < [in_file]
[command] > [out_file]
[command] < [in_file] > [out_file]

3. Input Redirection [command] < [in_file]
The command now takes input from in_file instead of stdin
Examples
cat < file1.txt
grep hamburger < menu.txt

4. Output Redirection [command] > [out_file]
Prints the output from command to out_file instead of stdout
Example
echo hello > file1.txt

5. Combination Redirection
[command] < [in_file] > [out_file]
The command takes input from the in_file instead of stdin and prints output to the out_file instead of stdout
Example
cat < file1.txt > duplicate.txt

6. How to Implement Redirection
We need to duplicate standard I/O using the ‘dup’ command
int dup(int oldfd);
int dup2(int oldfd, int newfd);
Dup makes a duplicate of the oldfd file descriptor and returns the new file descriptor

7. Steps for Input Redirection
Open the input file
Close stdin
Issue dup() on the input file
dup will automatically choose lowest closed file descriptor – stdin
Input file and stdin are now the same file descriptor
Close input file

8. Input Redirection Visual
shell
file descriptors
0 stdin
1 stdout
2 stderr
fork()
file descriptors
0 stdin
1 stdout
2 stderr
shell

9. Input Redirection Visual
open(input_file, O_RDONLY)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin
1 stdout
2 stderr
shell
shell

10. Input Redirection Visual
open(input_file, O_RDONLY)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin
1 stdout
2 stderr
3 input_file
shell
shell
Can be any number > 2…

11. Input Redirection Visual
close(0)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin
1 stdout
2 stderr
3 input_file
shell
shell

12. Input Redirection Visual
dup(3)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
1 stdout
2 stderr
3 input_file
shell
shell

13. Input Redirection Visual
dup(3)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 input_file
1 stdout
2 stderr
3 input_file
shell
shell

14. Input Redirection Visual
close(3)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 input_file
1 stdout
2 stderr
3 input_file
shell
shell

15. Steps for Output Redirection
Open the output file
Close stdout
Issue dup() on the output file
dup will automatically choose lowest closed file descriptor – stdout
Input file and stdout are now the same file descriptor
Close output file

16. Output Redirection Visual
shell
file descriptors
0 stdin
1 stdout
2 stderr
fork()
file descriptors
0 stdin
1 stdout
2 stderr
shell

17. Output Redirection Visual
open(output_file,
O_RDWR | O_CREAT | O_TRUNC)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin
1 stdout
2 stderr
shell
shell

18. Output Redirection Visual
open(output_file,
O_RDWR | O_CREAT | O_TRUNC)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin
1 stdout
2 stderr
3 output_file
shell
shell

19. Output Redirection Visual
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin
1 stdout
2 stderr
3 output_file
shell
close(1)
shell

20. Output Redirection Visual
dup(3)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin 1
2 stderr
3 output_file
shell
shell

21. Output Redirection Visual
dup(3)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin
1 output_file
2 stderr
3 output_file
shell
shell

22. Output Redirection Visual
close(3)
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin
1 output_file
2 stderr
3 output_file
shell
shell

23. Note on Redirection
Only perform redirection in the child process
Why?

24. Pipes
One program can receive output from another without an explicit temporary file
command1 | command 2
Same as
command1 > tmpfile
command2 < tmpfile
rm tmpfile

25. Pipes command1 | command 2
Since two commands will be run, we will need to create two children
We also need to create an I/O “pipe” between the children processes

26. Pipes Use the system call ‘pipe’ int pipe(int filedes[2]);
Creates a pair of file descriptors for pipe I/O and places them in filedes[]
Returns 0 on success

27. Pipe int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin
1 stdout
2 stderr
shell

28. Pipe int p1_to_p2[2]; pipe(p1_to_p2); file descriptors 0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
shell

29. Pipe shell file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0]
int p1_to_p2[2];
pipe(p1_to_p2);
fork()
int p1_to_p2[2];
pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
shell

30. Pipe shell file descriptors 0 stdin 1 stdout 2 stderr 3 p1_to_p2[0]
int p1_to_p2[2];
pipe(p1_to_p2);
fork()
int p1_to_p2[2];
pipe(p1_to_p2);
int p1_to_p2[2];
pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
shell
shell

31. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
shell
shell
shell

32. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin 1
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
dup(4)
shell
shell
shell

33. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
dup(4)
shell
shell
shell

34. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
shell
shell
shell

35. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
shell
shell
shell

36. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
shell
shell
shell

37. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
shell
shell
shell

38. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
dup(3)
shell
shell
shell

39. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
0 p1_to_p2[0]
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
dup(3)
shell
shell
shell

40. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
0 p1_to_p2[0]
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
shell
shell
shell

41. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
0 p1_to_p2[0]
1 stdout
2 stderr
shell
shell
shell

42. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
0 p1_to_p2[0]
1 stdout
2 stderr
execv(bin_path,
argv)
shell
shell
shell

43. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
0 p1_to_p2[0]
1 stdout
2 stderr
execv(bin_path,
argv)
shell
shell
shell

44. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
0 p1_to_p2[0]
1 stdout
2 stderr
execv(bin_path,
argv)
shell
shell
shell

45. Pipe int p1_to_p2[2]; pipe(p1_to_p2); int p1_to_p2[2]; pipe(p1_to_p2);
file descriptors
0 stdin
1 stdout
2 stderr
waitpid(…)
file descriptors
0 stdin
1 p1_to_p2[1]
2 stderr
execv(bin_path,
argv)
file descriptors
0 p1_to_p2[0]
1 stdout
2 stderr
execv(bin_path,
argv)
shell
shell
shell

46. Background Processes sarah@trogdor:~$ ./sleep
A process is run in the foreground if the shell waits for the process to complete before returning you to the prompt
./sleep

47. Background Processes
A process is run in the background if the shell prompt returns right away, even through the process is still running.
./sleep &
Look to bash for example

48. Waiting Problem
When we call wait() or waitpid(), the parent blocks, or waits, until one of the children has finished
Fine for foreground processes
How can we make shell continue and return even if the child has not finished yet?
Use waitpid() option WNOHANG

49. Background example – main loop
int num_processes=0; int status=0; /* Main loop */ { if((pid = fork())==0) { */child executes long processes*/ } else { num_processes++; if(waitpid(-1, &status, WNOHANG) > 0) { num_processes--; } } /* End main loop */

50. What about on exit? You must wait on all children still running
Use variable like num_processes in previous example

51. Foreground Execution
fork()
Shell
Shell
waitpid(-1, &status, 0)

52. Foreground Execution fork() Shell Shell execv(bin_path, argv)
waitpid(-1, &status, 0)

53. Foreground Execution
fork()
Shell
Shell
waitpid(-1, &status, 0)

54. Background Execution
fork()
Shell
Shell
waitpid(-1, &status,
WNOHANG)

55. Background Execution fork() Shell Shell execv(bin_path, argv)
waitpid(-1, &status,
WNOHANG)

56. Background Execution
fork()
Shell
Shell
waitpid(-1, &status,
WNOHANG)

57. Summary Pieces necessary to complete project 1
Part 4 – Input/Output redirection
Part 5 – Pipes
Part 6 – Background processing

58. Next Recitation
Introduction of project 2
Kernel modification!

59. Reminders
Project due date
Any questions?