1 “Star Wars” Revisited A Case Study In Ethics and Safety-Critical Software Professor Kevin Bowyer University of Notre Dame Copyright, Kevin W. Bowyer, 2000,2001. All Rights Reserved. (Rev. 7/3/01)

2 Ballistic Missile Defense A defensive system – to counter enemy offensive weapons. Ballistic missiles follow a predictable trajectory.

3 Ballistic Missile Defense The U.S. has a long history of BMD research and systems. For example, the “Safeguard” system (1960s-70s) was meant to protect U.S. launch sites from pre-emptive attack.

4 Ballistic Missile Defense The most important software elements of a BMD system are command and control. Software development begins with analysis leading to requirements and specifications.

5 Software Life Cycle System Engineering Analysis Design Coding Testing Maintenance

6 Software Life Cycle System engineering – Analyze the overall system, Specify system requirements, Decide on elements of system to implement in software.

7 Software Life Cycle Software requirements – Specify required functionality, performance, and interfaces for the software elements.

8 President Reagan’s SDI In March 1983, President Reagan called for a “Strategic Defense Initiative” (SDI). The SDI program came to be popularly called “Star Wars.”

9 President Reagan’s SDI

10 Pause for Analysis What overall requirements can you derive for the SDI system from Reagan’s speech?

11 President Reagan’s SDI “I call upon the scientific community … to give us the means of rendering these nuclear weapons impotent and obsolete.”

12 President Reagan’s SDI “I am directing … to define a long-term R&D program to begin to … eliminate the threat posed by strategic nuclear missiles.”

13 SDI Computing Task The software must make the sensors, weapons, control, and communications elements of the system work together to satisfy requirements.

14 SDI Computing Panel A panel of experts studied the SDI computing problem (at $1,000 / day consulting!). David Parnas resigned from the panel and declared the software to be impossible.

CPSR-MIT Debate The Computing Professionals for Social Responsibility (CPSR) sponsored a debate at MIT, featuring members of the SDI computing panel.

CPSR-MIT Debate Michael Dertouzos, moderator David Parnas, against SDI (Joseph Weizenbaum, against) Charles Seitz, for SDI (Danny Cohen, for)

17 Michael Dertouzos, moderator

18 Pause for Analysis Based on Dertouzos’ overview, what more can be said about the computing requirements for SDI?

19 Michael Dertouzos “Star Wars” – Can the computing requirements be met? Assume that the sensors and weapons can be developed and put in space, are affordable, …

20 Michael Dertouzos Parameters of the problem – watch 50 million km 2 area, track up to 3,000 missiles, up to 30,000 warheads up to 300,000 decoys, It all happens in minutes.

21 Michael Dertouzos Elements of the task – Form target tracks. Consistent, distributed data. Allocate weapons to tracks. Communicate with humans. Withstand attack.

CPSR-MIT Debate Michael Dertouzos, moderator David Parnas, against SDI (Joseph Weizenbaum, against) Charles Seitz, for SDI (Danny Cohen, for)

23 David Parnas, arguing against

24 Pause for Analysis Sketch Parnas’ argument in premise-conclusion style: Since Premise, and Premise, … Therefore Conclusion. (Hint: identify conclusion first.)

25 Parnas’ Conclusion Which is proper conclusion of his technical argument? U.S. should not pursue SDI. SDI will make U.S. weaker. It is not possible to build trustworthy SDI software.

26 Parnas’ Conclusion It is not possible to build trustworthy SDI software. “trustworthy” = reliability similar to that of your car starting in the morning.

27 Parnas’ Premises Since: Specifications cannot be known in advance, (because enemy controls factors such as target / decoy features, attack load and structure, …)

28 Parnas’ Premises Since: Specifications not known in advance, Realistic testing is essentially impossible, (because, for example, link / node failures under attack are not known in advance)

29 Parnas’ Premises Since: Specifications not known in advance, Realistic testing is not possible, Hard real-time deadlines do not allow repair during use (attack is over in minutes)

30 Parnas’ Premises Since: Specifications not known in advance, Realistic testing is not possible, No chance to fix software during use, No foreseeable advance in software tech changes this (not language, methodology, …)

31 Parnas’ Argument Since: Specifications not known in advance, Realistic testing is not possible, No chance to fix software during use, No foreseeable technology changes this, Therefore – It is not possible to construct SDI software that you could trust to work.