30-Sep-86 21:56:35-PDT,20290;000000000000 Mail-From: NEUMANN created at 30-Sep-86 21:53:59 Date: Tue 30 Sep 86 21:53:59-PDT From: RISKS FORUM (Peter G. Neumann -- Coordinator) Subject: RISKS-3.71 DIGEST Sender: NEUMANN@CSL.SRI.COM To: RISKS-LIST@CSL.SRI.COM RISKS-LIST: RISKS-FORUM Digest, Tuesday, 30 September 1986 Volume 3 : Issue 71 FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: Deliberate overrides? (Herb Lin, Alan M. Marcum, Eugene Miya) "Friendly" missiles and computer error - more on the Exocet (Robert Stroud) Re: Reliability, complexity, and confidence in SDI (Michal Young) My understanding of "path" and "bathtub curve" (Bob Estell) More artificial than intelligent? (Autokeywords) (Bob Estell) A Viking lander query (PGN) Note on ARPANET congestion (Nancy Cassidy) [BETTER READ <==\ Indeed, the network is getting old (Jonathan Young) THESE TWO] <=====\ | [MULTIPLE COPIES OF RISKS-3.70? Some of you were lucky enough to get 4 | copies, 2 after last reboot! Foonly Wizard David Poole strikes again? / Retries may also be due to net delays, timeouts in ACKS? Sorry. PGN] / The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. (Contributions to RISKS@CSL.SRI.COM, Requests to RISKS-Request@CSL.SRI.COM) (Back issues Vol i Issue j available in CSL.SRI.COM:RISKS-i.j. Summary Contents in MAXj for each i; Vol 1: RISKS-1.46; Vol 2: RISKS-2.57.) ---------------------------------------------------------------------- Date: Tue, 30 Sep 1986 00:58 EDT From: LIN@XX.LCS.MIT.EDU To: "Scott E. Preece" Cc: RISKS@CSL.SRI.COM Subject: Deliberate overrides? From: Scott E. Preece ... users should be allowed to override automated controls in almost all cases AND designers should make very, very sure that the effort to override is proportional to the danger of the override. The problem rarely comes with a competent person who wants to override. The real problems are those people who are incompetent who want to override. I have faith (should I?) in pilots as a rule, so I am generally in favor of manual overrides in airplanes. I don't in drivers as a rule, so I am much less favorbaly disposed to overrides in autos. I recognize that in some instances, the unavailability of an override will cause difficulties for a competent operator, but it will also surely prevent a number of difficulties for incompetent operators, and on balance, I think that that trade is worth while in autos, but probably not in planes. ------------------------------ Date: Tue, 30 Sep 86 09:56:41 PDT From: marcum@Sun.COM (Alan M. Marcum, Consulting) Subject: Deliberate Overrides - mechanical, even To: RISKS@CSL.SRI.COM Though perhaps not strictly computer related, I thought the following might be of interest to the Risks forum. The Piper Arrow is a four-place, single-engine airplane with retractable landing gear. Piper has a wonderful airspeed switch in the landing gear system which will automatically lower the gear if the airspeed is too low. One side effect of this is that during a low-speed climb, the gear may drop (or might never come up during takeoff). Climbing with the gear down will seriously erode climb performance (up to 500 feet per minute, with max. climb around 1000 fpm), just when you want MAXIMUM climb performance! To overcome this, Piper installed a "gear override" handle, which can be latched in the OVERRIDE position. Many, many Arrow pilots routinely take off with the override engaged, to ensure that the gear retract when the pilot wants them up. Why did Piper install this mechanism? The reason most often cited is to help prevent gear-up landings. It is interesting that a number of Arrow pilots have landed gear-up, having forgotten to disengage the override after having gotten into the habit of depending on it. I've flown retractable singles built by Piper, Cessna, Beech, and Mooney. Piper is the only one with the airspeed override. All manufacturers, including Piper, have a warning horn which sounds if power is reduced past some threshhold with the gear up, though. What's the point? In my opinion, the automatic system increases pilot workload during a critical time (takeoff and initial climb). It's not something on which one should EVER depend: it might fail. It's prone to lower the gear at inopportune moments -- times a pilot would absolutely never lower the gear; times when having the gear down is seriously more dangerous than having the gear up (certain emergency situations, for example). And, you need the override. Certainly, performance- or functionality-limiting devices can be useful. They must be thought through carefully, and considered as part of the whole, rather than as an isolated system. Alan M. Marcum Sun Microsystems, Technical Consulting ...!{dual,decvax}!sun!nescorna!marcum Mountain View, California ------------------------------ From: eugene@AMES-NAS.ARPA (Eugene Miya) Date: 30 Sep 1986 1330-PDT (Tuesday) To: risks@sri-csl.ARPA Subject: Re: Deliberate overrides and multiple causes (blame) From: "Scott E. Preece" > /**** ccvaxa:fa.risks / LIN@XX.LCS.MI / 2:47 am Sep 29, 1986 ****/ > > From: Charles R. Fry [...] > > From: LIN@XX.LCS.MIT.EDU [...] Just a point of information: the Soviets just announced that they planned to get rid of reactor control rod overrides, and that one manual override at TMI accentuated the problem ("But overall system worked" summarizing the pro-nuclear viewpoint). Is it possible to write a rule without exception? >"You know, if I do what you've asked, the bomb is going to fall on the wing > and probably strip off your starboard control surfaces." >"Yes, I know, do it anyway." Yes, I can see this happening, but it reminds me of the film Dark Star. "Talk to the bomb . . . about phenomenology....." --eugene miya, NASA Ames Research Center, eugene@ames-aurora.ARPA ------------------------------ From: Robert Stroud Date: Tue, 30 Sep 86 14:43:24 gmt To: risks@csl.sri.com Subject: Re: "Friendly" missiles and computer error - more on the Exocet There is a very interesting BBC TV documentary in the Horizon series called "In the wake of HMS Sheffield" which is well worth seeing if you get the chance. It discusses the failures in technology during the Falklands war and the lessons which have been learnt from them, and includes interviews with participants on both sides. Naturally the fate of HMS Sheffield features prominently, and the chronology given by Rob MacLachlan matches the program in most respects. However, I'm afraid it says nothing about the Exocet homing signal being friendly - I was specifically looking out for this. Instead, according to the documentary, the device which should have detected the homing signal is situated next to the satellite transmission device and was simply swamped by the signal from a telephone call to London in progress at the time - this backs up Peter's definitive account. A couple of other points from the documentary are worth mentioning. Chaff was indeed effective in helping one ship avoid an Exocet (I forget which one) but it is by no means fool proof. The fuse needs to be set manually on deck and must be exact, taking into account lots of factors like wind direction, ship's course, distance from missile, etc. If you get it wrong, the distraction comes too early or too late. There was a nice piece of computer graphics showing the difference half a second could make - needless to say, they are working on an automatic fuse! The Argentinian planes were able to avoid radar detection using a technique called "pecking the lobes". Basically they exploit the shape of the radar cone and the curvature of the earth by flying level until they detect a radar signal, then losing height and repeating the process. As Rob said, they only need to rise up high enough to be detected at the last minute when they fire the Exocets and turn for home - even this trace would only be visible very briefly on the radar display and could easily be missed. Thereafter the Exocets are silent until the last few seconds when they lock onto the target to make last minute course corrections. This problem has been dealt with by building radar devices that can be used from helicopters several thousand feet up so they can see further over the horizon. There was also a discussion about whether it would be feasible to install anti-missile weapons in cargo ships such as the Atlantic Conveyor (sunk twice by the Argentinians with Exocets who mistook it for one of the aircraft carriers). Apparently, installing a weapon would be possible, but to be effective it would need all the command & control computer systems as well to keep track of everything else that was going on, and that would not be feasible. Robert Stroud, Computing Laboratory, University of Newcastle upon Tyne. ARPA robert%cheviot.newcastle@cs.ucl.ac.uk (or ucl-cs.ARPA) UUCP ...!ukc!cheviot!robert ------------------------------ To: ESTELL ROBERT G cc: risks Subject: Re: Reliability, complexity, and confidence in SDI software Date: Mon, 29 Sep 86 22:57:46 -0800 From: Michal Young Bob's message, and some of the replies, seem to be using the term `path' in a sense I am unfamiliar with, since they refer to (large but) finite numbers of paths in software. If software contains loops, isn't the number of paths infinite? And therefore, after any finite amount of use, isn't the percentage of paths tested actually zero? If there is another commonly accepted meaning of `path' through a piece of software, please fill me in on it. I have a similar problem with the term `state.' It seems to be used to refer to major states like `ready to run' and `running', whereas a fault may be sensitive to smaller-grain state like `i = 0 and j > 999'. It may be possible to design software to have a small number of major states, but the number of possible data+control states of any useful program is very large indeed. > BOTTOM LINEs: > > 1. The curve for debugging software has a DOWNslope and length that is > some function of the number of possible paths through the code. > ... > 3. Confidence builds as one approaches the 90% [or other arbitrary level] > point in testing the number of possible paths. > > 4. The reason that we haven't built confidence in the past is that we've > often run thousands of hours, without knowing either: > > a. how many paths got tested; or > b. how many paths remained untested. By the terminology I am familiar with, 3 is "never" and 4(b) is "an infinite number" for every useful piece of software, always. --Michal Young, UC Irvine, young@ics.uci.edu ------------------------------ Date: 30 Sep 86 09:04:00 PST From: "ESTELL ROBERT G" Subject: My understanding of "path" and "bathtub curve" To: "young" cc: risks@csl.sri.com I don't claim to use "path" in a way that may be common in graduate courses in software engineering. My use is based on the highway map analog; e.g., there are many paths through the LA freeway system that one might take from Irvine to Mammoth on a ski weekend. One can drive any of the paths any number of times [loops?]; for lack of good all-way interchanges, some paths might not work well [design errors?]; because of temporary traffic congestion, some paths might be troublesome on some days [data sensitivity?]. I agree that software *is* sensitive to "minor" state conditions; e.g., loop counts of "zero" and "n+1" [where "n" was the intended limit] are notorious. I contend that it *should NOT* be; i.e., that proper design and testing can reduce such errors to a tolerable range. A goal of good software design is to construct "modules" whose internal states are insensitive to all legal arguments, and whose entry code screens out all illegal arguments; at least that's my personal understanding of one [of several] key benefits of "data hiding" and "defensive programming." Another respondent disputed the "downslope" claim, because his experience was that the error rate degenerates to some constant level. Well, all the bathtubs I've seen do have bottoms. One can expect some non-zero number of bugs to persist; let's only hope that it's tolerably low - lower than during "alpha testing." Finally, if some "new release" goes badly sour [e.g., the "new" ARPANET s/w?] because it tries to "add on" [vice "design in?"] new features, maybe that's the equivalent to the "wear out" upslope in mechanical designs. That may be what we've seen with some older operating systems that tried to "add on" time sharing, security, or multi-processor logic. Bob ------------------------------ Date: 30 Sep 86 10:14:00 PST From: "ESTELL ROBERT G" Subject: More artificial than intelligent? (Autokeywords) To: "risks" Computer titles on documents are going to take over. Don't fight it. It could be worse; they might have to be "bar coded." Instead, just use "human" sub-titles; e.g. ANTLERS, TREETOPS, MYSTERY; (or "Who Goosed the Moose?") ------------------------------ Date: Tue 30 Sep 86 20:26:06-PDT From: Peter G. Neumann Subject: A Viking lander query To: RISKS@CSL.SRI.COM Is there a RISKS reader who can report on what the Viking lander software really did? Was it used for landing? or just for communication and control of onboard equipment? "Working the first time" would be much more impressive if it were for landing, whereas the rest is more easily testable on the ground. ------------------------------ Date: Mon, 22 Sep 86 12:22:13 EDT From: Nancy Cassidy Subject: Note on ARPANET congestion ===================================================================== Report on Investigation Request #: IR86-0051-ARPANET-SY Report #: 1 Date of Report: 9/22/86 Priority: 2 ===================================================================== Reporting: open ===================================================================== IR Title: ARPANET congestion Summary of Problem: ------------------ ARPANET users are experiencing unacceptable delays caused by network congestion. This problem is believed to be attributed to the network's dramatic increase in traffic at the rate of 33% in recent months. However, additional bandwidth and PSNs have not been added to the network to sufficiently support this constant increase in traffic. As a result, severe congestion causes users to experience long delays. Another problem exists for users on MILNET who must access the BBNNET (especially users on DDN1 and DDN2). Currently, there is no gateway between the MILNET and BBNNET. Instead, traffic passes through the ARPANET to an ARPANET Gateway in order to reach the BBNNET. The critical congestion problems the ARPANET is experiencing causes TELNET and FTP connections to time out and mail messages from MILNET hosts to take up to 2-3 days to be delivered to BBNNET hosts. One other result of network congestion is the Monitoring Center's ability to effectively monitor operations. The number of traps and status messages has increased proportionately to the severity of the congestion. This dramatic increase in network messages received by the MC consumes CPU space and slows down C/70 performance to the point where it affects monitoring and control of the network. [Further reporting and recommendations truncated...] ------------------------------ Date: Tue, 30 Sep 86 12:59:47 edt From: Jonathan Young Subject: Indeed, the network is getting old To: Neumann@CSL.SRI.COM Here at Yale we have been aware of two problems: host tables are overflowing and mail is bouncing. Actually, we think that SENDMAIL connections (more often from BSD4.3 machines) are timing out and retrying. This has resulted in dozens of copies of certain messages. I enclose a copy of a message from our network administrator. I'm very surprised that others haven't commented about the virtual unavailability of the ARPANET. On the other hand, Yale's connection is via a 9600 BAUD LINE to Harvard. Sigh. Jonathan (YOUNG-JONATHAN@YALE.ARPA) [Is that anywhere near the 50 YARD LINE? (rELIability!) PGN] From: Morrow Long To: department Subject: ARPAnet mail problems We began to see a large problem with repeating incoming arpanet mail messages in August (when cheops was still yale.arpa - the mail name host) - especially in the department bboard where a MIT site was flooding our newsgroups and bulletin boards with the mail internet bulletin board messages. After christening Yale-Eli as Yale.ARPA (a dedicated SMTP mail server) we have continued to experience the problem with repeating messages emanating from some hosts. From statistics we have gathered on the problem we have noticed that many of the problem hosts are running 4.3bsd. Our problem may not be due to 4.3bsd TCP/IP (nor Sendmail/SMTP) but may be brought on by problems with arpanet congestion/delays wrecking session protocols. To alleviate and eventually rectify the problem we have taken the following steps: 1. We have notified the administrators of the remote sites to remove the repeating messages from their spool queues. 2. We are tracing Sendmail/SMTP debugging messages to session logfiles to capture maximum information. 3. Luis has agreed to act as moderator for one of the most troublesome groups ('apollo@yale'), screening out duplicates before reposting them to the world. 4. A 'sweep' daemon has been created and installed on Eli to check for duplicate messages to bboards and mailing list in the mail queue and remove them for exact matches on Message-ID, sender and subject. At least one copy is always allowed through. Even this drastic program will allow repeat messages if they arrive outside of the queue sweep window for duplicates. 5. We will be investigating the 4.3bsd Unix sendmail program for incompatibilities with our SMTP servers. H. Morrow Long, Computing Facility [This is just the tip of the iceberg on reports and messages. The TCP-IP BBOARD IS OVERFLOWING. All sorts of contributing factors are being discussed. Dramatic increase in net traffic, total saturation of the IMPs, hosts that stick with 4.2bsd instead of 4.3, weak gateways, mail distributions to multiple users at the same site, etc. Who knows? No one yet. The total collapse of the ARPANET on 27 Oct 1980 was only for four hours or so, and has not happened since; this fiasco has been going on for at least three weeks, and the network seems to be rotting completely. So, if you got this far in the issue, and are getting a private copy of RISKS, please let me know if your site now has a BBOARD or redistribution and you can live without a private copy of RISKS. (Each time I suggest that I actually do get a few willing people.) PGN] ------------------------------ End of RISKS-FORUM Digest ************************ -------