# data[].attributes.attachment_link.readable_text
Mark your calendars! Sept. 9 is hereby declared Debugging Day. It's been associated with removing bugs for more than 50 years now but is rarely formally celebrated. So let's start the tradition this year.

It all began with a log entry from 1947 by Harvard University's Mark II technical team. The now-classic entry features a moth taped to the page, time-stamped 15:45, with the caption "Relay #70 Panel F (moth) in relay" and the proud boast, "First actual case of bug being found."

The moth in the relay. Photo: U.S. Naval Historical Center

OK, the history of computer bugs didn't really begin on this date (see " Moth in the machine" for the real story), but nevertheless, its anniversary seems a perfect time to examine famous bugs and other ghosts in the machine.

Here is a highly selective -- and therefore incomplete -- collection of infamous software bugs. Unlike the relatively benign tale of the moth in the relay, some bugs have wreaked disaster, embarrassment and destruction on the world. Some have literally killed people.

Don't expect this collection to contain tales of the Ping of Death or other faults exploited by hackers and malware -- such as the Spanair crash of 2008 or the possibly apocryphal tale of the CIA sabotaging the Soviet gas pipeline. Nor will it include deliberate decisions by programmers that came back to haunt them later, as with Y2K.

Instead, this story is about outright programming errors that caused key failures in their own right.

Have I missed anything important? Consider this a call for nominations for the biggest bugs of all time. These are my suggestions; if you have any honorable mentions, bring 'em on. The worst anyone can do is swat them.

Space bugs

Programming errors that derail high-profile space-exploration missions -- especially bugs that cause spectacular explosions -- are frightening, expensive and career-killingly embarrassing for those who let them slip through. They provide extremely vivid reminders for all of us to check and recheck (and recheck and recheck and recheck) every line of code.

The Mars Climate Orbiter doesn't orbit

Back in physics class, our teachers leaped all over answers that consisted of a number. If the answer was 2.5, they'd take their red pens and write "2.5 what? Weeks? Puppies? Demerits?" And proceed to mark the answer wrong.

Back then, we thought that they were just being pedantic. But it's the kind of error that can burn up a $327.6 million project in minutes. It did in 1998, when the Mars Climate Orbiter built by NASA's Jet Propulsion Laboratory approached the Red Planet at the wrong angle. At this point, it could easily have been renamed the Mars Climate Bright Light in the Upper Atmosphere, and shortly afterward been renamed the Mars Climate Debris Drifting Through the Sky.

There were several problems with this spacecraft -- its uneven payload made it torque during flight, and its project managers neglected some important details during several stages of the mission. But the biggest problem was that different parts of the engineering team were using different units of measurement. One group working on the thrusters measured in English units of pounds-force seconds; the others used metric Newton-seconds. And whoever checked the numbers didn't use the red pen like a pedantic high-school teacher.

The result: The thrusters were 4.45 times more powerful than they should have been. If this goof had been spotted earlier, it could have been compensated for, but it wasn't, and the result of that inattention is now lost in space, possibly in pieces.

# data[].attributes.attachment_link.title
Epic failures: 11 infamous software bugs

# data[].attributes.attachment_link.description
Mark your calendars! Sept. 9 is hereby declared Debugging Day. It's been associated with removing bugs for more than 50 years now but is rarely formally celebrated. So let's start the tradition this year. It all began with a log entry from 1947 by Harvard University's Mark II technical team.

# data[].attributes.body
https://he.wikipedia.org/wiki/%D7%91%D7%90%D7%92_2000

# data[].attributes.body
החתול שלך היטלר הכי יפה בעולם~!

נ.ב. 72 זה ציון לחלשים

# data[].attributes.body
www.devtopics.com/20-famous-software-disasters

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-
software-bugs.html?page=2

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html?page=2

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html?page=2

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html?page=2

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html?page=2

# data[].attributes.body
http://m.globes.co.il/news/m/article.aspx?did=1000178440

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html?page=2

# data[].attributes.body
http://www.themarker.com/technation/1.2212108

# data[].attributes.body
http://mobile.mako.co.il/nexter-computers/software/Article-1239072e473d151006.htm

# data[].attributes.body
312052772 רומן

# data[].attributes.body
http://royal.pingdom.com/2009/03/19/10-historical-software-bugs-with-extreme-consequences/http://royal.pingdom.com/2009/03/19/10-historical-software-bugs-with-extreme-consequences/

# data[].attributes.body
שחר שני 303161590

http://www.ynet.co.il/articles/0,7340,L-4010559,00.html

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html

# data[].attributes.body
Smart ship USS Yorktown was left dead in the water in 1997 for nearly 3 hours after a divide by zero error.[29]

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html

# data[].attributes.body
http://mobile.mako.co.il/nexter-computers/software/Article-1239072e473d151006.htm

# data[].attributes.body
https://www.cs.tau.ac.il/~nachumd/horror.html

# data[].attributes.body
https://www.cs.tau.ac.il/~nachumd/horror.html

# data[].attributes.body
http://www.computerworlduk.com/galleries/infrastructure/top-10-software-failures-of-2014-3599618/http://www.computerworlduk.com/galleries/infrastructure/top-10-software-failures-of-2014-3599618/

# data[].attributes.body
305334096
עידו נוטס

# data[].attributes.body
http://m.globes.co.il/news/m/article.aspx?did=1000178440http://m.globes.co.il/news/m/article.aspx?did=1000178440

# data[].attributes.body
https://www.cs.tau.ac.il/~nachumd/horror.html

# data[].attributes.attachment_link.description
Some software bugs have been linked to disasters. Bugs in code that controls the Therac-25 radiation therapy machine were directly responsible for patient deaths in the 1980s. In 1996, the European Space Agency's US$1 billion prototype Ariane 5 rocket had to be destroyed less than a minute after launch due to a bug in the on-board guidance computer program.

# data[].attributes.body
http://www.themarker.com/technation/1.2212108

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html

# data[].attributes.body
http://m.globes.co.il/news/m/article.aspx?did=1000178440http://m.globes.co.il/news/m/article.aspx?did=1000178440

# data[].attributes.body
http://m.aviationweek.com/commercial-aviation/ntsb-investigates-cfm56-failure-southwest-boeing-737

# data[].attributes.body
מתן פאר 305186702

https://www.youtube.com/watch?v=tWc4gGMQ3hQ

# data[].attributes.body
https://gcn.com/articles/1998/08/31/smart-ship-inquiry-a-go.aspx

# data[].attributes.body
מור הרשמן 305064123
http://m.globes.co.il/news/m/article.aspx?did=1000178440

# data[].attributes.body
314157108 לנה קירכלי
https://www.cs.tau.ac.il/~nachumd/horror.html

# data[].attributes.body
http://m.ynet.co.il/Articles/1685428http://m.ynet.co.il/Articles/1685428

# data[].attributes.body
http://m.globes.co.il/news/m/article.aspx?did=109485

# data[].attributes.body
http://www.haaretz.co.il/.premium-1.2320844

# data[].attributes.body
http://www.haaretz.co.il/.premium-1.2320844

# data[].attributes.body
http://archive.wired.com/software/coolapps/news/2005/11/69355?currentPage=all

# data[].attributes.body
https://www.google.co.il/amp/www.computerworlduk.com/galleries/infrastructure/top-10-software-failures-of-2014-3599618/%3Famp?client=ms-android-samsung

# data[].attributes.body
https://www.google.co.il/amp/www.computerworlduk.com/galleries/infrastructure/top-10-software-failures-of-2014-3599618/%3Famp?client=ms-android-om-lge

# data[].attributes.body
https://www.scientificamerican.com/article/pogue-5-most-embarrassing-software-bugs-in-history/

# data[].attributes.body
https://sandipsandilya.wordpress.com/2011/01/17/20-famous-software-disasters/

# data[].attributes.attachment_link.title
כישלון: בעיות תוכנה מעכבות פריסת שירותי הטלוויזיה של AT&T בארה"ב - גלובס

# data[].attributes.body
30816451

# data[].attributes.body
https://en.m.wikipedia.org/wiki/Therac-25

# data[].attributes.body
www.haaretz.co.il/.premium-1.2320844

# data[].attributes.headline
מרשוד יא הבלי

# data[].attributes.subject
מרשוד יא הבלי

# data[].attributes.body
https://www.google.co.il/amp/foreignpolicy.com/2012/01/03/the-ten-biggest-american-intelligence-failures/amp/?client=ms-android-samsung

# data[].attributes.headline
נתנאל אזולאי 

# data[].attributes.subject
נתנאל אזולאי 

# data[].attributes.body
203917034
https://he.m.wikipedia.org/wiki/באג_הפנטיום

# data[].attributes.body
https://pdfs.semanticscholar.org/b8d6/ca6bcd0b74ed1c5017e8cf19104bc6146f67.pdf

# data[].attributes.body
https://www.lucidchart.com/techblog/2015/08/31/the-worst-mistake-of-computer-science/

# data[].attributes.body
http://www.tricentis.com/blog/2016/04/07/software-fail-watch-2016-quarter-one/http://www.tricentis.com/blog/2016/04/07/software-fail-watch-2016-quarter-one/

# data[].attributes.body
en.m.wikipedia.org/wiki/List_of_software_bugs

# data[].attributes.body
https://www.google.co.il/amp/foreignpolicy.com/2012/01/03/the-ten-biggest-american-intelligence-failures/amp/?client=ms-android-samsung

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html

# data[].attributes.body
https://en.m.wikipedia.org/wiki/Software_bughttps://en.m.wikipedia.org/wiki/Software_bug

# data[].attributes.body
https://www.google.co.il/amp/www.computerworlduk.com/galleries/infrastructure/top-10-software-failures-of-2014-3599618/%3Famp?client=ms-android-samsung

# data[].attributes.body
https://he.m.wikipedia.org/wiki/באג_2000

# data[].attributes.body
https://www.google.co.il/amp/foreignpolicy.com/2012/01/03/the-ten-biggest-american-intelligence-failures/amp/?client=ms-android-samsung

# data[].attributes.body
https://www.scientificamerican.com/article/pogue-5-most-embarrassing-software-bugs-in-history/

# data[].attributes.body
http://www.cs.ucr.edu/~gupta/teaching/260-08/Papers/p291-hangal.pdf

# data[].attributes.attachment_link.title
רשמית: תקלת תוכנה היא שהובילה לביטול טיסות בארה"ב

# data[].attributes.attachment_link.description
במהלך סוף השבוע בוטלו יותר מ-440 טיסות במזרח המדינה מסיבה לא ברורה. בעוד שבתחילה נחשד כי מדובר במתקפת האקרים מוצלחת, עתה מודיעה מנהלת התעופה הפדרלית כי הגורם הוא עדכון תוכנה עמוס באגים

# data[].attributes.attachment_link.readable_text
The Therac-25 was not a device anyone was happy to see. It was a radiation therapy machine. In layman's terms it was a "cancer zapper"; a linear accelerator with a human as its target. Using X-rays or a beam of electrons, radiation therapy machines kill cancerous tissue, even deep inside the body. These room-sized medical devices would always cause some collateral damage to healthy tissue around the tumors. As with chemotherapy, the hope is that the net effect heals the patient more than it harms them. For six unfortunate patients in 1986 and 1987, the Therac-25 did the unthinkable: it exposed them to massive overdoses of radiation, killing four and leaving two others with lifelong injuries. During the investigation, it was determined that the root cause of the problem was twofold. Firstly, the software controlling the machine contained bugs which proved to be fatal. Secondly, the design of the machine relied on the controlling computer alone for safety. There were no hardware interlocks or supervisory circuits to ensure that software bugs couldn't result in catastrophic failures.

The case of the Therac-25 has become one of the most well-known killer software bugs in history. Several universities use the case as a cautionary tale of what can go wrong, and how investigations can be lead astray. Much of this is due to the work of [Nancy Leveson], a software safety expert who exhaustively researched the incidents and resulting lawsuits. Much of the information published about the Therac (including this article) is based upon her research and 1993 paper with [Clark Turner] entitled " An Investigation of the Therac-25 Accidents". [Nancy] has since published updated information in a second paper which is also included in her book.

History and development

The Therac-25 was manufactured by Atomic Energy of Canada Limited (AECL). It was the third radiation therapy machine by the company, preceded by the Therac-6 and Therac-20. AECL built the Therac-6 and 20 in partnership with CGR, a French company. When the time came to design the Therac-25, the partnership had dissolved. However, both companies maintained access to the designs and source code of the earlier machines. The Therac-20 codebase was developed from the Therac-6. All three machines used a PDP-11 computer.
Therac-6 and 20 didn't need that computer, though. Both were designed to operate as standalone devices. In manual mode, a radiotherapy technician would physically set up various parts of the machine, including the turntable to place one of three devices in the path of the electron beam. In electron mode, scanning magnets would be used to spread the beam out to cover a larger area. In X-ray mode, a target was placed in the electron beam with electrons striking the target to produce X-ray photons directed at the patient. Finally, a mirror could be placed in the beam. The electron beam would never switch on while the mirror was in place. The mirror would reflect a light which would help the radiotherapy technician to precisely aim the machine.

On the Therac-6 and 20, hardware interlocks prevented the operator from doing something dangerous, say selecting a high power electron beam without the x-ray target in place. Attempting to activate the accelerator in an invalid mode would blow a fuse, bringing everything to a halt. The PDP-11 and associated hardware were added as a convenience. The technician could enter a prescription in on a VT-100 terminal, and the computer would use servos to position the turntable and other devices. Hospitals loved the fact that the computer was faster at setup than a human. Less setup time meant more patients per day on a multi-million dollar machine.

When it came time to design the Therac-25, AECL decided to go with computer control only. Not only did they remove many of the manual controls, they also removed the hardware interlocks. The computer would keep track of the machine setup and shut things down if it detected a dangerous situation.

The accidents

The Therac-25 went into service in 1983. For several years and thousands of patients there were no problems. On June 3, 1985, a woman was being treated for breast cancer. She had been prescribed 200 Radiation Absorbed Dose (rad) in the form of a 10 MeV electron beam. The patient felt a tremendous heat when the machine powered up. It wasn't known at the time, but she had been burned by somewhere between 10,000 and 20,000 rad. The patient lived, but lost her left breast and the use of her left arm due to the radiation.

On July 26, a second patient was burned at The Ontario Cancer Foundation in Hamilton, Ontario, Canada. This patient died in November of that year. Autopsy ruled that the death was due to a particularly aggressive cervical cancer. Had she lived however, she would have needed a complete hip replacement to correct the damage caused by the Therac-25.

In December of 1985, a third woman was burned by a Therac-25 installed in Yakima, Washington. She developed a striped burn pattern on her hip which closely matched the beam blocking strips on the Therac-25. This patient lived, but eventually needed skin grafts to close the wounds caused by radiation burns.

On March 21, 1986, a patient in Tyler, Texas was scheduled to receive his 9th Therac-25 treatment. He was prescribed 180 rads to a small tumor on his back. When the machine turned on, he felt heat and pain, which was unexpected as radiation therapy is usually a painless process. The Therac-25 itself also started buzzing in an unusual way. The patient began to get up off the treatment table when he was hit by a second pulse of radiation. This time he did get up and began banging on the door for help. He received a massive overdose. He was hospitalized for radiation sickness, and died 5 months later.

On April 11th, 1986, a second accident occurred in Tyler, Texas. This time the patient was being treated for skin cancer on his ear. The same operator was running the machine as in the March 21st accident. When therapy started, the patient saw a bright light, and heard eggs frying. He said it felt like his face was on fire. The patient died three weeks later due to radiation burns on the right temporal lobe of his brain and brain stem.

The final overdose occurred much later, this time at Yakima Valley hospital in January, 1987. This patient later died due to his injuries.

Investigation

After each incident, the local hospital physicist would call AECL and the medical regulation bureau in their respective countries. At first AECL denied that the Therac-25 was capable of delivering an overdose of radiation. The machine had so many safeguards in place that it frequently threw error codes and paused treatment, giving less than the prescribed amount of radiation. After the Ontario incident, it was clear that something was wrong. The only way that kind of overdose could be delivered is if the turntable was in the wrong position. If the scanning magnets or X-ray target were not in position, the patient would be hit with a laser-like beam of radiation.

AECL carefully ran test after test and could not reproduce the error. The only possible cause they could come up with was a temporary failure in the three microswitches which determined the turntable's position. The microswitch circuit was re-designed such that the failure of any one microswitch could be detected by the computer. This modification was quickly added and was in place for the rest of the accidents.

If this story has a hero, it's [Fritz Hager], the staff physicist at the East Texas Cancer Center in Tyler, Texas. After the second incident at his facility, he was determined to get to the bottom of the problem. In both cases, the Therac-25 displayed a "Malfunction 54" message. The message was not mentioned in the manuals. AECL explained that Malfunction 54 meant that the Therac-25's computer could not determine if there a underdose OR overdose of radiation.

The same radiotherapy technician had been involved in both incidents, so [Fritz] brought her back into the control room to attempt to recreate the problem. The two "locked the doors" NASA style, working into the night and through the weekend trying to reproduce the problem. With the technician running the machine, the two were able to pinpoint the issue. The VT-100 console used to enter Therac-25 prescriptions allowed cursor movement via cursor up and down keys. If the user selected X-ray mode, the machine would begin setting up the machine for high-powered X-rays. This process took about 8 seconds. If the user switched to Electron mode within those 8 seconds, the turntable would not switch over to the correct position, leaving the turntable in an unknown state.

It's important to note that all the testing to this date had been performed slowly and carefully, as one would expect. Due to the nature of this bug, that sort of testing would never have identified the culprit. It took someone who was familiar with the machine - who worked with the data entry system every day, before the error was found. [Fritz] practiced, and was eventually able to produce Malfunction-54 himself at will. Even with this smoking gun, it took several phone calls and faxes of detailed instructions before AECL was able to obtain the same behavior on their lab machine. [Frank Borger], staff physicist for a cancer center in Chicago proved that the bug also existed in the Therac-20's software. By performing [Fritz's] procedure on his older machine, he received similar error, and a fuse in the machine would blow. The fuse was part of a hardware interlock which had been removed in the Therac-25.

As the investigations and lawsuits progressed, the software for the Therac-25 was placed under scrutiny. The Therac-25's PDP-11 was programmed completely in assembly language. Not only the application, but the underlying executive, which took the place of an operating system. The computer was tasked with handling real-time control of the machine, both its normal operation and safety systems. Today this sort of job could be handled by a microcontroller or two, with a PC running a GUI front end.

AECL never publicly released the source code, but several experts including [Nancy Leveson] did obtain access for the investigation. What they found was shocking. The software appeared to have been written by a programmer with little experience coding for real-time systems. There were few comments, and no proof that any timing analysis had been performed. According to AECL, a single programmer had written the software based upon the Therac-6 and 20 code. However, this programmer no longer worked for the company, and could not be found.

The Aftermath

The FDA declared the Therac-25 "defective". AECL issued software patches and hardware updates which eventually allowed the machine to return to service. The lawsuits were settled out of court. It seemed like the problems were solved until January 17th, 1987, when another patient was overdosed at Yakima, Washington. This problem was a new one: A counter overflow. If the operator sent a command at the exact moment the counter overflowed, the machine would skip setting up some of the beam accessories - including moving the stainless steel aiming mirror. The result was once again an unscanned beam, and an overdose. The patient died 3 months later.

It's important to note that while the software was the lynch pin in the Therac-25, it wasn't the root cause. The entire system design was the real problem. Safety-critical loads were placed upon a computer system that was not designed to control them. Timing analysis wasn't performed. Unit testing never happened. Fault trees for both hardware and software were not created. These tasks are not only the responsibility of the software engineers, but the systems engineers on the project. Therac-25 is long gone, but its legacy will live on. This was the watershed event that showed how badly things can go wrong when software for life-critical systems is not properly designed and adequately tested.

Posted in Tagged classic hacks, AECL, Radiotherapy, safety, Software Engineering, Therac-25 Featured, Slider

# data[].attributes.attachment_link.description
The Therac-25 was not a device anyone was happy to see. It was a radiation therapy machine. In layman's terms it was a "cancer zapper"; a linear accelerator with a human as its target. Using X-rays or a beam of electrons, radiation therapy machines kill cancerous tissue, even deep inside the body.

# data[].attributes.headline
אוראל 2 מימיות שליטטטה 💪

# data[].attributes.subject
אוראל 2 מימיות שליטטטה 💪

# data[].attributes.body
307877340
http://m.globes.co.il/news/m/article.aspx?did=109485

# data[].attributes.attachment_link.description
Blue screen while a public show of Windows 98 with Bill Gates.

# data[].attributes.body
https://en.m.wikipedia.org/wiki/Software_bug

# data[].attributes.body
http://mishadoff.com/blog/critical-software-bugs-part-1/http://mishadoff.com/blog/critical-software-bugs-part-1/

# data[].attributes.body
https://he.m.wikipedia.org/wiki/באג_2000

# data[].attributes.body
https://en.m.wikipedia.org/wiki/Software_bug

# data[].attributes.body
http://mishadoff.com/blog/critical-software-bugs-part-1/http://mishadoff.com/blog/critical-software-bugs-part-1/

# data[].attributes.body
306969957

# data[].attributes.body
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.htmlhttp://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html

# data[].attributes.headline
ליאור אוחיון

# data[].attributes.subject
ליאור אוחיון

# data[].attributes.body
https://www.scientificamerican.com/article/pogue-5-most-embarrassing-software-bugs-in-history/https://www.scientificamerican.com/article/pogue-5-most-embarrassing-software-bugs-in-history/

# data[].attributes.headline
עידן עזר הוא המלךךךך

# data[].attributes.subject
עידן עזר הוא המלךךךך

# data[].attributes.body
https://he.wikipedia.org/wiki/Concon
דניאל טיבי

# data[].attributes.body
עמאר
http://www.computerworld.com/article/2515483/enterprise-applications/epic-failures--11-infamous-software-bugs.html

# data[].attributes.body
http://m.ynet.co.il/Articles/3327303

# data[].attributes.body
https://he.m.wikipedia.org/wiki/Concon

# data[].attributes.body
https://he.wikipedia.org/wiki/%D7%91%D7%90%D7%92_%D7%94%D7%A4%D7%A0%D7%98%D7%99%D7%95%D7%9D
הדר שושן

# data[].attributes.body
https://he.m.wikipedia.org/wiki/באג_2000.slideshare.net/mobile/aipmm/70-26633757

# data[].attributes.body
https://he.wikipedia.org/wiki/באג_הפנטיום

# data[].attributes.body
http://tech.walla.co.il/item/2858075

# data[].attributes.body
https://www.google.co.il/amp/www.computerworlduk.com/galleries/infrastructure/top-10-software-failures-of-2014-3599618/%3Famp?client=ms-android-om-lge

# data[].attributes.body
https://en.m.wikipedia.org/wiki/Software_bug

# data[].attributes.attachment_link.description
Some software bugs have been linked to disasters. Bugs in code that controls the Therac-25 radiation therapy machine were directly responsible for patient deaths in the 1980s. In 1996, the European Space Agency's US$1 billion prototype Ariane 5 rocket had to be destroyed less than a minute after launch due to a bug in the on-board guidance computer program.